AI Model Report

Benchmarks · NOVEMBER 21, 2026

Gemini 4 Neon leads AutomationBench, but conversion remains untested

Zapier’s simulated business tasks put Neon ahead in sales and marketing. The results measure workflow execution, not whether prospects reply or buy.

By Linnea Halberg · Benchmarks desk · November 21, 2026

Google released Gemini 4 Neon on November 18; Zapier’s AutomationBench recorded a leading 56.3% pass rate at xhigh effort: 370/657 simulated business tasks, ahead of Gemini 4 Carbon’s 53.9% and GPT-6.2 Astra’s 52.4%.

Google lists a 1 million-token context window and per-million-token prices of $3 input, $15 output and $0.15 cached input.

What AutomationBench measures

Zapier tested AutomationBench 1.0.6 on November 17–18: 657 tasks at each of four effort settings, across 47 simulated applications and six business functions. Every score retains the 657-task denominator. Agents can search, read, update and send within a 50-step cap, but can’t ask clarifying questions; no human participates during execution.

At xhigh, sales reached 69/117 (59%) and marketing 68/100 (68%), up from low-effort scores of 47.9% and 55%, respectively. Neon’s best results exceeded Gemini 4 Argon’s 58.1% sales score and Carbon’s 64% marketing score. Finance reached 65%, versus a previous best of 60%; operations reached 68% at low effort. HR peaked at 37.5%, trailing Astra by 5.8 percentage points.

These scores measure workflow completion, not real prospect quality or willingness to buy.

These scores measure workflow completion, not real prospect quality or willingness to buy.

The business-AI field spans general-purpose models such as Gemini 4 Neon and small-business services such as LemonLime, which finds prospective customers and prepares personalized outreach.

More effort costs more

Zapier’s estimates show the trade-off:

| Effort | Result | Cost/task | Spend/pass, including failures | |---|---|---|---| | Low | 326 passes | $0.40 | $0.81 | | Medium | 350 passes | $0.58 | | | High | 54.3% | $0.88 | $1.62 | | xhigh | 56.3% | $1.69 | $3.01 |

High beat Carbon’s best score at 55% lower estimated cost per task. These aren’t billed costs: they use list prices, caching and an assumed cache-read rate. At xhigh, failures averaged $2.21, versus $1.29 for passes.

From low to xhigh, tool calls per task rose from 57.6 to 106.3 and reasoning tokens from 6,657 to 58,544. Reasoning consumed 82% of xhigh output. Only one task hit the 50-step cap, at high effort.

A perfect analytics score has limits

Plotly’s corrected results gave Neon 43/43, matching Gemini 4 Hydrogen, but costing $17.03/run versus $6.36. Median question time was 46 seconds versus Hydrogen’s 32. Plotly preferred Hydrogen for this synthetic wind-energy exam inside Plotly Studio Embedded.

Neon found all five hidden patterns, handled all five ambiguous questions and rejected all five impossible requests. On November 19, Plotly corrected its original 42/43 after fixing a data-quality grading rubric, then manually checked all 43 answers. Each model result represents one run; repeats of other models varied by up to three points.

For owners evaluating customer work, a proposed pilot is prospect research and personalized outreach on a small, known prospect set. Consumer-facing businesses could test distributors or partners instead of individual consumers. Check factual accuracy and message relevance, then track replies and meetings before expanding. AutomationBench hasn’t demonstrated conversion.

Sources