AI Model Report

Benchmarks · NOVEMBER 20, 2026

Gemini 4 Neon scores 56.3% on business tasks—not customer growth

Zapier’s simulated sales and marketing tasks make Neon worth a bounded pilot, but correction time and total cost matter more than the headline rank.

By Linnea Halberg · Benchmarks desk · November 20, 2026

Google began rolling out Gemini 4 Neon November 18; Zapier reports its xhigh setting passed 370 of 657 simulated business tasks, or 56.3%, ahead of Carbon’s 53.9% and GPT-6.2 Astra’s 52.4%.

That’s workflow execution, not customer growth. The result doesn’t establish reliability on a business’s own data.

What AutomationBench measures

Zapier ran AutomationBench 1.0.6 November 17–18: 657 tasks at each of four effort levels, a 50-step cap, 47 simulated apps and six business functions. Agents receive no clarification or human intervention; passing requires every deterministic assertion about the resulting state to hold. CRM records, inbox threads and spreadsheets contain traps including stale rows and near-duplicate names.

At xhigh, Neon passed 68 of 100 marketing tasks and 69 of 117 sales tasks; low effort passed 55 and 56 respectively. Zapier also lists Neon’s best marketing and sales results as 68.0% and 59.0%, beating Carbon’s 64.0% and Argon’s 58.1%.

Roboflow’s separate November 19 Vision Evals snapshot puts Neon first among 73 models at 91.0% overall: first in counting, OCR and reasoning, eighth in identification. Cost was $0.022 per sample; time was 20.7 seconds.

Higher effort, higher failure costs

Zapier’s effort comparison:

| Effort | Result | Estimated cost/task | |---|---:|---:| | Low | 326 passed | $0.40 | | Medium | 350 passed | $0.58 | | High | 54.3% | $0.88 | | xhigh | 56.3% | $1.69 |

Including failures, estimated spend per passed task rises from $1.62 at high to $3.01 at xhigh. Failed xhigh tasks averaged $2.21; passes averaged $1.29.

From low to xhigh, tool calls increased from 57.6 to 106.3 per task; reasoning tokens rose from 6,657 to 58,544, accounting for 82% of xhigh output. Zapier bills reasoning as output in its estimates. Its Carbon comparisons also use a different cache-read basis from the public leaderboard.

Google lists per-million-token API rates of $3 input, $0.15 cached input and $15 output. Google DeepMind’s Gemini 4 Neon model card specifies up to 1 million combined input-output tokens, up to 64,000 default synchronous output tokens, a March 2026 knowledge cutoff and hallucination risk.

Compare workloads before committing

Artificial Analysis’s Gemini 4 Neon Intelligence Index evaluation measures a different workload: xhigh costs $6.44 per task and takes roughly 21 minutes versus Carbon’s 36. Its roughly 17% saving uses Carbon’s standard pricing. Carbon’s temporary $2/$10 input/output rates will run through November 30, below Neon’s $3/$15.

The Astra and Sol pricing and coding benchmark analysis offers another comparison, not a substitute for testing.

"Test whether Neon produces usable customer-acquisition work—not whether benchmark completion forecasts sales."

For a small-team owner, that suggests one bounded pilot: campaign variants or prospect-record updates, with human review. Compare usable outputs, correction time and total cost against the current process before expanding. Don’t treat completion as sales; measure customer-growth outcomes separately.

Sources