All AutosheetBench tasks

Combine revenue bars with a rate line

ChartsMediumLLM judge scoringLatest run

Passed by 8 of 12 products on their best model. Failed: Claude in Excel, ChatGPT in Excel, Gemini in Sheets and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Create a chart with Revenue as bars and conversion rate as a line

Data

Starting sheet · rows 1–21 of 25
ABC
1DateRevenueConversion Rate
22025-02-01280000.018
32025-02-02294500.0214
42025-02-03309000.0248
52025-02-04323500.0282
62025-02-05324000.0316
72025-02-06338500.02
82025-02-07353000.0234
92025-02-08367500.0268
102025-02-09368000.0302
112025-02-10382500.0336
122025-02-11397000.022
132025-02-12411500.0254
142025-02-13412000.0288
152025-02-14426500.0322
162025-02-15441000.0356
172025-02-16455500.024
182025-02-17456000.0274
192025-02-18470500.0308
202025-02-19485000.0342
212025-02-20499500.0376

Chart

Starting sheet
A
1

Expected result

How each AI configuration did

15 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A combo chart: revenue as bars, conversion rate as a line
  • Two y-axes
  • The rate axis reads about 3%, not 0.3%
  • Dates in order, a title and a legend

Passed 15

Autosheet in Excel Opus 5.5 · Medium

Passed19s · 6¢

Autosheet in Excel GPT-6 Astra · Low

Passed11s · 9¢

Autosheet in Sheets Opus 5.5 · Medium

Passed20s · 7¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed8s · 2¢

Autosheet in Excel Haiku 5.5 · High

Passed12s · 0.2¢

Claude in Sheets Opus 5.5 · Medium

Passed27s · 7¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed33s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed21s · 3¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed55s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed59s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed27s · 0.3¢

Autosheet in Excel GPT-6 Luna · High

Passed1m 5s · 1¢

Copilot in Excel Auto

Passed47s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed34s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed55s · 0.8¢

Failed 7

Claude in Excel Opus 5.5 · High

Failed35s · 20¢

Why it failed: The conversion-rate axis has a broken number format, 0\.0%: rates of 1.8% to 3.8% read as 0.2% to 0.4%.

ChatGPT in Excel GPT-6 Astra · Medium

Failed38s · 66¢

Why it failed: The conversion-rate axis has a broken number format, 0\.0%: rates of 1.8% to 3.8% read as 0.2% to 0.4%.

Claude in Excel Haiku 5.5 · High

Failed33s · 1¢

Why it failed: Secondary axis shows values around 0.03% instead of 3%.

Claude in Sheets Haiku 5.5 · High

Failed29s · 1¢

Why it failed: Title is present, but the legend swatches lack series text labels.

Gemini in Sheets Auto

Failed27s · N/A

Why it failed: Conversion rate axis uses decimal values instead of percentage formatting.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed42s · 53¢

Why it failed: No combo chart was created.

ChatGPT in Sheets GPT-6 Luna · High

Failed24s · 0.4¢

Why it failed: No combo chart was created.