All AutosheetBench tasks

Chart revenue by quarter

ChartsEasyLLM judge scoringLatest run

Passed by 11 of 12 products on their best model. Failed: ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Create a bar chart showing revenue by quarter

Starting spreadsheet

Sheet1
ABCD
1QuarterRevenueExpensesProfit
2Q1 20251250008900036000
3Q2 20251420009100051000
4Q3 20251380009500043000
5Q4 20251670009800069000

Expected result

How each AI configuration did

20 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A bar or column chart of the quarterly data
  • Revenue is the plotted series

Passed 20

Autosheet in Excel Opus 5.5 · Medium

Passed15s · 5¢

Autosheet in Excel GPT-6 Astra · Low

Passed8s · 7¢

Autosheet in Sheets Opus 5.5 · Medium

Passed16s · 4¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed7s · 2¢

Claude in Excel Opus 5.5 · High

Passed17s · 17¢

Autosheet in Excel Haiku 5.5 · High

Passed8s · 0.1¢

Claude in Sheets Opus 5.5 · Medium

Passed18s · 4¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed31s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed14s · 3¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed33s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed56s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed18s · 0.4¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed13s · 39¢

Claude in Excel Haiku 5.5 · High

Passed21s · 0.9¢

Claude in Sheets Haiku 5.5 · High

Passed27s · 1¢

Autosheet in Excel GPT-6 Luna · High

Passed18s · 0.3¢

Copilot in Excel Auto

Passed39s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed26s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed28s · 0.5¢

Gemini in Sheets Auto

Passed20s · N/A

Failed 2

ChatGPT in Sheets GPT-6 Astra · Medium

Failed41s · 52¢

Why it failed: No chart object was created in the workbook.

ChatGPT in Sheets GPT-6 Luna · High

Failed23s · 0.4¢

Why it failed: No chart was created.