All AutosheetBench tasks

Build an e-commerce KPI dashboard

ChartsMediumLLM judge scoringLatest run

Passed by 9 of 12 products on their best model. Failed: ChatGPT + Drive MCP, Gemini in Sheets and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

In a new Dashboard sheet, build me a dashboard with charts for orders, revenue, and customer count — I need to see the trends at a glance

Add a chart showing the return rate trend too

Starting spreadsheet

Sheet1
ABCDEF
1MonthOrdersRevenueReturnsAvg Order ValueCustomer Count
2Jan 20251240873006270.40980
3Feb 20251380961007169.641050
4Mar 202515201098008972.241180
5Apr 202514101012007471.771120
6May 202516501184009271.761290
7Jun 2025178013170010373.991380
8Jul 202516201195009873.771250
9Aug 202515901158008872.831230
10Sep 2025187014020010874.971440
11Oct 2025201015460011976.921560
12Nov 2025234018470014278.931810
13Dec 2025268021740015881.122070

Expected result

How each AI configuration did

16 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • At least 2 trend charts of the monthly KPIs
  • No pie charts for the monthly data
  • A chart of the return rate

Passed 16

Autosheet in Excel Opus 5.5 · Medium

Passed59s · 24¢

Autosheet in Excel GPT-6 Astra · Low

Passed43s · 36¢

Autosheet in Sheets Opus 5.5 · Medium

Passed1m 28s · 30¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed21s · 6¢

Claude in Excel Opus 5.5 · High

Passed1m 37s · 49¢

Autosheet in Excel Haiku 5.5 · High

Passed30s · 0.6¢

Claude in Sheets Opus 5.5 · Medium

Passed1m 9s · 23¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed3m 6s · N/A

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed2m 58s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed3m 2s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed1m 37s · 1¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed2m 43s · $1.65

Claude in Excel Haiku 5.5 · High

Passed55s · 2¢

Claude in Sheets Haiku 5.5 · High

Passed2m 20s · 3¢

Copilot in Excel Auto

Passed2m 37s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed3m 24s · 3¢

Failed 6

Autosheet in Excel GPT-6.1 Sol · Medium

Failed1m 10s · 15¢

Why it failed: Dashboard KPI cards contain visible hash overflow (###) for primary metrics.

Autosheet in Excel GPT-6 Luna · High

Failed1m 46s · 2¢

Why it failed: Document 1 shows pervasive hash overflow (###) across data columns B, E, and H, clipped column headers ('IV Ord', 'IV Rev', 'IV Cust', 'IV Return R'), and clipped chart titles due to…

ChatGPT + Drive MCP GPT-6 · Medium

Failed1m 32s · N/A

Why it failed: The 'TOTAL REVENUE' KPI card is materially clipped, showing '$1,576,7' instead of '$1,576,700', which conceals numbers and makes the value misleading/unreadable.

Gemini in Sheets Auto

Failed5m · N/A

Why it failed: Still working at the 5-minute time limit. No chart for returns or return rate was created; the four charts plot Revenue, Orders, Customer Count, and Average Order Value.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed2m 25s · $1.54

Why it failed: The dashboard has no charts.

ChatGPT in Sheets GPT-6 Luna · High

Failed3m 28s · 2¢

Why it failed: The dashboard has no charts.