All AutosheetBench tasks

Turn a chart into a horizontal bar chart

ChartsMediumLLM judge scoringLatest run

Passed by 10 of 12 products on their best model. Failed: ChatGPT + Drive MCP and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Make this a horizontal bar chart instead.

Starting spreadsheet

Sheet1
ABCDEFGHIJKLM
1YearRevenueProfit
220211250000180000
320221430000215000
420231590000248000
520241710000286000
620251880000324000
7
8
9
10
11
12
13
14
15
16
17

Expected result

How each AI configuration did

19 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • The chart is edited in place, colors kept
  • It is now a horizontal bar chart

Passed 19

Autosheet in Excel Opus 5.5 · Medium

Passed10s · 3¢

Autosheet in Excel GPT-6 Astra · Low

Passed10s · 9¢

Autosheet in Sheets Opus 5.5 · Medium

Passed19s · 12¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed5s · 2¢

Claude in Excel Opus 5.5 · High

Passed14s · 8¢

Autosheet in Excel Haiku 5.5 · High

Passed10s · 0.2¢

Claude in Sheets Opus 5.5 · Medium

Passed21s · 6¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed25s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed27s · 3¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed31s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed34s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed20s · 0.4¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed19s · 35¢

Claude in Excel Haiku 5.5 · High

Passed13s · 0.5¢

Claude in Sheets Haiku 5.5 · High

Passed35s · 1¢

Autosheet in Excel GPT-6 Luna · High

Passed20s · 0.3¢

Copilot in Excel Auto

Passed25s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed10s · 0.4¢

Gemini in Sheets Auto

Passed38s · N/A

Failed 3

ChatGPT + Drive MCP GPT-6 · Medium

Failed28s · N/A

Why it failed: While the chart was replaced cleanly with only one chart remaining on the sheet, the original series colors were not retained: Revenue and Profit do not match their original theme colors…

ChatGPT in Sheets GPT-6 Astra · Medium

Failed17s · 29¢

Why it failed: The existing chart was not updated with the requested modifications.

ChatGPT in Sheets GPT-6 Luna · High

Failed17s · 0.3¢

Why it failed: The chart was not updated or modified; the agent took no action on the workbook.