All AutosheetBench tasks

Chart traffic sources, then make it a pie

ChartsMediumLLM judge scoringLatest run

Passed by 9 of 12 products on their best model. Failed: ChatGPT in Excel, Copilot in Excel and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Create a bar chart showing where our traffic is coming from

Can you make it a pie chart instead? The bar chart is hard to read with this many sources

Starting spreadsheet

Sheet1
ABCD
1SourceSessionsBounce RateAvg Duration
2Google Organic1842042%3:24
3Direct1185038%4:01
4Facebook734061%1:52
5Email521029%5:17
6Bing388047%2:45
7LinkedIn264033%4:38
8Twitter/X192068%1:14
9Instagram138072%0:58
10Reddit89055%2:09
11Pinterest45064%1:31

Expected result

How each AI configuration did

17 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • First a bar chart of sessions by source
  • Then the same chart as a pie
  • The pie keeps the same data
  • A readable pie: filled slices in distinct colors, each source named, and any percentages right

Passed 17

Autosheet in Excel Opus 5.5 · Medium

Passed26s · 10¢

Autosheet in Excel GPT-6 Astra · Low

Passed17s · 18¢

Autosheet in Sheets Opus 5.5 · Medium

Passed27s · 10¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed14s · 4¢

Claude in Excel Opus 5.5 · High

Passed52s · 31¢

Autosheet in Excel Haiku 5.5 · High

Passed17s · 0.3¢

Claude in Sheets Opus 5.5 · Medium

Passed40s · 13¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed50s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed30s · 6¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed2m 51s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed1m 16s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed36s · 0.4¢

Claude in Excel Haiku 5.5 · High

Passed26s · 1¢

Claude in Sheets Haiku 5.5 · High

Passed39s · 1¢

ChatGPT + Drive MCP GPT-6 · Medium

Passed58s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed44s · 1¢

Gemini in Sheets Auto

Passed1m 29s · N/A

Failed 5

ChatGPT in Excel GPT-6 Astra · Medium

Failed42s · 95¢

Why it failed: Every slice has no fill and no outline, so the pie is invisible and the legend shows no colors. The labels' number format, 0\.0%, also reads Google Organic's 34% share as 3.4%.

Autosheet in Excel GPT-6 Luna · High

Failed36s · 0.8¢

Why it failed: The pie has no legend and no labels: nothing says which slice is which source.

Copilot in Excel Auto

Failed44s · N/A

Why it failed: Every slice is the same blue, so the legend can't tell the sources apart.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed55s · 85¢

Why it failed: No pie chart was created after turn 2.

ChatGPT in Sheets GPT-6 Luna · High

Failed1m 41s · 1¢

Why it failed: No chart was created.