All AutosheetBench tasks

Make a pie chart of revenue by year

ChartsHardLLM judge scoringLatest run

Passed by 1 of 12 products on their best model: Copilot in Excel.

Prompt and starting spreadsheet

Task files

Prompt

Create a pie chart of revenue by year.

Starting spreadsheet

Data
AB
1YearRevenue
2202050000
3202070000
4202160000
5202175000
6202265000
7202277000
8202372000
9202386000
10202481000
11202490000
12202592000
13202597000

Expected result

How each AI configuration did

2 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A pie chart with one slice per year, sized by revenue
  • Each year is identified
  • Each year's revenue amount is shown
  • A clear title and readable labels

Passed 2

Autosheet in Excel Haiku 5.5 · High

Passed34s · 0.5¢

Copilot in Excel Auto

Passed1m 32s · N/A

Failed 20

Autosheet in Excel Opus 5.5 · Medium

Failed24s · 7¢

Why it failed: The chart displays only percentage labels (13%, 15%, 17%, etc.) and does not display the actual aggregated revenue amounts on the chart.

Autosheet in Excel GPT-6 Astra · Low

Failed48s · 29¢

Why it failed: Only percentages are shown on the slices; actual revenue dollar values are missing.

Autosheet in Sheets Opus 5.5 · Medium

Failed39s · 21¢

Why it failed: Labels show percentages rather than aggregated revenue amounts.

Autosheet in Excel Sonnet 5.5 · Medium

Failed9s · 5¢

Why it failed: No distinct pie slices are rendered because all aggregated revenue values evaluated to $0.

Claude in Excel Opus 5.5 · High

Failed23s · 16¢

Why it failed: The slices show only their percentages, not each year's revenue, which a pie of revenue by year should show.

Claude in Sheets Opus 5.5 · Medium

Failed44s · 10¢

Why it failed: Only percentage values are visible on the slices; actual revenue figures are missing.

Claude Code + Sheets MCP Opus 5.5 · Medium

Failed51s · N/A

Why it failed: Aggregated revenue amounts are not shown on or alongside the slices.

Autosheet in Excel GPT-6.1 Sol · Medium

Failed39s · 7¢

Why it failed: The chart shows percentages (e.g., '13%', '15%') rather than actual revenue amounts (e.g., '$120,000', '120000').

Claude Code + Autosheet MCP Opus 5.5 · Medium

Failed1m 46s · N/A

Why it failed: Labels display only percentage shares rather than the actual aggregated revenue values.

ChatGPT + Autosheet MCP GPT-6 · Medium

Failed55s · N/A

Why it failed: Data labels show percentages only; actual aggregated revenue amounts are omitted.

Autosheet in Sheets Haiku 5.5 · High

Failed1m 6s · 0.6¢

Why it failed: No revenue amounts or data labels are visible on the chart.

ChatGPT in Excel GPT-6 Astra · Medium

Failed59s · 85¢

Why it failed: The slices show only their percentages, not each year's revenue, which a pie of revenue by year should show.

Claude in Excel Haiku 5.5 · High

Failed27s · 2¢

Why it failed: Labels display only percentage shares rather than actual revenue amounts.

Claude in Sheets Haiku 5.5 · High

Failed1m 10s · 2¢

Why it failed: The chart displays only percentages on the slices; the actual aggregated revenue amounts are not visible on the chart.

Autosheet in Excel GPT-6 Luna · High

Failed50s · 0.9¢

Why it failed: Slices are identical in size (~16-17% each) rather than reflecting revenue values.

ChatGPT + Drive MCP GPT-6 · Medium

Failed40s · N/A

Why it failed: Actual aggregated revenue values are not shown on the chart.

ChatGPT in Excel GPT-6 Luna · High

Failed1m 36s · 1¢

Why it failed: Slices are identical in size rather than proportional to aggregated revenue.

Gemini in Sheets Auto

Failed27s · N/A

Why it failed: Revenue amounts are not displayed on the chart slices or in the legend.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed46s · 56¢

Why it failed: No pie chart was created in the workbook.

ChatGPT in Sheets GPT-6 Luna · High

Failed51s · 0.5¢

Why it failed: No pie chart was created in the workbook.