All AutosheetBench tasks

Pivot the top 5 products by revenue

Pivot tablesEasyLLM judge scoringLatest run

Passed by 11 of 12 products on their best model. Failed: ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

In a new Pivot sheet at A3, create a pivot with only the top 5 products by revenue.

Data

Starting sheet
ABC
1ProductRevenueCategory
2Alpha18200A
3Beta13400A
4Gamma9900B
5Delta22100B
6Epsilon11800C
7Zeta7600C
8Eta19800A
9Theta8600B
10Iota17400B
11Kappa9200C

Pivot

Starting sheet
A
1

Expected result

How each AI configuration did

20 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A real pivot table on Pivot!A3
  • Revenue summed by product
  • Filtered to the top 5 products

Passed 20

Autosheet in Excel Opus 5.5 · Medium

Passed20s · 7¢

Autosheet in Excel GPT-6 Astra · Low

Passed24s · 17¢

Autosheet in Sheets Opus 5.5 · Medium

Passed52s · 11¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed16s · 4¢

Claude in Excel Opus 5.5 · High

Passed17s · 10¢

Autosheet in Excel Haiku 5.5 · High

Passed45s · 0.8¢

Claude in Sheets Opus 5.5 · Medium

Passed50s · 14¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed1m 5s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed44s · 10¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed1m 16s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed55s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed37s · 0.4¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed1m 22s · 69¢

Claude in Excel Haiku 5.5 · High

Passed24s · 0.9¢

Claude in Sheets Haiku 5.5 · High

Passed1m 39s · 4¢

Autosheet in Excel GPT-6 Luna · High

Passed1m 1s · 1¢

Copilot in Excel Auto

Passed1m 11s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed56s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed1m 1s · 0.7¢

Gemini in Sheets Auto

Passed1m 46s · N/A

Failed 2

ChatGPT in Sheets GPT-6 Astra · Medium

Failed40s · 48¢

Why it failed: A real pivot table was not created; a formula-based table using INDEX/MATCH and LARGE was created instead.

ChatGPT in Sheets GPT-6 Luna · High

Failed38s · 0.4¢

Why it failed: No pivot table: the summary is built with formulas instead.