All AutosheetBench tasks

Pivot sales by region and product, ranked

Pivot tablesMediumLLM judge scoringLatest run

Passed by 10 of 12 products on their best model. Failed: Gemini in Sheets and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

In a new Pivot sheet at A3, create a pivot table of total sales by Region and Product ranked by sales for both.

Data

Starting sheet
ABCD
1RegionProductSalesDate
2EastGamma7002026-01-05
3EastAlpha1502026-01-12
4EastBeta1002026-01-20
5NorthAlpha5002026-01-07
6NorthBeta2002026-01-11
7NorthAlpha1002026-01-18
8SouthGamma4002026-01-03
9SouthAlpha3002026-01-09
10SouthBeta502026-01-25
11WestBeta2602026-01-04
12WestAlpha2502026-01-13
13WestGamma402026-01-27

Pivot

Starting sheet
A
1Create pivot table here

Expected result

How each AI configuration did

18 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A real pivot table on Pivot!A3
  • Sales summed by region, then product
  • Both levels sorted by sales, highest first
  • Region subtotals and a grand total

Passed 18

Autosheet in Excel Opus 5.5 · Medium

Passed43s · 10¢

Autosheet in Excel GPT-6 Astra · Low

Passed1m 15s · 50¢

Autosheet in Sheets Opus 5.5 · Medium

Passed32s · 8¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed24s · 6¢

Claude in Excel Opus 5.5 · High

Passed26s · 15¢

Autosheet in Excel Haiku 5.5 · High

Passed34s · 0.6¢

Claude in Sheets Opus 5.5 · Medium

Passed33s · 7¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed49s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed1m 35s · 37¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed1m 30s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed55s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed50s · 0.4¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed2m 35s · $1.54

Claude in Excel Haiku 5.5 · High

Passed13s · 0.5¢

Autosheet in Excel GPT-6 Luna · High

Passed1m 41s · 2¢

Copilot in Excel Auto

Passed1m 35s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed55s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed4m 55s · 7¢

Failed 4

Claude in Sheets Haiku 5.5 · High

Failed1m 41s · 3¢

Why it failed: Within regions, products are not sorted by sales descending; they remain sorted alphabetically.

Gemini in Sheets Auto

Failed2m 41s · N/A

Why it failed: Product was placed in columns rather than as the second row level under Region.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed1m 15s · 66¢

Why it failed: A manual summary table was created instead of a real pivot table.

ChatGPT in Sheets GPT-6 Luna · High

Failed2m 37s · 1¢

Why it failed: No pivot table: the summary is built with formulas instead.