All AutosheetBench tasks

Pivot revenue, cost and margin

Pivot tablesMediumLLM judge scoringLatest run

Passed by 8 of 12 products on their best model. Failed: Autosheet in Excel, Claude in Excel, ChatGPT in Excel and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

In a new Pivot sheet at A3, create a pivot table with revenue, cost, and profit margin percentage.

Data

Starting sheet
ABC
1ProductRevenueCost
2Alpha1820012400
3Beta134009100
4Gamma99006200
5Delta2210014600
6Epsilon118007800

Pivot

Starting sheet
A
1

Expected result

How each AI configuration did

11 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A real pivot table on Pivot!A3
  • Products as rows, revenue and cost summed
  • Margin % as a field inside the pivot

Passed 11

Autosheet in Sheets Opus 5.5 · Medium

Passed15s · 4¢

Autosheet in Excel Haiku 5.5 · High

Passed25s · 0.4¢

Claude in Sheets Opus 5.5 · Medium

Passed30s · 6¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed43s · N/A

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed52s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed1m 1s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed42s · 0.4¢

Copilot in Excel Auto

Passed44s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed32s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed2m 3s · 4¢

Gemini in Sheets Auto

Passed2m 4s · N/A

Failed 11

Autosheet in Excel Opus 5.5 · Medium

Failed31s · 10¢

Why it failed: Profit margin percentage is an external worksheet formula column outside the pivot definition.

Autosheet in Excel GPT-6 Astra · Low

Failed24s · 18¢

Why it failed: Profit margin percentage was placed in column D outside the pivot table using worksheet formulas (=IFERROR((B4-C4)/B4,0)) instead of being included as a value field inside the pivot table.

Autosheet in Excel Sonnet 5.5 · Medium

Failed15s · 3¢

Why it failed: Profit margin percentage was not included as a value field inside the pivot table; it was placed in column E outside the pivot range (A3:D9) using regular worksheet formulas.

Claude in Excel Opus 5.5 · High

Failed31s · 16¢

Why it failed: The margin sits in cells beside the pivot, computed with GETPIVOTDATA formulas, not as a field of the pivot.

Autosheet in Excel GPT-6.1 Sol · Medium

Failed42s · 7¢

Why it failed: Profit margin percentage was placed in column D outside the pivot table using worksheet formulas (=IFERROR((B4-C4)/B4,0)) rather than as a calculated field inside the pivot table.

ChatGPT in Excel GPT-6 Astra · Medium

Failed1m 2s · 66¢

Why it failed: The margin sits in formulas beside the pivot, in column D, not as a field of the pivot.

Claude in Excel Haiku 5.5 · High

Failed16s · 0.5¢

Why it failed: Profit margin percentage was written as worksheet formulas in column D (D3:D9) outside the pivot table rather than being included as a value field inside the pivot table.

Claude in Sheets Haiku 5.5 · High

Failed1m 7s · 2¢

Why it failed: Profit margin percentage was added as external worksheet formulas in column D (cells D4:D9) outside the pivot table rather than as a value field inside the pivot itself.

Autosheet in Excel GPT-6 Luna · High

Failed1m 18s · 6¢

Why it failed: Profit margin percentage was placed in column D outside the pivot table as regular worksheet formulas, directly violating the requirement that it be included as a value field inside the…

ChatGPT in Sheets GPT-6 Astra · Medium

Failed48s · 58¢

Why it failed: A real pivot table was not created; a manual summary table using SUMIF formulas was built instead.

ChatGPT in Sheets GPT-6 Luna · High

Failed1m · 0.7¢

Why it failed: No pivot table: the summary is built with SUMIF formulas instead.