All AutosheetBench tasks

Pivot cumulative revenue over time

Pivot tablesHardLLM judge scoringLatest run

Passed by 6 of 12 products on their best model. Failed: Autosheet in Sheets, Claude Code + Sheets MCP, Claude in Sheets, ChatGPT + Autosheet MCP, ChatGPT + Drive MCP and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

In a new Pivot sheet at A3, create a pivot table showing cumulative revenue over time.

Data

Starting sheet
AB
1DateRevenue
22025-01-174200
32025-01-032500
42025-01-256100
52025-01-083100
62025-01-124700
72025-01-215200

Pivot

Starting sheet
A
1

Expected result

How each AI configuration did

13 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A real pivot table on Pivot!A3
  • Dates in ascending order
  • A correct running total, a field of the pivot itself

Passed 13

Autosheet in Excel Opus 5.5 · Medium

Passed14s · 5¢

Autosheet in Excel GPT-6 Astra · Low

Passed20s · 16¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed12s · 3¢

Claude in Excel Opus 5.5 · High

Passed1m 8s · 42¢

Autosheet in Excel Haiku 5.5 · High

Passed16s · 0.3¢

Autosheet in Excel GPT-6.1 Sol · Medium

Passed29s · 5¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed1m 2s · N/A

ChatGPT in Excel GPT-6 Astra · Medium

Passed45s · 68¢

Claude in Excel Haiku 5.5 · High

Passed44s · 2¢

Claude in Sheets Haiku 5.5 · High

Passed1m 13s · 2¢

Autosheet in Excel GPT-6 Luna · High

Passed58s · 1¢

Copilot in Excel Auto

Passed46s · N/A

Gemini in Sheets Auto

Passed2m 50s · N/A

Failed 9

Autosheet in Sheets Opus 5.5 · Medium

Failed24s · 6¢

Why it failed: The pivot table itself only shows non-cumulative Sum of Revenue (2500, 3100, 4700, etc.), not the running total over time.

Claude in Sheets Opus 5.5 · Medium

Failed33s · 7¢

Why it failed: The pivot table itself only displays regular sum of revenue ($2,500, $3,100, etc.), not cumulative revenue.

Claude Code + Sheets MCP Opus 5.5 · Medium

Failed43s · N/A

Why it failed: The pivot table itself does not show running total revenue; its value field only shows discrete revenue amounts (2500, 3100, etc.).

ChatGPT + Autosheet MCP GPT-6 · Medium

Failed1m 16s · N/A

Why it failed: The pivot table itself does not show running total revenue; it only shows regular daily revenue in column B.

Autosheet in Sheets Haiku 5.5 · High

Failed29s · 0.3¢

Why it failed: The pivot table itself only displays regular sum of revenue per date, not the running total.

ChatGPT + Drive MCP GPT-6 · Medium

Failed45s · N/A

Why it failed: The pivot table itself does not show running total; it only shows regular Sum of Revenue.

ChatGPT in Excel GPT-6 Luna · High

Failed2m 57s · 2¢

Why it failed: The pivot table itself does not show cumulative revenue; the pivot contains only standard 'Sum of Revenue' values in column B.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed47s · 54¢

Why it failed: A real pivot table was not created; a formula-based summary table was built instead.

ChatGPT in Sheets GPT-6 Luna · High

Failed1m 17s · 0.7¢

Why it failed: No pivot table: the summary is built with formulas instead.