All AutosheetBench tasks

Group rows by category

Spreadsheet manipulationHardDeterministic scoringLatest run

Passed by 9 of 12 products on their best model. Failed: Copilot in Excel, Gemini in Sheets and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Group the rows by category (alphabetical order) so I can collapse them.

Starting spreadsheet

Sheet1
ABC
1CategoryItemAmount
2TravelFlights1200
3OfficeSupplies150
4MarketingAds900
5TravelHotel800
6OfficeSoftware600
7MarketingEvents1100
8TravelMeals300

A passing output

Autosheet in Excel · Opus 5.5

How each AI configuration did

11 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked automatically.
  • Rows are grouped into 3 collapsible categories

Passed 11

Autosheet in Excel Opus 5.5 · Medium

Passed18s · 5¢

Autosheet in Excel GPT-6 Astra · Low

Passed12s · 7¢

Autosheet in Sheets Opus 5.5 · Medium

Passed31s · 7¢

Claude in Excel Opus 5.5 · High

Passed22s · 9¢

Claude in Sheets Opus 5.5 · Medium

Passed31s · 8¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed40s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed30s · 4¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed59s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed1m 3s · N/A

ChatGPT in Excel GPT-6 Astra · Medium

Passed59s · 55¢

ChatGPT + Drive MCP GPT-6 · Medium

Passed35s · N/A

Failed 11

Autosheet in Excel Sonnet 5.5 · Medium

Failed5s · 1¢

Why it failed: The rows aren't grouped by category.

Autosheet in Excel Haiku 5.5 · High

Failed20s · 0.2¢

Why it failed: The rows aren't grouped by category.

Autosheet in Sheets Haiku 5.5 · High

Failed22s · 0.2¢

Why it failed: The rows aren't grouped by category.

Claude in Excel Haiku 5.5 · High

Failed12s · 0.4¢

Why it failed: The rows aren't grouped by category.

Claude in Sheets Haiku 5.5 · High

Failed39s · 1¢

Why it failed: The rows aren't grouped by category.

Autosheet in Excel GPT-6 Luna · High

Failed16s · 0.3¢

Why it failed: The rows aren't grouped by category.

Copilot in Excel Auto

Failed18s · N/A

Why it failed: The rows aren't grouped by category.

ChatGPT in Excel GPT-6 Luna · High

Failed33s · 0.5¢

Why it failed: The rows aren't grouped by category.

Gemini in Sheets Auto

Failed3m 7s · N/A

Why it failed: The rows aren't grouped by category.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed24s · 33¢

Why it failed: The rows aren't grouped by category.

ChatGPT in Sheets GPT-6 Luna · High

Failed25s · 0.4¢

Why it failed: The rows aren't grouped by category.