All AutosheetBench tasks

Add seniority columns in the table's format

FormattingMediumLLM judge scoringLatest run

Passed by 9 of 12 products on their best model. Failed: ChatGPT in Excel, ChatGPT + Autosheet MCP and ChatGPT in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Calculate employee seniority and year of beginning

Starting spreadsheet

Generate and autofill formulas
A
1Employee start date
22007-03-15
32012-08-22
42020-01-10
52003-11-05
62017-06-18
72001-09-27
82023-04-02
92010-12-14
102015-07-30
112005-02-19

A passing output

Autosheet in Excel · Opus 5.5

How each AI configuration did

12 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • Columns B and C hold seniority and start year
  • They're formatted like column A

Passed 12

Autosheet in Excel Opus 5.5 · Medium

Passed14s · 3¢

Autosheet in Sheets Opus 5.5 · Medium

Passed26s · 15¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed7s · 2¢

Claude in Excel Opus 5.5 · High

Passed21s · 13¢

Claude in Sheets Opus 5.5 · Medium

Passed31s · 6¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed37s · N/A

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed38s · N/A

Claude in Excel Haiku 5.5 · High

Passed20s · 0.6¢

Claude in Sheets Haiku 5.5 · High

Passed31s · 0.5¢

Copilot in Excel Auto

Passed18s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed36s · N/A

Gemini in Sheets Auto

Passed1m 32s · N/A

Failed 10

Autosheet in Excel GPT-6 Astra · Low

Failed19s · 19¢

Why it failed: Columns B and C headers do not match column A: cells B1 and C1 have black font on a light blue background rather than white font on a dark blue background.

Autosheet in Excel Haiku 5.5 · High

Failed13s · 0.2¢

Why it failed: Columns B and C were not formatted to match column A; specifically, B1 and C1 headers have default black text on white fill instead of white text on dark blue background.

Autosheet in Excel GPT-6.1 Sol · Medium

Failed21s · 3¢

Why it failed: Columns B and C were not formatted like column A; headers B1 and C1 have black text on a light background instead of white font on a dark blue background.

ChatGPT + Autosheet MCP GPT-6 · Medium

Failed46s · N/A

Why it failed: Headers B1 and C1 have a white background with black text instead of matching column A's dark blue background with white font.

Autosheet in Sheets Haiku 5.5 · High

Failed19s · 0.2¢

Why it failed: Columns B and C were not formatted like column A; headers B1 and C1 have default black text instead of white text on a dark blue background.

ChatGPT in Excel GPT-6 Astra · Medium

Failed27s · 41¢

Why it failed: Headers B1 and C1 were not formatted like A1; they have black font on a white background rather than white font on a dark blue background.

Autosheet in Excel GPT-6 Luna · High

Failed11s · 0.2¢

Why it failed: Columns B and C were not formatted to match column A; headers B1 and C1 remain plain black text on a white background rather than white font on a dark blue background.

ChatGPT in Excel GPT-6 Luna · High

Failed18s · 0.4¢

Why it failed: Headers B1 and C1 do not have the dark blue background or white font seen in A1.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed29s · 35¢

Why it failed: Columns B and C were not formatted like column A; specifically, the headers B1 and C1 lack the dark blue background and white font present in A1.

ChatGPT in Sheets GPT-6 Luna · High

Failed36s · 0.4¢

Why it failed: Headers B1 and C1 were not formatted like A1; they have black font without the dark blue header background fill.