All AutosheetBench tasks

Add thousands separators

FormattingHardLLM judge scoringLatest run

Passed by 1 of 12 products on their best model: Gemini in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Add thousands separators to the big numbers.

Starting spreadsheet

Metrics
AB
1MetricValue
2Monthly users1250000
3Transactions987654
4Page views8432100
5Downloads45231

A passing output

Autosheet in Excel · GPT-6 Astra

How each AI configuration did

3 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • Every value in B2:B5 displays thousands separators.
  • The actual final spreadsheet image clearly shows every number in Metrics!B2:B5 in full within column B, including all digits and thousands separators. Fail this checkpoint if any number is replaced by hash marks (#####), clipped, truncated, shown in scientific notation, or otherwise not fully readable. The image must establish readability for all four cells; correct stored values or number-format codes alone are insufficient.
  • No currency, percentage, or abbreviated K/M suffix is introduced.
  • The underlying numeric values remain unchanged.

Passed 3

Autosheet in Excel GPT-6 Astra · Low

Passed4s · 4¢

Autosheet in Sheets Haiku 5.5 · High

Passed24s · 0.1¢

Gemini in Sheets Auto

Passed46s · N/A

Failed 19

Autosheet in Excel Opus 5.5 · Medium

Failed5s · 2¢

Why it failed: Cells B2 and B4 are displayed as hash marks (######) in the rendered sheet image rather than readable numbers.

Autosheet in Sheets Opus 5.5 · Medium

Failed13s · 2¢

Why it failed: Numbers in B2 and B4 are truncated/clipped by the column boundary and not fully readable in the spreadsheet image.

Autosheet in Excel Sonnet 5.5 · Medium

Failed4s · 0.8¢

Why it failed: Document 1 shows cells B2, B3, and B4 rendered as hash marks (######) rather than showing the full numbers.

Claude in Excel Opus 5.5 · High

Failed9s · 6¢

Why it failed: The separators are applied, but column B is too narrow: B2 and B4 show ######.

Autosheet in Excel Haiku 5.5 · High

Failed6s · 0.1¢

Why it failed: In Document 1, cells B2, B3, and B4 are replaced by hash marks (######) due to insufficient column width, failing the readability requirement.

Claude in Sheets Opus 5.5 · Medium

Failed14s · 5¢

Why it failed: The separators are applied, but column B is too narrow: B2 and B4 are cut off at its right edge, hiding their last digits.

Claude Code + Sheets MCP Opus 5.5 · Medium

Failed25s · N/A

Why it failed: Column B is too narrow, causing B2 ('1,250,000') to be clipped to '1,250,00' and B4 ('8,432,100') to be clipped to '8,432,10'.

Autosheet in Excel GPT-6.1 Sol · Medium

Failed8s · 0.9¢

Why it failed: The actual final spreadsheet image clearly shows cells B2:B4 replaced by hash marks ('######') rather than showing the full numbers.

Claude Code + Autosheet MCP Opus 5.5 · Medium

Failed28s · N/A

Why it failed: Cells B2 and B4 are clipped by the column border, cutting off trailing digits.

ChatGPT + Autosheet MCP GPT-6 · Medium

Failed29s · N/A

Why it failed: In Document 1, column B is too narrow (width 6.25), visibly clipping the trailing zero from B2 ('1,250,00') and B4 ('8,432,10').

ChatGPT in Excel GPT-6 Astra · Medium

Failed16s · 38¢

Why it failed: The separators are applied, but column B is too narrow: B2 and B4 show ######.

Claude in Excel Haiku 5.5 · High

Failed4s · 0.3¢

Why it failed: Cells B2 and B4 display hash marks (######) instead of the full numbers in the actual spreadsheet image.

Claude in Sheets Haiku 5.5 · High

Failed16s · 0.4¢

Why it failed: The rendered sheet image clearly shows truncation/clipping in cells B2 ('1,250,00') and B4 ('8,432,10') due to insufficient column width.

Autosheet in Excel GPT-6 Luna · High

Failed9s · 0.2¢

Why it failed: Cells B2 and B4 are displayed as hash marks ('######') rather than showing the full numbers.

Copilot in Excel Auto

Failed15s · N/A

Why it failed: In the actual final spreadsheet image, cells B2 and B4 are replaced by hash marks (######) rather than showing the full numbers.

ChatGPT + Drive MCP GPT-6 · Medium

Failed34s · N/A

Why it failed: In Document 1, the numbers in B2 ('1,250,00') and B4 ('8,432,10') are clipped/truncated at the right column boundary, missing their final digit.

ChatGPT in Excel GPT-6 Luna · High

Failed15s · 0.4¢

Why it failed: In the actual spreadsheet image, B2 and B4 are replaced by hash marks (######) rather than showing the full numbers.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed19s · 30¢

Why it failed: Cells B2 (1,250,000) and B4 (8,432,100) are visibly clipped at the right border of column B, cutting off the final digit.

ChatGPT in Sheets GPT-6 Luna · High

Failed16s · 0.3¢

Why it failed: Cells B2 (1,250,000) and B4 (8,432,100) are visibly clipped at the right edge of column B in the spreadsheet image.