All AutosheetBench tasks

Switch a chart's axis to a log scale

ChartsHardLLM judge scoringLatest run

Passed by 5 of 12 products on their best model: Autosheet in Excel, Claude in Excel, ChatGPT in Excel, Copilot in Excel and Claude in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Switch the axis to a log scale.

Starting spreadsheet

Sheet1
ABCDEFGHIJKLM
1YearRevenueProfit
220211250000180000
320221430000215000
420231590000248000
520241710000286000
620251880000324000
7
8
9
10
11
12
13
14
15
16
17

Expected result

How each AI configuration did

13 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • The chart is edited in place, colors kept
  • The y-axis uses a log scale

Passed 13

Autosheet in Excel Opus 5.5 · Medium

Passed14s · 5¢

Autosheet in Excel GPT-6 Astra · Low

Passed10s · 9¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed5s · 2¢

Claude in Excel Opus 5.5 · High

Passed18s · 14¢

Autosheet in Excel Haiku 5.5 · High

Passed6s · 0.1¢

Claude in Sheets Opus 5.5 · Medium

Passed21s · 6¢

Autosheet in Excel GPT-6.1 Sol · Medium

Passed16s · 3¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed16s · 36¢

Claude in Excel Haiku 5.5 · High

Passed11s · 0.4¢

Claude in Sheets Haiku 5.5 · High

Passed39s · 2¢

Autosheet in Excel GPT-6 Luna · High

Passed15s · 0.4¢

Copilot in Excel Auto

Passed25s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed57s · 3¢

Failed 9

Autosheet in Sheets Opus 5.5 · Medium

Failed11s · 11¢

Why it failed: The value axis remains on a linear scale; no logarithmic scaling was applied.

Claude Code + Sheets MCP Opus 5.5 · Medium

Failed30s · N/A

Why it failed: The value axis remains linear (0 to 2,000,000) rather than logarithmic, as confirmed by both Document 1 and the native workbook inspection.

Claude Code + Autosheet MCP Opus 5.5 · Medium

Failed34s · N/A

Why it failed: The existing chart was not updated; the agent left the chart in its original state without making the requested changes.

ChatGPT + Autosheet MCP GPT-6 · Medium

Failed40s · N/A

Why it failed: The chart was not updated or modified.

Autosheet in Sheets Haiku 5.5 · High

Failed30s · 0.4¢

Why it failed: The chart was not updated; the agent's update calls failed and the chart remained in its initial unedited state.

ChatGPT + Drive MCP GPT-6 · Medium

Failed35s · N/A

Why it failed: The chart was not updated with the requested changes.

Gemini in Sheets Auto

Failed2m 7s · N/A

Why it failed: The chart was not updated or modified.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed14s · 26¢

Why it failed: The value axis remains linear with no logBase set.

ChatGPT in Sheets GPT-6 Luna · High

Failed13s · 0.3¢

Why it failed: The existing chart was not updated; the agent made no modifications to the chart or sheet.