All AutosheetBench tasks

Use a log scale on data with zeros

ChartsHardLLM judge scoringLatest run

Passed by 4 of 12 products on their best model: Autosheet in Excel, Claude in Excel, ChatGPT in Excel and Copilot in Excel.

Prompt and starting spreadsheet

Task files

Prompt

Create a chart with log scale for y

Data

Starting sheet
AB
1CategoryValue
2A0
3B1
4C5
5D50
6E500
7F5000
8G50000
9H250000

Chart

Starting sheet
A
1

Expected result

How each AI configuration did

10 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • A chart with a log y-axis
  • Every non-zero point shows; zeros don't break it
  • A clear title; any legend at the top

Passed 10

Autosheet in Excel Opus 5.5 · Medium

Passed21s · 7¢

Autosheet in Excel GPT-6 Astra · Low

Passed25s · 18¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed15s · 4¢

Claude in Excel Opus 5.5 · High

Passed52s · 25¢

Autosheet in Excel Haiku 5.5 · High

Passed5m · 0.2¢

Autosheet in Excel GPT-6.1 Sol · Medium

Passed27s · 4¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed54s · 61¢

Claude in Sheets Haiku 5.5 · High

Passed29s · 2¢

Autosheet in Excel GPT-6 Luna · High

Passed35s · 0.8¢

Copilot in Excel Auto

Passed34s · N/A

Failed 12

Autosheet in Sheets Opus 5.5 · Medium

Failed53s · 13¢

Why it failed: The y-axis does not have a log scale applied; it is a linear axis (ticks 0.00, 2.00, 4.00, 6.00) plotting helper log10 values.

Claude in Sheets Opus 5.5 · Medium

Failed34s · 7¢

Why it failed: Zero is handled as a gap, but non-zero category B (value 1) has zero bar height and is invisible.

Claude Code + Sheets MCP Opus 5.5 · Medium

Failed32s · N/A

Why it failed: The y-axis is linear (intervals of 50,000 up to 250,000) rather than logarithmic, confirmed by hasLogarithmicValueAxis being false.

Claude Code + Autosheet MCP Opus 5.5 · Medium

Failed59s · N/A

Why it failed: The chart uses a standard linear axis plotting log-transformed helper data rather than an actual log scale axis.

ChatGPT + Autosheet MCP GPT-6 · Medium

Failed1m 34s · N/A

Why it failed: Y-axis remains linear with equal increments of 50,000; native feature inventory confirms hasLogarithmicValueAxis is false.

Autosheet in Sheets Haiku 5.5 · High

Failed44s · 0.5¢

Why it failed: A log scale was not applied to the y-axis; the chart features a linear axis from 0 to 300,000 and hasLogarithmicValueAxis is false.

Claude in Excel Haiku 5.5 · High

Failed22s · 0.5¢

Why it failed: Category B has value 1 and is not visible against the axis baseline.

ChatGPT + Drive MCP GPT-6 · Medium

Failed43s · N/A

Why it failed: The y-axis does not have logarithmic scaling enabled; it is a linear axis (ticks 0, 2, 4, 6) plotting pre-computed log10 values.

ChatGPT in Excel GPT-6 Luna · High

Failed56s · 0.7¢

Why it failed: Data series are not plotted and data points are absent from the chart.

Gemini in Sheets Auto

Failed31s · N/A

Why it failed: The y-axis is on a linear scale (0 to 300,000 in increments of 50,000), and workbook metadata confirms hasLogarithmicValueAxis is false with logBase: null.

ChatGPT in Sheets GPT-6 Astra · Medium

Failed37s · 46¢

Why it failed: No chart was created.

ChatGPT in Sheets GPT-6 Luna · High

Failed16s · 0.3¢

Why it failed: No chart was created.