All AutosheetBench tasks

Look up and color letter grades

FormulasEasyDeterministic scoringLatest run

Passed by 11 of 12 products on their best model. Failed: Gemini in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

Write a formula that looks up each student's letter grade from the grading scale sheet

Highlight any F grades in red and A grades in green

Students

Starting sheet · rows 1–21 of 22
ABCD
1StudentIDNameScoreLetter Grade
2S001Aisha Okonkwo92
3S002Brandon Thistlethwaite67
4S003Camila Fuentes45
5S004Derek Johansson83
6S005Elena Petrov78
7S006Femi Adeyemi95
8S007Grace Liu71
9S008Hector Ramirez58
10S009Ingrid Sorensen88
11S010James McCallister62
12S011Karina Volkov97
13S012Leo Tanaka74
14S013Maya Gupta51
15S014Nathan Blackwood86
16S015Olivia Ferreira79
17S016Priya Sharma93
18S017Quinn O'Brien66
19S018Ravi Krishnamurthy82
20S019Sofia Delacroix47
21S020Tyler Hutchinson89

Scale

Starting sheet
ABC
1Grading Scale Reference
2Updated Fall 2024
3Min ScoreMax ScoreGrade
4059F
56069D
67079C
78089B
890100A

A passing output

Autosheet in Excel · Opus 5.5

How each AI configuration did

20 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked automatically.
  • Students!D2:D21 hold the expected values, from formulas
  • Highlight F red (3 accepted ways)
  • Highlight A green (3 accepted ways)

Passed 20

Autosheet in Excel Opus 5.5 · Medium

Passed32s · 11¢

Autosheet in Sheets Opus 5.5 · Medium

Passed48s · 14¢

Autosheet in Excel Sonnet 5.5 · Medium

Passed19s · 5¢

Claude in Excel Opus 5.5 · High

Passed31s · 28¢

Autosheet in Excel Haiku 5.5 · High

Passed32s · 0.6¢

Claude in Sheets Opus 5.5 · Medium

Passed48s · 14¢

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed52s · N/A

Autosheet in Excel GPT-6.1 Sol · Medium

Passed48s · 11¢

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed2m 1s · N/A

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed1m 21s · N/A

Autosheet in Sheets Haiku 5.5 · High

Passed1m 14s · 0.6¢

ChatGPT in Excel GPT-6 Astra · Medium

Passed1m 8s · $1.17

Claude in Excel Haiku 5.5 · High

Passed30s · 1¢

Claude in Sheets Haiku 5.5 · High

Passed1m 5s · 2¢

Autosheet in Excel GPT-6 Luna · High

Passed52s · 1¢

Copilot in Excel Auto

Passed1m 3s · N/A

ChatGPT + Drive MCP GPT-6 · Medium

Passed1m 27s · N/A

ChatGPT in Excel GPT-6 Luna · High

Passed1m 57s · 2¢

ChatGPT in Sheets GPT-6 Astra · Medium

Passed1m 15s · 88¢

ChatGPT in Sheets GPT-6 Luna · High

Passed2m 22s · 2¢

Failed 2

Autosheet in Excel GPT-6 Astra · Low

Failed18s · 15¢

Why it failed: 20 cells are wrong, such as D2: empty instead of "A".

Gemini in Sheets Auto

Failed5m · N/A

Why it failed: Still working at the 5-minute time limit. 20 cells are wrong, such as D2: empty instead of "A". D2:D21 hold no formula, and 1 other problem.