All AutosheetBench tasks

Find hardcoded numbers among formulas

FormulasEasyLLM judge scoringLatest run

Passed by 11 of 12 products on their best model. Failed: Gemini in Sheets.

Prompt and starting spreadsheet

Task files

Prompt

In column D, which numbers are hardcoded rather than formulas?

Starting spreadsheet

Analysis · rows 1–21 of 101
ABCDEFGHI
1ProductSalesCalculationTotalCountryCommitted CapitalLookup ProductLookup Value
2Alpha102030Germany1250000Alpha100
3Bravo481948299638France2500000Bravo200
4Charlie250305079Germany1750000Charlie300
5Delta900#DIV/0!5000Germany500000Delta400
6Echo12005005979Spain900000Echo500
7Foxtrot775#N/A#DIV/0!Italy650000
8Product 001113722743411France412500
9Product 002127425483822Spain425000
10Product 003141128224233Italy437500
11Product 004154830964644Germany450000
12Product 005168533705055France462500
13Product 006182236445466Spain475000
14Product 007195939185877Italy487500
15Product 008209641926288Germany500000
16Product 009223344666699France512500
17Product 010237047407110Spain525000
18Product 011250750147521Italy537500
19Product 012264452887932Germany550000
20Product 013278155628343France562500
21Product 014291858368754Spain575000

A passing output

Autosheet in Excel · Opus 5.5
Transcript excerptExpected answer
I'll scan D2:D101 for values that aren't formulas. Find hardcoded values in column D 5 cells in D2:D101 hold typed-in values instead of formulas: Cell Hardcoded value D3 9638 D5 5000 D7 #DIV/0! (typed in as text, not the result of a formula) D47 14100 D88 26400 All the other filled cells in column D are formulas. I didn't change anything in the sheet.

How each AI configuration did

21 of 22 passRun files
Excel and Sheets · All types · All models · All outputs

Pass criteria

All must be met, checked by an LLM judge.
  • Names the 4 hardcoded totals: D3, D5, D47 and D88
  • Calls no other cell a hardcoded total

Passed 21

Autosheet in Excel Opus 5.5 · Medium

Passed7s · 2¢
Transcript excerptExpected answer
I'll scan D2:D101 for values that aren't formulas. Find hardcoded values in column D 5 cells in D2:D101 hold typed-in values instead of formulas: Cell Hardcoded value D3 9638 D5 5000 D7 #DIV/0! (typed in as text, not the result of a formula) D47 14100 D88 26400 All the other filled cells in column D are formulas. I didn't change anything in the sheet.

Autosheet in Excel GPT-6 Astra · Low

Passed6s · 4¢
Transcript excerptExpected answer
I’ll check the 100 data cells in column D for numbers entered directly rather than calculated by formulas. Identify hardcoded numbers in column D There are 4 hardcoded numbers in column D: D3 9,638 D5 5,000 D47 14,100 D88 26,400

Autosheet in Sheets Opus 5.5 · Medium

Passed11s · 4¢
Transcript excerptExpected answer
Column D has 4 hardcoded numbers. All the other data rows use formulas. | Cell | Hardcoded value | What a =SUM(B:C) formula for that row would give | |---|---|---| | D3 | 9638 | 9648 (4819 + 4829), so it's 10 short | | D5 | 5000 | An error, because C5 is #DIV/0! | | D47 | 14100 | About 19440, so the typed value looks out of date | | D88 | 26400 | About 36291, so the typed value looks out of date | Two other cells you may want to check: - D7 shows "#DIV/0!" but has no formula behind it. Someone probably typed or pasted the error in as text. - D4 (=SUM(B2:B4)) and D6 (=SUM(B2:B5)) are formulas, but they add up a running total down column B. Every other row adds B and C for that row, so these two don't match the pattern. I didn't change anything in the sheet.

Autosheet in Excel Sonnet 5.5 · Medium

Passed5s · 1¢
Transcript excerptExpected answer
Find hardcoded values in D In D2:D101, 4 numbers are hardcoded instead of formulas: Cell Value D3 9638 D5 5000 D47 14100 D88 26400 D7 also holds a hardcoded #DIV/0! error. It's typed in as a value, not produced by a formula. Every other non-empty cell in is a formula. I haven't changed anything. I can replace these with formulas such as =SUM(B3:C3) or highlight them if you want.

Claude in Excel Opus 5.5 · High

Passed20s · 10¢
Transcript excerptExpected answer
Column D ("Total") has 100 data rows (D2:D101). 95 of them are formulas. The other 5 are typed-in values: Cell Product Typed value What the formula would give (=B+C) D3 Bravo 9,638 9,648 (4,819 + 4,829), so it's out by 10 D5 Delta 5,000 Can't be worked out, because C5 shows #DIV/0! D7 Foxtrot #DIV/0! Not a real error: it's typed in as a value, not produced by a formula D47 Product 040 14,100 19,440 (6,480 + 12,960), so it's out by 5,340 D88 Product 081 26,400 36,291 (12,097 + 24,194), so it's out by 9,891 Separately, two cells in column D are formulas but don't follow the column's pattern. Every other formula adds up its own row (=SUM(Bn:Cn)), but these two add up part of column B instead: D4 =SUM(B2:B4) D6 =SUM(B2:B5) I checked every cell in D2:D101 and changed nothing. If you'd like, I can replace the 5 typed values and fix D4 and D6 with =SUM(Bn:Cn). Nothing was written to the workbook. Both steps only read column D, so no cells changed. What the read covered: I checked the formula or typed value in every cell of D2:D101 (100 rows). 95 are formulas and 5 are typed-in values (, ). Two of the formulas, in and, don't follow the rest of the column. I didn't check how the cells look on screen (number formats), since that doesn't change whether a value is typed in or calculated.

Autosheet in Excel Haiku 5.5 · High

Passed6s · 0.1¢
Transcript excerptExpected answer
I'll scan column D for cells that contain typed numbers instead of formulas. Find hardcoded numbers in column D Four cells in column D contain typed numbers instead of formulas: D3 9638 D5 5000 D47 14100 D88 26400 The other numeric cells in D2:D101 are formulas. I haven't changed anything. Do you want me to replace these with the formulas used in the rest of the column?

Claude in Sheets Opus 5.5 · Medium

Passed25s · 8¢
Transcript excerptExpected answer
Out of D2:D101, four cells hold typed numbers instead of formulas. The other rows all use =SUM(Bn:Cn). | Cell | Product | Hardcoded value | What =SUM(B:C) would give | |---|---|---|---| | D3 | Bravo | 9,638 | 9,648 (4,819 + 4,829), so it's 10 off | | D5 | Delta | 5,000 | Nothing usable: C5 shows #DIV/0! | | D47 | Product 040 | 14,100 | 19,440 (6,480 + 12,960) | | D88 | Product 081 | 26,400 | 36,291 (12,097 + 24,194) | Other problems in column D: - D7 has no formula either. It shows "#DIV/0!" but no formula is producing it, so it looks like the text was typed in. C7 next to it shows #N/A. - D4 uses =SUM(B2:B4) and D6 uses =SUM(B2:B5). These add up Sales going down the column instead of across the row, which doesn't match every other row. I checked every cell in D2:D101 for whether it holds a formula. I haven't changed anything. I can replace the hardcoded and mismatched cells with the standard row formula if you want.

Claude Code + Sheets MCP Opus 5.5 · Medium

Passed23s · N/A
Transcript excerptExpected answer
In column D (Total) on the Analysis tab, four cells contain typed-in numbers instead of formulas: Cell Product Hardcoded value Likely formula That formula would give D3 Bravo 9,638 =SUM(B3:C3) 9,648 D3 =SUM(B3:C3) D5 Delta 5,000 unclear (D4 and D6 use running sums of column B) — D5 D47 Product 040 14,100 =SUM(B47:C47) 19,440 D47 =SUM(B47:C47) D88 Product 081 26,400 =SUM(B88:C88) 36,291 D88 =SUM(B88:C88) None of the four matches what the formula next to it would give, so all of them look wrong as well as hardcoded. D7 (Foxtrot) is also typed in, but it holds the text "#DIV/0!" rather than a real formula error. It isn't a number, but it should be fixed too. The other cells from D2 to D101 are formulas. Rows 2 and 8–101 use =SUM(Bn:Cn), and D4 and D6 use =SUM(B2:B4) and =SUM(B2:B5). I didn't change anything in the sheet.

Autosheet in Excel GPT-6.1 Sol · Medium

Passed8s · 1¢
Transcript excerptExpected answer
I’ll check D2:D101 for numeric values entered directly rather than calculated by formulas. Identify hardcoded numbers in column D Column D contains 4 hardcoded numbers rather than formulas: D3 9,638 D5 5,000 D47 14,100 D88 26,400

Claude Code + Autosheet MCP Opus 5.5 · Medium

Passed33s · N/A
Transcript excerptExpected answer
On the Analysis tab (the only tab in the file), column D ("Total", D2:D101) has 4 numbers typed in instead of calculated by a formula: Cell Typed-in value Row label What the formula would give D3 9,638 Bravo 9,648 (4,819 + 4,829), so it's 10 off D5 5,000 Delta An error, because C5 holds the text "#DIV/0!" D47 14,100 Product 040 19,440 (6,480 + 12,960) D88 26,400 Product 081 36,291 (12,097 + 24,194) Most cells in column D use formulas that add the Sales and Calculation values in the same row, like =SUM(B8:C8). That covers D2 and D8–D101, except D47 and D88. Three other cells in column D also look wrong, though they aren't typed-in numbers: D4 ( =SUM(B2:B4) ) and D6 ( =SUM(B2:B5) ) are formulas, but they add Sales from several rows instead of adding Sales and Calculation for their own row. D7 contains the text "#DIV/0!" typed in rather than a formula or a number. I didn't change anything in the spreadsheet.

ChatGPT + Autosheet MCP GPT-6 · Medium

Passed33s · N/A
Transcript excerptExpected answer
I checked column D in the Analysis tab using Autosheet. There are 4 hardcoded numbers (values entered directly rather than calculated by formulas). Cell Hardcoded value D3 9,638 D5 5,000 D47 14,100 D88 26,400 Additional finding: Cell D7 contains a hardcoded text value, #DIV/0!, rather than an actual formula error. All other populated numeric cells in column D contain formulas. No changes were made to the spreadsheet.

Autosheet in Sheets Haiku 5.5 · High

Passed10s · 0.1¢
Transcript excerptExpected answer
Hardcoded numbers in column D: D3 (9638), D5 (5000), D47 (14100), and D88 (26400). D7 also holds a typed value, the error #DIV/0!, rather than a formula. Every other row from D2 to D101 uses =SUM(Bn:Cn), except D4 and D6, which sum a range of column B instead.

ChatGPT in Excel GPT-6 Astra · Medium

Passed14s · 33¢
Transcript excerptExpected answer
Column D contains 4 hardcoded numbers rather than formulas: - D3: 9,638 (Analysis!D3) - D5: 5,000 (Analysis!D5) - D47: 14,100 (Analysis!D47) - D88: 26,400 (Analysis!D88)

Claude in Excel Haiku 5.5 · High

Passed10s · 0.4¢
Transcript excerptExpected answer
Four cells in column D are hardcoded numbers instead of formulas: - D3: 9638 - D5: 5000 - D47: 14100 - D88: 26400 Each one should be =SUM(Bn:Cn) like the rest of the column. None of them match their row sums: - Row 3: B+C = 9,648, not 9,638 (a 10 difference). - Row 5: C5 is #DIV/0!, so the formula would also return an error. The 5,000 may be a manual workaround. - Row 47: B+C = 19,440, not 14,100. - Row 88: B+C = 36,291, not 26,400. D7 is also a typed #DIV/0! error, not a formula. Its source cells have an #N/A in C7. Separately, D4 and D6 use =SUM(B2:B4) and =SUM(B2:B5), which sum only column B, unlike the other rows. These are formulas, so I haven't changed anything. Let me know if you want me to restore the row formulas in D3, D5, D47 and D88, and how you want D7 and the D4/D6 subtotals handled. I checked column D across the full used range (A1:I101) and read the four flagged rows and row 7 directly.

Claude in Sheets Haiku 5.5 · High

Passed3m 10s · 0.6¢
Transcript excerptExpected answer
Four numbers in column D are typed in rather than formulas: D3 (9638), D5 (5000), D47 (14100), and D88 (26400). Each should be a =SUM(Bn:Cn) total like its neighbours. The header in D1 is text, and D7 holds a typed-in #DIV/0! error rather than a number. The typed values don't match what the formulas would give: - D3: the formula would return 9,648, not 9,638. - D47: the formula would return 19,440, not 14,100. - D88: the formula would return 36,291, not 26,400. - D5: the formula would return an error, because C5 is a typed-in #DIV/0!. Two other things I noticed, which I haven't changed: - D4 and D6 are formulas but use a different pattern, summing column B down the rows (=SUM(B2:B4), =SUM(B2:B5)) rather than across each row like the others. - C5 is also a typed-in #DIV/0! rather than a formula. I checked D1:D101 on the Analysis sheet, and no other hardcoded values turned up. I haven't made any changes. Do you want me to replace D3, D5, D7, D47 and D88 with the row-wise =SUM(Bn:Cn) formula, and fix C5 too?

Autosheet in Excel GPT-6 Luna · High

Passed7s · 0.2¢
Transcript excerptExpected answer
I’ll inspect the entries in column D and distinguish numeric values from formulas. The hardcoded numbers in column D are D3: 9,638; D5: 5,000; D47: 14,100; and D88: 26,400. D7 contains #DIV/0! as a non-formula error value, not a number.

Copilot in Excel Auto

Passed15s · N/A
Transcript excerptExpected answer
The hardcoded numbers in column D are: Analysis!D3 9,638 Analysis!D5 5,000 Analysis!D47 14,100 Analysis!D88 26,400 All other numeric cells in Analysis!D2:D101 are formulas. The hardcoded numbers in column D are: - Analysis!D3: 9,638 - Analysis!D5: 5,000 - Analysis!D47: 14,100 - Analysis!D88: 26,400 All other numeric cells in Analysis!D2:D101 are formulas.

ChatGPT + Drive MCP GPT-6 · Medium

Passed23s · N/A
Transcript excerptExpected answer
I checked column D in the Analysis tab of your Google spreadsheet. There are 4 hardcoded numbers (entered directly rather than calculated by formulas). Cell Hardcoded value D3 9,638 D5 5,000 D47 14,100 D88 26,400 Total: 4 hardcoded cells. All other populated numeric cells in column D are formula-driven. Open the Google spreadsheet No changes were made to the spreadsheet.

ChatGPT in Excel GPT-6 Luna · High

Passed9s · 0.4¢
Transcript excerptExpected answer
The hardcoded numbers in column D are 9,638 (D3), 5,000 (D5), 14,100 (D47), and 26,400 (D88). D7 also contains #DIV/0!, but it isn’t a number or a formula. (Analysis!D3, Analysis!D5, Analysis!D47, Analysis!D88, Analysis!D7)

ChatGPT in Sheets GPT-6 Astra · Medium

Passed16s · 34¢
Transcript excerptExpected answer
Column D contains 4 hardcoded numbers: ‌ D3: 9,638 AnalysisD3 ‌ D5: 5,000 AnalysisD5 ‌ D47: 14,100 AnalysisD47 ‌ D88: 26,400 AnalysisD88 All other numeric cells in D2 contain formulas. D7 contains #DIV/0!, so it is excluded from this list.

ChatGPT in Sheets GPT-6 Luna · High

Passed11s · 0.3¢
Transcript excerptExpected answer
The hardcoded numbers in column D are 9,638 (D3), 5,000 (D5), 14,100 (D47), and 26,400 (D88). AnalysisD3 AnalysisD5 AnalysisD47 AnalysisD88 D7 also has no formula, but it contains #DIV/0!, not a number. AnalysisD7

Failed 1

Gemini in Sheets Auto

Failed42s · N/A
Transcript excerptExpected answer
In Column D (Total), there are 6 hardcoded numbers instead of formulas: Row 3 (Bravo): 9,638 Expected formula result (Sales + Calculation): 9,648 (4,819 + 4,829) Row 4 (Charlie): 5,079 Expected formula result (Sales + Calculation): 280 (250 + 30) Row 5 (Delta): 5,000 Expected formula result: #DIV/0! error (since Column C contains #DIV/0!) Row 6 (Echo): 5,979 Expected formula result (Sales + Calculation): 1,700 (1,200 + 500) Row 47 (Product 040): 14,100 Expected formula result (Sales + Calculation): 19,440 (6,480 + 12,960) Row 88 (Product 081): 26,400 Expected formula result (Sales + Calculation): 36,291 (12,097 + 24,194) (Note: Row 7 / Foxtrot contains #DIV/0!, which is an error generated by a formula rather than a hardcoded number).

Why it failed: The assistant falsely identified D4 and D6 as hardcoded values, claiming there are 6 hardcoded numbers instead of only the four valid ones (D3, D5, D47, D88).