My button rules point to one variant for each button: primary, secondary, ghost or destructive. I gave the rules to 10 AI models with 24 new buttons the rules were never tuned on, and each model answered each button 3 times (as LLMs are non deterministic), 720 answers in all. In 711 of those 720 answers, 98.8%, the model picked the variant the rules point to. So the models read the rules correctly (that doesn't mean the rules are correct). My own answers, and other designers' answers, matched the rules on only about half the buttons.
Do my button rules still work on buttons they were never written for?
A wheel is one round of testing. Over 4 wheels I changed the rules and tested them on the same 32 buttons. By the end, 10 models got 948 of 960 answers right. That score could mean the rules work, or only that they fit those 32 buttons. In wheel 005, the same 10 models got 24 new buttons from products the first 32 do not cover. Each model answered each button 3 times.
Yes for the AI. The models chose the variant the rules call for in 711 of 720 answers. That is 98.8%, the same as before. The rules work on new buttons.
No for the rules. They matched my own answer on only 13 of the 24 buttons, so the models matched me 52.9% of the time. Other designers then answered the same buttons. They did not clearly side with me or with the rules.
Based on: one component, 10 models, 24 scenarios I wrote, and an early group of invited designers.
'Save for later' at a photo printing checkout. Every model saw the same task. The answer below is Claude Opus 5.5's, with its reasoning as written.
The taskContext: A photo printing checkout with an "Order prints" button. Next to it is a button that saves the selected photos to the user's account so they can order them later. The selection stays saved. Button label: "Save for later" Which variant should this button use?
ghost HR12 (applied at precedence P3): 'Save for later' postpones the ordering decision rather than completing the task another way, so it is ghost.
All 10 models chose ghost, in 30 of 30 answers. The rules treat saving for later as putting off the task.
secondary Me: secondary. Other designers: secondary about 90%, ghost about 10%.
Keeping the photos is a reasonable choice, not a way to skip the order. The models followed the rules. Most people chose differently.
Before the run, I expected about 90% of answers to match the rules, because the rules were tuned on other buttons. I wrote down that more than 95% would surprise me. The result was 98.8%. Every model matched the rules on at least 69 of 72 answers, and 5 models matched on all 72.
| Model | My answers | Answers the rules give |
|---|---|---|
| Claude Fable 5.1, low effort | 39 of 72 | 72 of 72 |
| Claude Opus 5.5 | 39 of 72 | 72 of 72 |
| Claude Sonnet 5.5 | 39 of 72 | 72 of 72 |
| Claude Haiku 5.5 | 37 of 72 | 70 of 72 |
| GPT-6 Astra | 37 of 72 | 70 of 72 |
| GPT-5.6 sol | 39 of 72 | 72 of 72 |
| Gemini 3.1 Pro | 39 of 72 | 72 of 72 |
| Gemini 3.8 Flash | 38 of 72 | 71 of 72 |
| Grok 4.7 | 38 of 72 | 71 of 72 |
| Grok 4.3 | 36 of 72 | 69 of 72 |
| All 10 | 381 of 720 | 711 of 720 |
All 9 misses were a choice between secondary and ghost:
| Button | The rules give | The models' other answer |
|---|---|---|
| 'Add to wishlist' on a product page | ghost | secondary in 4 of 30 |
| 'Assign to me' in a support ticket header | secondary | ghost in 3 of 30 |
| 'Send test email' on email settings | ghost | secondary in 2 of 30 |
Claude Haiku 5.5 is new, so before the main run it ran wheel 004's 32 scenarios with the same rules and scored 96 of 96. Runs: runs/, one folder per model.
I answered every button before seeing any model's answer. On 11 of the 24, my answer is not what my rules give. On those 11, all 10 models followed the rules in all 330 answers. So the gap is not between me and the AI. It is between me and what I wrote down.
Then other designers answered the same 24 buttons. On those 11, they sided with me on 6, with the rules on 3, and with neither on 2.
| Button | Me | Rules and models | Other designers, most common |
|---|---|---|---|
| 'Save for later' at a photo printing checkout | secondary | ghost | secondary, about 90% |
| 'Copy link' in a share dialog | ghost | secondary | ghost, about 80% |
| 'Undo changes' in a calendar event form | secondary | ghost | secondary, about 70% |
| 'Add to shopping list' on each of 30 recipe cards | secondary | ghost | secondary, about 65% |
| 'Delete' in an email inbox, emails restorable for 30 days | destructive | secondary | destructive, about 60% |
| 'Done' on a read-only fitness summary | primary | secondary | primary, about 55% |
| 'Continue without insurance' next to 'Add insurance' | primary | secondary | secondary, about 65% |
| 'Keep subscription' in a cancel dialog | ghost | secondary | secondary, about 50% |
| 'Add notes' in a slide editor | secondary | ghost | ghost, about 50% |
| 'Got it' on a traffic banner | ghost | secondary | primary, about 50% |
| 'Save job' on each of 20 job adverts | primary | ghost | secondary, about 55% |
Across all 24 buttons, other designers' most common answer matches mine on 15 buttons and the rules on 12. Neither I nor the rules are the standard.
For each variant I chose, this is how often other designers chose the same:
The split is smaller than it looks. On the buttons where secondary or ghost was an option, about 80% of designers' answers were one of the two. They agree the button should be quiet. They disagree on how quiet. They chose secondary more often than ghost, about 3 to 2. My rules lean towards ghost.
The models were also least sure here. All 9 of their misses were a choice between secondary and ghost.
These are the answers of an early group of invited designers, so percentages are rounded to the nearest 5. The survey is still open.
If you would like to participate in future surveys, join here.
These went wrong, in the answers and in my setup:
In each wheel, the problem first looked like the AI's. Each time, it turned out to be somewhere else.
| Wheel | What looked like the problem | What it turned out to be |
|---|---|---|
| 001 | The model picks the wrong button | It had nothing clear to follow. Written rules took one model from 54 to 87 of 87 |
| 002 | Only one model can do it | All 9 models from 4 companies did better with rules. Every miss came from 3 gaps in the rules |
| 003 | The models ignore the new rules | They followed them exactly. The new text said something I did not mean |
| 004 | A few misses left | The old rules partly worked by matching example words |
| 005 | The rules only work on the buttons they were tuned on | The rules held. My judgement and other designers' did not agree with them |
Every fix along the way was to the rules or the scenarios, never to the models.
Once the rules were clear, the AI stopped being the problem. What is left is deciding what the rules should say, and that is design judgement.
One limit on that claim: the AI was not perfect along the way. Claude Haiku 4.5 missed the same 6 answers in every run of wheel 004. And the models relied on example words until wheel 004 removed them.
Nothing changes in the system yet. The next wheel decides, for each of the 11 differences, whether the rule or my answer changes.
Read the rules↗The AI followed my rules 98.8% of the time. Other designers agreed with them on half the buttons.
The gap was never between people and machines. It is between designers, and closing it is the work.
The designers who answered the survey made the second half of this wheel possible. Thank you to Lucho Benedetto, Olga Libby, prz3mas, Sam White, Sonic, and the others who took part without a name.
| Setting | Value |
|---|---|
| Models | wheel 004's 10, with Claude Haiku 5.5 in place of Claude Haiku 4.5, at the same settings, each through its own application programming interface (API) |
| Information levels | rules only: sections 1 to 5 of the system's button rules, unchanged from wheel 004's run 3 |
| Scenarios | 24 new ones, 20 of the first 24 written and 4 added on the line between secondary and ghost |
| Repeats | 3 per scenario per model, each a fresh conversation, 720 answers |
| Scoring | against my answers, given before seeing any model's answer; also against what the rules give and against other designers' most common answer |
| Dates | runs on 13 October 2026; survey answers read on 2 November 2026 |
| Cost | about €2.24 in API credit, pay as you go, for 816 answers, with Claude Haiku 5.5's run on wheel 004's scenarios |
I wrote the question, the success criteria and what I expected before the run. The 4 scenarios were swapped before any model ran.
Reproduce it