We tested for 5 rounds and nothing in the AI needed fixing. 10 models followed written button rules in 98.8% of their answers. They followed them best when the rules described the situation and did not name screens or labels. The models cannot supply judgement or context. They cannot tell which button serves the person using the screen. On that question, other designers agreed with the rules half the time, and with me 58% of the time. The work that starts now is agreeing what the rules should say. AI cannot do that work.
What does a design system have to contain so that an AI reliably picks the right button?
A wheel is one round of testing. Each wheel asked one small part of that question: which variant a button gets. Primary, secondary, ghost, destructive, or none. I ran 5 wheels, with up to 10 models and 56 buttons. Then I asked other designers to answer the same buttons.
The models followed written rules almost perfectly, even on buttons they had never seen. They followed them best when the rules described the situation and did not name screens or labels. In almost every case where a result looked wrong, the model had done what the text said.
The models cannot supply judgement or context. They cannot say what a button should do for the person using it. My own answers and my own rules disagree on 11 of 24 new buttons. Other designers do not settle the question. That is design work only humans can do. This research is where it begins.
Based on one component, buttons, and one designer's rules. It does not show whether the same holds for larger decisions, such as layout.
In each wheel the result first looked like a problem with the AI. In almost every case, the model had followed the text it was given. We did not fix anything. What we learnt, wheel by wheel, is that the models follow written rules, and what the rules need to say.
| Wheel | What was tested | Score | What it turned out to be |
|---|---|---|---|
| 001 | One model, 29 buttons: names only, then documentation, then 12 written rules | 54, then 63, then 87 of 87 | The model had nothing clear to follow |
| 002 | 10 models from 4 companies, with the same rules | all 80 of 87 or more | Every miss came from 3 places where the rules had 2 possible meanings |
| 003 | 3 sentences added to close those gaps | misses 32, then 40 of 870 | The models followed the new text exactly. The text said something I did not mean |
| 004 | Rewritten rules, 10 models, 32 buttons | 948 of 960 | The old rules had partly worked by naming test buttons as examples. General rules scored higher, and kept their score on new buttons |
| 005 | The same rules on 24 new buttons | 711 of 720 match the rules | The rules kept their score. Other designers and I did not agree with them |
For wheel 005 I answered the 24 new buttons before any model ran. Then other designers answered the same 24 in the survey. This is how each of us compares with the rules, and with me.
| Who | Agreed with the rules | Agreed with me |
|---|---|---|
| The AI, 10 models, 720 answers | 98.8%, 711 of 720 | 52.9%, 381 of 720 |
| Me, 24 buttons | 13 of 24 buttons | — |
| Other designers, 235 answers | 50%, 118 of 235 | 58%, 136 of 235 |
| Other designers, most common answer per button | 12 of 24 buttons | 15 of 24 buttons |
The AI is the only one that agrees with the rules. The people, me included, agree with the rules on about half the buttons. They agree with me a little more often than that.
This is the table the survey shows a designer after each pack of buttons, with my own answers in the second column. The AI models had the written rules. The designers did not. 8 of the 24 wheel 005 buttons are shown. My answer counts as a match when it is the most common answer in that column.
| Button | Me | AI models | Designers |
|---|---|---|---|
| Book tableA restaurant's booking page | Primary | Primary30 of 30 answers | Primary9 primary |
| Delete vaultA password manager | Destructive | Destructive30 of 30 answers | Destructive8 destructive |
| DeleteAn email inbox where the user has selected 5 emails | Destructive | Secondary30 of 30 answers | Destructive6 destructive · 3 secondary |
| Continue without insuranceA flight booking step that recommends travel insurance for €12 | Primary | Secondary30 of 30 answers | Secondary6 secondary · 1 primary · 1 ghost |
| Save for laterA photo printing checkout with an "Order prints" button | Secondary | Ghost30 of 30 answers | Secondary7 secondary · 1 ghost |
| Copy linkA document editor's share dialog with a "Send invite" button | Ghost | Secondary30 of 30 answers | Ghost6 ghost · 1 secondary · 1 primary |
| Add to wishlistAn online shop's product page with an "Add to basket" button | Ghost | Ghost26 of 30 answers | Secondary7 secondary · 1 ghost |
| UnsubscribeA newsletter settings page reached from a link in an email | Primary | Primary30 of 30 answers | Ghost6 ghost · 2 secondary · 1 primary |
Designer counts are other designers only, with my own survey answers taken out. Each bar is one button, drawn in the style of the variants chosen: solid for primary, outlined for secondary, dashed for ghost, red for destructive.
For each variant I chose, how often other designers chose the same:
Designers agree on what matters most on a screen, and on what can hurt the user. Secondary and ghost are the quiet buttons, the ones that draw less attention than a primary. Designers disagree on how quiet a quiet button should be. On the buttons where secondary or ghost was an option, about 80% of their answers were one of the two. They chose secondary over ghost by about 3 to 2. The rules choose ghost more often. All 9 of the AI's misses were on the same choice, secondary or ghost.
On these 11 buttons every model followed the rules every time. Other designers agreed with me on 6, with the rules on 3, and with neither of us on 2.
| Button | Me | Rules and AI | Other designers, most common | Agreed with me |
|---|---|---|---|---|
| 'Save for later' at a photo printing checkout | secondary | ghost | secondary | about 90% |
| 'Copy link' in a share dialog | ghost | secondary | ghost | about 80% |
| 'Undo changes' in a calendar event form | secondary | ghost | secondary | about 70% |
| 'Add to shopping list' on each of 30 recipe cards | secondary | ghost | secondary | about 65% |
| 'Delete' in an email inbox, emails restorable for 30 days | destructive | secondary | destructive | about 60% |
| 'Done' on a read-only fitness summary | primary | secondary | primary | about 55% |
| 'Add notes' in a slide editor | secondary | ghost | ghost | about 30% |
| 'Continue without insurance' next to 'Add insurance' | primary | secondary | secondary | about 20% |
| 'Keep subscription' in a cancel dialog | ghost | secondary | secondary | about 10% |
| 'Got it' on a traffic banner | ghost | secondary | primary | about 20% |
| 'Save job' on each of 20 job adverts | primary | ghost | secondary | about 20% |
On 2 more buttons most designers disagreed with both me and the rules. On 'Unsubscribe' on a newsletter page, the rules and I chose primary and most designers chose ghost. On 'Add to wishlist' on a product page, the rules and I chose ghost and most designers chose secondary.
These are the answers of an early group of invited designers, so percentages are rounded to the nearest 5. The survey is still open.
If you would like to take part in future surveys, join here.
There is one limit on this claim. The AI was not perfect along the way. Claude Haiku 4.5 missed the same 6 answers in every run of wheel 004. What the data supports is this. Given clear, general rules, the models follow them. Writing the rules is the work.
The button rules in the system have not changed since wheel 004. The next step is not another model run. It is a session with designers to agree what the rules should be. We will start with the 11 buttons where my answers and my rules differ.
Read the rules↗Nothing in the AI needed fixing. It followed my rules 98.8% of the time. Other designers agreed with them half the time, and with me 58% of the time.
The gap was never between people and machines. It is between designers, over what a button should do for the person using it. The AI did its part in 5 rounds of testing. Agreeing on the rules is the design work only humans can do. That is not the end of this research. It is the beginning of the work.