Nobody Can Tell You What They Want
Every enhancement request I’ve ever been handed was a guess, be it a help desk ticket or a formal requirements document. The person asking wasn’t lying and wasn’t incompetent; they were describing what they could see from where they stood. The operator sees a screen that won’t do what they need, so they ask for new fields. Accounting sees a number that came out wrong and wants a change to the calculation. Each of them is telling the truth about a symptom, and neither is telling you the requirement. Getting from the symptom to the requirement is the work, and AI can’t do that part for you, because it means talking to people who don’t yet know what they mean.
In “What Is a Developer for Now?” I said those were business questions that eventually become programming decisions, and then I moved on without showing what answering one looks like. This article is the part I skipped.
“It Doesn’t Work” Is Not a Requirement
We once got a request for a new display grid in the ERP system. The data was the same data that already came out of an existing report, only in a different order. The request also asked for a popup window to apply filters to the grid even though the grid control already had filtering built into it. The ERP is a cumbersome beast to program, so read literally, the request meant paying real money to duplicate a report and bolt on a feature the grid already had. What we had been handed were pictures showing a proposed solution, drawn by somebody describing what they imagined would help. So we asked why the order mattered and what they did with the data once it was sorted and filtered that way. We also asked whether anyone had ever found the filtering that was already there.
After much discussion, we built the grid, retired the report, and discarded the popup. The grid turned out to be right as a replacement for the report, where the request had asked for an addition. Retiring the report is what made the work worth paying for. Nobody had asked for that part. The popup went away because the only reason anyone wanted it was that they had never found the filtering functionality.
Sometimes, the requesters are spot on, but most times they miss a relevant fact. A new grid on a screen is far easier to picture than a sort option they never noticed or filtering capabilities that were missed. Expecting them to hand over a specification is expecting them to do my job. They would do it badly, for the good reason that it isn’t their job.
The Requirement Is Never What You’re Handed
What arrives looks like a requirement and is usually a solution with the reasoning stripped off. In “Perfect Software” I wrote about an address book that got a county field and then a country field at a company that has never shipped anything outside the state. Those fields arrived without anyone deciding they were needed, because they seemed like the sort of thing that should be there.
The opposite case is worse because it’s invisible. One field looks meaningless in the database and accounting considers it critical. Another was repurposed by logistics for their own scheduling process years ago, and they never told anybody. Delete either one and you quickly find out the hard way how important that field is. No AI reading your schema will warn you, because that knowledge lives in a person and was never written down. It’s the tribal knowledge gained by long-term users that fills in the backstory to the requests.
Two questions get most of the way there. Who wants this, and what will they do differently once it exists? And which system is right when two of them disagree about the same number? Hardly anybody asks that one, and it does the most damage when it goes unanswered.
Developers Don’t Know What They Want Either
Developers aren’t exempt from this, and neither am I. A prompt is a requirement document. When I write “add retry logic to the serial reader,” it feels as though I’ve specified something, but the sentence leaves out every decision that matters. It doesn’t say how many retries, whether a partial message gets discarded or buffered until the rest arrives, what marks the end of a message, or what should happen when the device doesn’t fail but simply goes quiet for a minute.
The AI won’t stop and ask. It will make every one of those choices for me, confidently, and hand back something that runs. When a person hands me a vague requirement, I can see that it’s vague, because I have to go back to them about it. When I hand myself one, the gap gets filled in before I notice it was there because I have the rest in my head.
The tell is accepting the first answer. If I had actually decided what I wanted, I would be comparing what came back against it; instead I read it, it looks reasonable, and in it goes. I said in “What Is a Developer for Now?” that I won’t accept generated code unless I can explain what it does, name the business reason it exists, and describe how it fails. The middle test catches this, since I can’t name the business reason for something I never decided to build. It is also why “the AI got it wrong” is usually the wrong diagnosis. The tool answered the question I asked, and I hadn’t worked out what I was asking.
The Questions Worth Asking
The questions that pay off are boring and specific, and I keep coming back to three of them.
What do you do today instead? The workaround tells me more than the request did, because it shows what the person needs to accomplish rather than how they think it ought to be built.
What happens if this is wrong? “We lose a shift” and “somebody fixes a typo” call for two very different amounts of rigor, and I can’t tell which one I’m building without asking.
What would make you say this is finished? If nobody can answer that, the work isn’t ready to build, and no amount of generated code will make it ready.
There is a way to fail on this side as well, which is asking questions as a way of not starting. Three questions with specific answers beat twenty questions and a meeting. The goal is to find the two or three decisions that change what gets built, make them on purpose, and write down who made them. Everything else gets discovered by building something small and putting it in front of somebody, which is the same question in a more useful form.
People can tell you what hurts and what they have already tried, and that is worth having. The requirement is what you build out of it, with them, on purpose. AI made producing the code cheap, and it did nothing for the part where somebody has to go and find out what the code is for.