Ford has spent the last few months quietly rehiring the engineers it spent the last few years quietly letting go. The official line in the trade press is the gentler version of an uncomfortable fact: AI tooling moved fast through the easy 80% of the work and then stalled on the last 20%, the part that required knowing things nobody had bothered to write down. So the people who knew those things got a call.

A senior engineer's desk — a green "ready to merge" CI status, support thread, and 2 a.m. repro steps

This is not really an article about Ford, and it is not really an article about cars. It is an article about what AI tools are and are not, and it is the clearest real-world illustration I have seen of an argument I have been making in the AI book I am writing: the danger isn't that the model gets the answer wrong. The danger is that it gets the answer wrong and tells you it's done.

1 A tool, not an oracle

Most people really don't understand how a large language model actually works. For a long time, I understood and did not use them. Then I decided to write a book and had to use them. The simplest characterization is that An LLM is an iterative search over a problem space, converging on the response with the lowest estimated cost against the query it was given. Stretching the context window out to a million tokens didn't change that mechanism — it just gave the search a much larger space to range over, which is why a modern model can credibly discuss tax law, Rust lifetimes, and 14th-century shipbuilding in the same conversation. Breadth went up. The mechanism stayed exactly what it was: bounded optimization against a stated objective, not judgment exercised by something that understands the objective the way a person who has lived it does.

Charles Poon, Ford’s vice president of vehicle hardware engineering, added,

“Mistakenly we thought that by just introducing artificial intelligence and ingesting the design requirements that we had, that that would produce a high-quality product.”

Bounded optimization against a stated objective is a precise description of a very useful tool. It is not a description of a colleague that understands the deep nuance of the problem space.

2 Two ways an answer can fail you

Everyone has made their peace with the first failure mode by now. Ask a model something underspecified, or something whose true answer depends on context that was never written down anywhere it could read, and it will frequently produce a fluent, confident, incorrect response. We have a name for this — "hallucination" — and most engineers have internalized the discipline of checking anything important against halucinated responses.

The second failure mode is subtle and worse. It's worse because it doesn't look like a failure. Instead, it looks like success. This is the model that writes the migration, runs the tests, and reports back: done — all tests passing, ready to merge. Nothing it said was false. The tests pass. The code runs. Nevertheless the answer is, in some small but important way, incomplete — an edge case it had no way to know about, a downstream consumer it never saw, an operational assumption that lived only in the head of the person who built the original system.

$ git log --oneline -1
a3f9e21 migrate billing to new tax schema

$ ci status
✓ unit tests (412/412)
✓ integration tests (38/38)
✓ lint, typecheck

  ready to merge

Every line of the above is true. None of it tells you whether the new schema still handles the rounding exception that was added in 2019 after a support ticket whose resolution is no longer in the database. That exception lives in exactly one place: the memory of whoever fixed it. A model trained on the repository has the code. It does not have the ticket, the Slack thread, or the 2 a.m. phone call. "Declared complete" is the model reporting honestly on the problem space it had access to. The problem is that this is not the problem space the production job actually lives in.

3 What the gray hair is actually for

Putting it bluntly: when an organization replaces the senior knowledge workers with AI tools, they are effectively eliminating the backstop that protected against loss inducing actions.

This is the gap Ford rehired into. The engineers who came back didn't come back because they type faster than a model or know more syntax. They came back because they have spent decades accumulating the part of the job that was never written down anywhere a model could ingest it: which failure modes a confident-looking design is hiding, which "this is finished" is actually finished, and which one needs a second look from someone who has been burned by exactly this kind of incompleteness before. Call it institutional memory, call it scar tissue, call it taste — it is built entirely from cases that didn't make it into the documentation because at the time nobody thought they were going to matter.

That is precisely the judgment an optimizer over a stated objective cannot supply, because the objective it was handed didn't include the thing the gray hair would have flagged. The model isn't being careless. It is being asked a smaller question than the one that actually needed answering, and it is answering that smaller question very, very well.

4 A trade worth making in both directions

The useful response to this is not to distrust the tools or slow down adoption — Ford's modernization work got further, faster, with AI in the loop than it would have without it; that part worked. The useful response is to notice that the speed and the judgment are now supplied by different people, and to put them in the same room on purpose.

Pair a "with hair" engineer fluent in the tools with a "gray hair" engineer fluent in the failure modes, and something better than either alone starts to happen. The junior engineer gets work converging in days that used to take weeks, because the AI collapses the mechanical half of the job. The senior engineer gets a force multiplier on judgment that used to bottleneck on their own typing speed. And each side is quietly teaching the other the half they're missing — one teaching the prompts and the workflows, the other teaching what "done" actually has to mean before you believe it.

The model will keep telling you the migration is ready to merge. Whether it actually is remains, for now, a question only a human who has been wrong about that exact thing before knows how to ask.

← All Code & CS posts Home