AI & The Future of Engineering
AI Is Accelerating Code. Who Is Validating It?
When software becomes easier to produce, independent validation, operational visibility, and human accountability become more consequential.

We've made software easier to produce. We haven't made it equally easy to trust.
That imbalance interests me more than the familiar argument about whether AI-generated code is good or bad. A useful assistant can remove tedious work, suggest an implementation I had not considered, and make a difficult interface easier to explore. Those benefits are real possibilities. They do not automatically tell us how to evaluate the resulting software.
The engineering question is what happens to our validation capacity when the volume of proposed change grows. If writing an implementation becomes faster while understanding its consequences remains demanding, the constraint moves. We can produce a larger queue of decisions without becoming better equipped to make them.
Production speed is not decision speed
Typing was never the only expensive part of software engineering. Discovering what a system should do, recognizing contradictory requirements, preserving invariants across boundaries, and deciding which risks are acceptable all require context. An implementation can be syntactically convincing while answering the wrong question.
Imagine an assistant proposing a cache for an account summary endpoint. The code may be readable and the tests may verify cache hits and expiration. The consequential question could still be whether a customer is allowed to see stale information after a payment. That answer lives partly in product expectations, data contracts, and operational behavior, not merely in the generated function.
This is an illustrative design problem, not an allegation about a particular model or a measured defect rate. The point is that improving implementation speed does not settle the decisions that surround the implementation.
When generated tests share the same assumption
I welcome generated tests when they help expose an unfamiliar boundary or make repetitive checks cheaper. But I become cautious when an implementation and its tests are treated as independent confirmation simply because they are separate files.
If both were produced from the same incomplete prompt, they may encode the same misunderstanding. A cache implementation and a test asserting its behavior can agree perfectly about a freshness rule that the business never accepted. Agreement is useful evidence of consistency; it is weaker evidence that the underlying assumption is justified.
The same problem exists in human-written software. AI makes it especially important to ask where an expectation came from. Was it inferred from the proposed code, derived from an independent specification, challenged by a domain expert, or observed in an existing contract? Independence is a property of reasoning and evidence, not a property of file names.
Make independent challenge affordable
For a consequential change, I would ask a reviewer to state the invariant before inspecting the suggested implementation. What must remain true? What would constitute unacceptable behavior? Which assumptions would an adversarial example challenge? That creates a reference point that is less likely to drift toward whatever the code happens to do.
Different validation techniques can then answer different questions. Contract tests examine boundaries. Property-based tests explore stated invariants across generated inputs. Mutation testing can expose assertions that fail to notice meaningful code changes. A focused manual review can challenge authorization or domain assumptions. None of these techniques is a universal guarantee.
The useful choice depends on risk. I would spend different effort on formatting a diagnostic message than on changing how a payment retry is reconciled. Applying the same ceremony to every suggestion wastes attention that consequential changes actually need.
Human accountability needs a place in the workflow
Saying that a human remains responsible is easy. Designing a workflow in which that responsibility is practical is harder. A reviewer who receives an enormous generated change shortly before release may have nominal authority without enough time or context to exercise it.
I want changes small enough to explain, assumptions visible enough to challenge, and ownership clear enough that someone can pause a release. An author should be able to describe why a dependency was introduced, which data crosses a boundary, and how failure is handled. Accepting a suggestion does not transfer those obligations to the tool.
The NIST Generative AI Profile (opens in a new tab) is a useful broader reference for considering risks and responsibilities around generative AI. It is not evidence that a particular coding assistant makes a particular application unsafe. Local decisions still need local evaluation.
Observe what the code actually changes
Pre-release checks cannot represent every interaction in a running system. For the account-summary example, I would want visibility into freshness, invalidation behavior, error paths, and the customer-visible consequences of a failed dependency. The relevant observations should follow the risk being introduced.
Observability is not a certificate issued after deployment. Instrumentation can be incomplete; sampling can conceal rare paths; an aggregate can hide a small group of affected users. We need to ask whether a signal could actually reveal the failure we are worried about, rather than whether the service has a dashboard.
A limited rollout can make a change easier to examine and reverse. It does not make every experiment acceptable. Changes involving sensitive data, irreversible side effects, or financial integrity may need stronger pre-release constraints because discovering the problem in production could already be too late.
Change the economics of validation
When I talk about making validation cheaper, I do not mean removing independent review or replacing understanding with another score. I mean reducing the effort required to collect and interpret relevant evidence. Clear contracts, representative test environments, traceable assumptions, and reusable operational checks all help.
AI can assist here too: organizing a change summary, suggesting counterexamples, identifying affected interfaces, or drafting a test that an engineer then challenges. The output should remain a proposal. Its value comes from the uncertainty it helps resolve, not the confidence of its wording.
AI is changing the economics of software creation. We need to change the economics of software confidence, too.
For me, that means evaluating a development workflow by how well it supports a defensible decision, not simply by how quickly it generates a patch. A faster proposal is valuable when we can understand, challenge, and operate it responsibly. Otherwise, we have accelerated the arrival of a decision we are still unprepared to make.