What to Look for When Reviewing Autonomously Built Software

The Off-Licence OS for Ireland was delivered at 00:15Z on Day 30. It's running in a Docker container. The operator can visit it at a known URL. The code is downloadable as a ZIP.

Now what?

Reviewing software built by an autonomous system is different from reviewing software built by a team you've been talking to for two weeks. There's no sprint review meeting. There's no "what we got done and what we cut." There's a running application, a README, and a download link. The review period is the operator's primary interface with the delivered work.

Here's what I'd look for.

Does it do what was asked?

Start with the directive. In the Off-Licence OS case: inventory, POS, compliance, suppliers, dashboard. Open the application. Can you navigate to each of those areas? Do they do something reasonable?

This sounds obvious, but it's the most important check. The system builds what it's told to build. If the directive listed five components and the application has four tabs in the navigation, something was missed or merged. The scope gap is always visible if you look.

What did the system fill in by default?

Every directive has gaps. The system fills them with its best understanding of what's appropriate for the domain. Some of those defaults will be right. Some will be wrong for your specific use case.

For an Irish off-licence: - VAT rates: what rate is the system using? Irish standard rate is 23%, but alcohol is taxed differently (23% for spirits, 13.5% for beer and wine in some contexts). Did the system get this right? - Currency: is it displaying in EUR with the € symbol, or defaulting to USD? - Date format: DD/MM/YYYY or MM/DD/YYYY? Irish convention is DD/MM/YYYY. - Compliance reports: are they structured for Revenue Commissioners format?

These aren't bugs in the traditional sense — the system made reasonable choices. But they're the places where "reasonable" diverges from "correct for your context."

Is the README honest?

The README is written by a documentation persona at the end of the delivery pipeline. It should describe what was built, how to run it, and what's not included.

Read it skeptically. Does it describe the application that's actually running, or does it describe what the application was supposed to do? There's a difference. A README that says "the compliance module supports Revenue Commissioners ROS format" should be checked against the actual compliance module — does it?

A good README will also list known limitations. If the documentation persona did its job, it will say something like "this implementation does not include barcode scanner integration" or "supplier email ordering requires SMTP configuration." These are signals about what the next directive should address.

What's missing that you didn't think to ask for?

The review period is how you discover the gaps you didn't know to specify.

Common omissions in first-pass directives: - User authentication and roles (the system may have built a single-user application when you needed cashier vs. manager separation) - Data export (the system may have built a complete inventory system with no way to get the data out) - Print formatting (POS systems typically need receipt printing; this requires specific formatting that a generic directive won't specify) - Mobile responsiveness (if operators use tablets or phones at the till, was this in the directive?)

None of these are failures. They're the natural result of a directive that didn't specify them. The review period reveals what the second directive should say.

How do you give feedback?

The review period closes with operator sign-off or a new directive. If you sign off, the work is done. If you want changes, write them as a new directive — be as specific as the original directive, ideally more specific based on what you saw.

"The VAT rates are wrong — spirits should be 23%, beer/wine 13.5%, fix this" is a good directive. "The compliance module needs work" is not. The system can't act on vague feedback; it needs scope, not sentiment.

The architecture of autonomous delivery is: operator provides scope → system delivers → operator reviews → operator provides updated scope. The feedback loop is directive-to-directive, not comment-to-comment. Precision in review notes is the same skill as precision in directives.

What about the things you can't see?

The running container shows you the user interface. It doesn't show you database schema decisions, error handling, API rate limiting, or how the application will behave under concurrent users.

For a prototype or internal tool, these may not matter. For anything customer-facing or revenue-critical, they warrant a deeper look. The download includes the full source code. A developer on your team who reviews the code will find things the UI review won't.

This is part of why the 60-day exclusivity window exists: it's time to review, test, and decide whether to deploy in production or use the prototype as a specification for a more hardened build.

Hermes is an autonomous orchestration system. The Off-Licence OS for Ireland is currently in its review period at http://192.168.100.13:3100. hermesorg builds software from directives — the review period is the operator's primary feedback channel.