The Tickbox and the Roadmap

There is a question everyone who services medical equipment eventually gets asked. An auditor picks a device and says: show me the record of its last service, and prove the instruments that produced these readings were within calibration. Businesses fail that moment more often than they fail the work itself. The work happened once. The evidence has to survive for years, in a form a stranger can check.

Medeq sells software to the people who do that work, and we build and run that platform. So the question is ours to answer too. This month we made ourselves answer it properly: we took ISO 13485, the standard medical device businesses run their quality systems under, and mapped every requirement in it against what the software verifiably does. All 72 of them, clause by clause.

Not one row earned the top rating. That was deliberate, and it is the most useful thing about the whole exercise.

“Our software makes you compliant.” Every tool in a regulated market wants that sentence on its pricing page, and every experienced quality manager winces when they read it.

Here is why it cannot be true. Open the standard and read what it asks for: documented procedures, management responsibilities, decisions about scope and risk that only the business itself can take. A quality system is something an organisation does. Software cannot do it on the organisation’s behalf, and software that claims to would be lying in a way an auditor is specifically trained to notice.

What software can do is narrower, and worth more when it is said plainly. It can produce the records the business presents as evidence inside its own quality system, control who touches them, and keep them for as long as the rules require. The claim is smaller. It is also checkable, which is the property the big claim never has.

Before rating a single clause, we built an inventory of what the two systems, the Field Service app and the Hub behind it, actually do. Checked in the code. Not remembered, and not read off a feature list.

Then every one of the 72 requirements got a rating against that inventory, with one rule that kept the document honest: any rating above “the software contributes” had to name the exact artefact an auditor could be handed. A signed report. A version history. A calibration certificate. No artefact, no rating.

We also refused to publish a percentage. “Compliant: 82%” looks rigorous and means nothing, and it would not survive ten minutes with a real auditor. So there is no roll-up number anywhere in the document, only the rows.

The rows that scored well all scored for the same reason: the software produces the record the clause demands. The strongest is traceability. The device serial is verified against the equipment register rather than typed. The job resolves the exact version of the checklist that was followed, and keeps it. Every measurement is stored with its acceptance limits and its verdict, not as free text. The instruments used are recorded per job, and could not have been checked out at all if their calibration had lapsed. Their certificates are bound into the report the customer signs. When the auditor asks their question, the answer is one sealed document.

And still: not full marks. The clause is discharged by the business’s own quality system, with our record inside it. That distinction survives an audit. The badge version does not.

The same method that found the strong rows found the weak ones, and refusing to look away from those is what makes the strong rows worth anything.

The analysis produced eighteen gaps, ranked by what an auditor would care about against what each one costs to fix. The one at the top hurt to write down. The standard requires that changes to a record remain identifiable. Ours were not: a completed job card could still be edited through the system’s own interfaces, and nothing recorded that it had happened. The signed PDF stays sealed, but the data behind it could drift away from the document the customer signed, and nothing would say so.

None of this makes the platform unsafe to use today. The records exist, the signing holds, and the chain of evidence is real. What was missing is tamper-evidence on the working data, and the right description of that state is a gap with an owner, not a scandal. The first fixes are with the team now, and the job is smaller than it sounds, because the pattern already exists in our own billing code, which keeps every version of every invoice it has ever generated and refuses to let one be deleted.

The uncomfortable part was never the fixing. It is writing the gap down in a document a client can read, with a number next to it. Most vendors will not do that part, and it is the only part that makes the rest of the document believable.

A tickbox says an assessment happened once. It tells you nothing about what was found, or what was done about it. A roadmap says: here is where the software stands against every requirement, here is the evidence behind each claim, here is what is missing, and here is the order it is being fixed in.

The first is easier to sell. The second is the only one that survives contact with an audit, because an auditor’s whole job is to walk the gap between what was claimed and what can be shown.

Medeq’s customers get the roadmap version, and so does Medeq: the document names which obligations the software helps discharge and which remain entirely the business’s own, because pretending the second list is empty is how a tool gets a quality manager in trouble. When the remaining work is done, we will say so in the same document, with the same scoring, and nothing will need to be walked back, because nothing was overclaimed.

The list is on the wall. We are working down it.