Quality management in the forensic unit

One careful examiner can produce one sound report. A forensic unit has to produce sound reports every week, from different people, with tools that update every month. Quality management is how a unit shows that its results do not depend on luck or on one person: methods are validated, staff are competent and tested, work is reviewed, errors are caught and fixed, and all of it is written down. Courts and defense experts increasingly ask for that evidence, not just for the report.

The standards in general terms

ISO/IEC 17025:2017 specifies, in the words of its abstract, "the general requirements for the competence, impartiality and consistent operation of laboratories". It applies to any organization performing laboratory activities, whatever its size, and accreditation bodies use it to recognize competence. Digital forensic units are commonly accredited against it. The international accreditation community also publishes guidance, Modules in a Forensic Science Process, that explains how accreditation bodies apply ISO/IEC 17025 to forensic work, digital evidence included (and ISO/IEC 17020 to inspection-type work such as scene examination).

Two information-security standards add the digital-evidence detail. ISO/IEC 27041:2015 gives guidance on showing that investigative methods are "fit for purpose": capturing requirements, validating against them, deciding how much validation is needed, and using vendor and third-party testing. ISO/IEC 27042:2015 covers analysis and interpretation, including how to select and justify methods, record enough for "independent scrutiny", and demonstrate the competence and proficiency of the team. The newer ISO 21043 series on forensic science gained parts on analysis, interpretation and reporting in 2025.

The table maps the ISO/IEC 17025 clauses that matter most to a crypto forensic unit.

Validated methods

ISO/IEC 17025 distinguishes verification, which shows that the unit can perform an established method correctly, from validation, which shows that a method the unit has developed or adapted, or uses outside its intended scope, is fit for its intended use. Much of crypto forensics falls into the second category, because tools and heuristics change faster than any standard method can be written.

Define the "method" at the level a report relies on: "extracting ERC-20 transfers for an address from a node", "applying common-input clustering to Bitcoin", "reconstructing a hop sequence from raw transactions". For each, a validation record following the ISO/IEC 27041 approach states:

1. the requirements: what the method must do, on which chains and data types, with what accuracy;
2. the test data: ground truth the unit knows independently, such as its own test transactions, a public dataset or results from a second method;
3. acceptance criteria, set before testing;
4. results, known limitations and the conditions under which the method must not be used;
5. when re-validation is triggered: a new tool version, a protocol upgrade, a new token type.

A common validation test compares a tool's output with a reference extracted directly from a node:

-- Transfers the tool missed (tool side null) or invented (reference side null)
SELECT COALESCE(r.tx_hash, t.tx_hash)     AS tx_hash,
       COALESCE(r.log_index, t.log_index) AS log_index,
       CASE WHEN t.tx_hash IS NULL THEN 'missing in tool'
            WHEN r.tx_hash IS NULL THEN 'extra in tool'
            ELSE 'amount differs' END     AS finding
FROM reference_transfers r
FULL OUTER JOIN tool_transfers t
  ON t.tx_hash = r.tx_hash AND t.log_index = r.log_index
WHERE t.tx_hash IS NULL OR r.tx_hash IS NULL
   OR t.raw_amount <> r.raw_amount;

The full outer join keeps rows that appear on either side only, so the query lists every transfer the tool missed, every transfer it reported that the node does not contain, and every amount mismatch. Suppose the reference holds 1,284 transfers and the query returns three rows, all from a fee-on-transfer token where the tool reports the sent amount instead of the received one. The validation record states that limitation, and examiners must check that token type by hand. Engines without FULL OUTER JOIN can union two left joins instead.

Review

Two kinds of review are usual before a report leaves the unit.

• Technical review: a second qualified examiner checks that the data supports each finding, the method was appropriate and correctly applied, alternative explanations were considered, and the conclusions and confidence language go no further than the evidence.
• Administrative review: references, txids, exhibit numbers, units, dates and formatting are consistent.

For findings on which a case turns, such as the link between a theft and an exchange deposit, some units use blind verification: a second examiner repeats the key step without seeing the first examiner's conclusion. Record the reviewer, the scope of the review and its outcome. Disagreements are resolved and recorded in the case file, never erased, because the record of a resolved disagreement shows the system working.

Competence and proficiency testing

Competence is documented per person and per method: training, supervised casework, a competence test, and formal authorization before independent work. ISO/IEC 17025 also requires the unit to monitor its results, typically through proficiency tests (an external provider sends a test case with a known answer) and interlaboratory comparisons (several units examine the same material). ISO/IEC 17043 sets requirements for proficiency-testing providers.

External schemes for crypto tracing are still scarce, so units build their own. A unit can create a known-answer case from its own test transactions, with a CoinJoin decoy and a bridge hop, and give it to examiners blind. It can also exchange anonymized cases with a partner unit and compare the hop sequences and confidence grades reached. Record the results, including failures, and act on them.

Error management

The Scientific Working Group on Digital Evidence (SWGDE) argues, in Establishing Confidence in Digital and Multimedia Evidence Forensic Results by Error Mitigation Analysis (version 2.0, 2018), that a single error rate is often misleading for digital evidence. Tool errors are mostly systematic, caused by software logic that fails under particular conditions, rather than random. Its answer is error mitigation analysis: identify the likely sources of error in each task, apply and document controls (tool testing, procedures, training, peer review, a second method), and quote error rates where they can be measured. NIST's scientific foundation review of digital investigation techniques (NISTIR 8354, 2022) found the techniques to be based on established computer science methods. It also warned that the meaning of artifacts can change as software changes, which applies directly to labels and wallet software.

In crypto work, errors usually fall into three groups:

When an error is found, ISO/IEC 17025 treats it as nonconforming work (clause 7.10) followed by corrective action (clause 8.7). Stop the affected work. Assess the impact, including on past cases: if a tool's cluster label proves wrong, query the case database for every report that relied on that label. Notify those who need to know, which may include prosecutors in closed cases. Find the root cause, fix it, and check later that the fix worked.

Documentation standards

A unit's documents are controlled: standard operating procedures with version numbers and approval dates, templates for notes and reports, validation records, competence records, and a register of tools and versions in use. Superseded versions are archived, not deleted, because a report from two years ago must be read against the procedure in force at the time. A useful test of the whole system is the one ISO/IEC 27042 implies: could another competent examiner, given only the case file, reach the same results and understand every judgment made along the way?

Exam focus: the scope of ISO/IEC 17025:2017 (competence, impartiality, consistent operation) and the role of accreditation guidance in applying it to digital evidence; ISO/IEC 27041 (fit-for-purpose methods, validation) and ISO/IEC 27042 (analysis, interpretation, independent scrutiny); the ISO/IEC 17025 requirements that drive decisions in a crypto unit: verify established methods but validate methods the unit develops or modifies, identify opinions and interpretations in reports, and treat errors as nonconforming work; the contents of a validation record, its stated limitations and re-validation triggers, tested against node-derived ground truth; technical, administrative and blind review; competence authorization, proficiency tests and interlaboratory comparisons; SWGDE error mitigation analysis and systematic tool errors; stopping work, assessing past cases and notifying after an error.