SERVICE
Algorithmic auditing
Independent testing of the models that decide things about people.
From notebook to production
Models now decide who gets an interview, who gets credit, which claim is reviewed and which customer is called. When one of those decisions is challenged, opinion is not evidence. An algorithmic audit produces the evidence.
Idiro audits models independently of the team that built them. We test the data, the performance, the fairness across protected groups and the documentation, then hand you a findings report you can put in front of a regulator, a customer or a board.
When organisations call us
A regulator or customer has asked
A questionnaire has landed and nobody can answer it with data.
A model is about to go live
Sign-off is blocked because no one has tested the thing for disparate outcomes.
An inherited portfolio
Models built over years by people who have left, with no inventory and no documentation.
A complaint or a challenge
A decision has been disputed and the explanation has to stand up.
What the audit covers
- Model inventory: what exists, who owns it, what decision it drives and which rules apply
- Data audit: provenance, representativeness, proxies for protected characteristics and leakage
- Performance testing: accuracy, calibration and stability, overall and by subgroup
- Fairness testing: selection rates, impact ratios and error parity across protected groups, including intersectional analysis
- Explainability: whether individual decisions can be explained to the person affected
- Documentation and controls: model cards, approval trails, human oversight and escalation
- Monitoring: drift thresholds, retraining triggers and who is accountable between audits
How it works
- Scope. We agree which models are in scope, which groups are tested and what evidence you need at the end.
- Transfer. Data moves under a processing agreement, pseudonymised where possible, or we run inside your environment.
- Test. We run the statistical work and review the documentation and controls.
- Review. Findings are peer reviewed internally before you see them.
- Report. You get the technical report, a ranked fix list and a summary written for a non-technical reader.
What you get
- A findings report with the numbers and the method behind them
- A fix list ranked by risk and effort
- An evidence pack for regulators, customers and auditors
- A publishable summary and, where relevant, notice wording
- A monitoring plan so the next audit is cheaper than this one
Audits are typically two to four weeks for a single model and scale with portfolio size. We are not a law firm. We provide the technical and statistical evidence, and we work alongside your counsel, who advises on the legal position.
Related case studies
Common questions
A model review asks whether the model works. An audit asks whether you can prove it works, prove it is not producing unfair outcomes, and show the documentation to back both claims.
Any model that affects a person directly: hiring, credit, pricing, fraud, claims, eligibility. Regulatory exposure and volume of decisions decide the order.
We need enough to test, which usually means outcome data with demographic attributes for the protected groups in scope. Pseudonymised data is normally sufficient and we work under a processing agreement.
Yes. We test outcomes rather than source code, so a black-box tool can still be audited on the decisions it produces.
Two to four weeks for a single model. Portfolio audits scale with the number of models and the state of the documentation.
No. We provide the technical and statistical evidence. Your counsel advises on the legal position, and we are used to working alongside them.
Talk to one of our consultants
A 30-minute call is usually enough to know whether an audit is worth doing and what it would cover. No slides.