Can AI Make Benefits Decisions Fairer?
Welfare systems already run on automated rules, and risk models now flag which claims get extra checks. A model can be accurate on average and still unfair to the person in front of it. The safety test is whether each decision can be traced to the law, explained to the claimant and reviewed by someone independent, and in the cases we examined that test was failed.
On this page
Better: Note and confidence
Moderate confidenceBetter at finding risk; uneven across groupsThe UK Department for Work and Pensions found its model for Universal Credit advances about three times more effective at identifying fraud risk than a random control sample, but it over-referred older claimants and non-UK nationals for review. Consistency helps only when the rule being applied is lawful.Faster: Note and confidence
Low confidenceSmall delays for those flagged; large ones when errors go unseenThe UK model added a median delay of one day for referred advances. In Australia, according to press reports of the Ombudsman's findings, an unlawful cancellation error went undetected from April 2022 to September 2023, which no speed gain offsets.Cheaper: Note and confidence
Low confidenceSavings reported; full costs unknownThe UK Department for Work and Pensions estimates £4.4 million saved over three years, but cannot separate what the model cost to build and run. Remediation after failures is not counted in any figure we found.Safer: Note and confidence
Governance-gatedSafe only where decisions are traceable, explained and contestableIn our reading, the Australian and Dutch failures we found were failures of traceability and explanation, not of headline accuracy, and the UK could test its model for fairness on only one protected characteristic. Australian reform is in train but the automated decision-making framework is not yet law.The problem, in numbers
Automated decisions are not new in Australian welfare: a provision letting the Secretary "arrange for use of computer programs to make decisions" entered the Social Security (Administration) Act in 2001. An Attorney-General's Department review counted 46 instances of Commonwealth primary legislation authorising automated decision-making, provisions it describes as not entirely consistent across the statute book.Source 1
What the people affected can see is much less. When the Information Commissioner reviewed 23 agencies with statutory authority to automate decisions, only four (17%) disclosed using it in their freedom of information publication scheme material, none of them clearly, and the Commissioner found no agency with a published policy on it.Source 2 The Robodebt Royal Commission, whose subject we cover in our review of AI and government speed, put this gap on the reform agenda.Source 1
What improves, and why
In our reading, the case for automation is consistency and targeting. Code applies a rule the same way every time, and a risk model can direct caseworker time to the claims most likely to be wrong. The UK Department for Work and Pensions found its machine learning model for Universal Credit advances about three times more effective at identifying fraud risk than a random control sample, and estimates it saved £4.4 million over three years. A caseworker, not the model, decides whether an advance is paid.Source 8
Consistency is only a virtue if the rule is right, and fairness can only be checked if someone can see the rule. In our reading, that is why the Robodebt Royal Commission's recommendations on automation turn on auditability and contestability rather than accuracy. Recommendation 17.1 asks the Commonwealth to consider a consistent legal framework and, where decisions are automated, a clear path to review, a plain-language explanation and business rules and algorithms open to independent expert scrutiny. Recommendation 17.2 asks it to consider a body to monitor and audit automated decision-making for its technical aspects and for fairness, bias and client usability.Source 1 The government committed to consider both.Source 1
A model can be accurate on average and still unlawful in a particular case. The test is whether anyone, including the claimant, can trace the decision back to the law.
What the evidence actually says
In our reading, Australia's clearest recent test is a rules engine, not AI: the system applying job seekers' mutual obligation penalties under the Targeted Compliance Framework. A Deloitte review for the employment department could not give assurance that the system was aligned with the law. It found a "critical lack of traceability" between legislation, policy and system logic, and over 370 business rules not consistently connected to their legal or policy basis. Three defects affected at least 1,326 people, two of them present from the start and undetected for over five years; the review called this a very low detected error rate. No record of the human discretionary decisions the law requires was provided. Assurance relied on reverse engineering the system, which the review said limits scrutiny by the Ombudsman, the courts and the tribunal.Source 3
According to press reports of its investigation, the Ombudsman found cancellations of income support "not lawful": 2022 amendments required a discretionary decision that was never coded in. Payments were cancelled for 964 people between April 2022 and July 2024, as AAP reported the findings. The error went undetected from April 2022 until September 2023, and it took almost ten more months to pause cancellations.Source 4 In our reading, the system did what it was coded to do, and a low error rate did not make it lawful.
In the Netherlands, a Hague court ruled in 2020 that the legislation behind SyRI, which linked government data to flag possible benefits and tax fraud, breached the right to private life and had no binding effect, finding its application "insufficiently transparent and verifiable".Source 6 Without insight into the risk model, the court said, it was hard to see how a person could defend themselves against a risk report. In the childcare benefits scandal, Amnesty International, an advocacy organisation, reported that "Dutch citizenship: yes/no" was a parameter in the tax authority's risk model, and that civil servants reviewing high-scoring applications were given no information on why the system had scored them, and that parents could not get meaningful information to defend themselves.Source 7
In the UK, the Department for Work and Pensions' July 2025 fairness assessment found its advances model over-referred claimants aged 45 and above and non-UK nationals for review, leaving those groups to find extra evidence. It could test only one of nine protected characteristics, age, because too few referred claimants gave answers on its equality questionnaire.Source 8 The department concluded that continuing to use the model was reasonable and proportionate.Source 8
Where it falls short
Accurate is not lawful. The compliance system cancelled payments exactly as coded, after the law had changed.Source 3,Source 4 In our reading, testing against past outcomes cannot find a rule that should no longer apply.
A human in the loop needs to see the reasons. Dutch reviewers were not told why a claim scored high, Amnesty found,Source 7 and the SyRI court found a human check insufficient while the selection method stayed hidden.Source 6
Fairness testing needs data agencies do not hold. The UK could check its model against only one protected characteristic, age.Source 8
Transparency collides with fraud control. Publishing full business rules may let fraudsters engineer claims,Source 1 and UK officials report the same tension.Source 8
Where governance decides the outcome
Australian rules are tightening, but the central reform is not yet law. Under the government's AI policy, whose current version took effect on 15 December 2025, non-corporate Commonwealth agencies must name accountable officials, publish AI transparency statements, keep use case registers and assess the impact of AI use cases.Source 5 From 10 December 2026, entities covered by the Privacy Act must state in their privacy policies the kinds of personal information and decisions involved where computer programs make, or do something substantially and directly related to making, decisions that could significantly affect people's rights or interests.Source 5 In July 2026 the government gave the Attorney-General the lead on a framework for automated decisions in federal agencies; the report we found gave no timetable.Source 5 In our reading, the working test is simple: for every automated decision, the agency can show the legal authority, the rule or model applied, the human who exercised any required discretion, the reasons given to the person, and the route to review.
How to think about adoption
Start with traceability, not the model. Map every rule and risk score to the provision that authorises it, and re-map whenever the law changes. Keep a record of each human decision the law requires, with the reasons the officer saw. Tell people when automation shaped their decision and how to seek review, in plain language. Test fairness on outcomes for the people flagged,. Give an independent auditor full access to rules and models, even where they cannot be published. In benefits, accuracy is the last check, not the first.
BFCS.ai does not fabricate figures. The systems examined here are rules engines or risk models rather than generative AI; we include them because benefits decisions are where automated judgement meets legal rights. Court rulings, audits, a consultancy review and an advocacy report answer different questions and are not pooled. We found no independent study showing that a welfare risk model improved fairness for claimants.
Sources
- 1
Attorney-General's Department, "Use of automated decision-making by government: Consultation paper," November 2024 (consultation closed 15 January 2025). consultations.ag.gov.au
Australian Government Architecture, "Automated decision-making: better practice guide" (DES166, responsible agency Commonwealth Ombudsman), summarising the guide's coverage of administrative law compliance, quality assurance to maintain accuracy, and transparency and accountability. architecture.digital.gov.au
Back to textWhat we took from it
Sets out Robodebt Royal Commission Recommendation 17.1 (consider a consistent legal framework; where automated decision-making is used, a clear path to seek review, plain-language website information on how the process works, and business rules and algorithms made available for independent expert scrutiny) and Recommendation 17.2 (consider a body to monitor and audit automated decision-making for technical aspects, fairness, avoiding bias and client usability); the Government committed to consider both and the department is leading the framework. Section 6A of the Social Security (Administration) Act 1999 was inserted in 2001; a desktop review identified 46 instances of primary legislation authorising automated decision-making, not entirely consistent across the statute book. Notes the Commonwealth Ombudsman's Better Practice Guide says business rules must be readily understandable and information about automated systems publicly available, and that publishing full business rules may let fraudsters engineer claims.
- 2
Office of the Australian Information Commissioner, "Automated decision making and public reporting under the Freedom of Information Act," 21 January 2026, from a desktop review in October 2025. oaic.gov.au
Back to textWhat we took from it
23 agencies were reviewed, each with statutory authority to use automated decision-making (the OAIC identified 64 statutes with such provisions across 12 portfolios). 4 agencies (17%) disclosed using it in their Information Publication Scheme material and none of those was clear about how; 9 (39%) referred to it without confirming use; 10 (43%) did not mention it; no reviewed agency had published guidelines or policies on automated decision-making.
- 3
Deloitte, "Targeted Compliance Framework: Final Report," independent review for the Department of Employment and Workplace Relations. dewr.gov.au
Back to textWhat we took from it
The department's page says the report was updated on 3 February 2026 to address identified corrections, replacing the report dated 26 September 2025 (itself replacing a 4 July 2025 version); we read the file currently served at the link below (last modified 3 February 2026), whose internal update note is still dated 26 September 2025 and says updates do not affect its findings. The figures below are as they appear in that current file. Advisory work, not an audit under Australian auditing standards. Findings: insufficient evidence that the framework and its IT system are comprehensively aligned with the authorising legislation; a "critical lack of traceability" linking legislation, policy and automated compliance logic; documentation of over 370 business rules not consistently connected to their legislative or policy basis; three defect issues identified July 2023 to May 2024 affecting at least 1,326 participants, two embedded from inception and undetected for over five years, which the review describes as a very low detected error rate; no record of human discretionary decision-making provided; assurance relies on reverse engineering the system, limiting review by the Ombudsman, the courts and the Administrative Review Tribunal.
- 4
Commonwealth Ombudsman investigation into automated cancellations of income support under the Targeted Compliance Framework, released 6 August 2025. itnews.com.au
Braue, D., Information Age (ACS), 7 August 2025: ia.acs.org.au
AAP, "Shadows of robodebt as welfare unlawfully cancelled," August 2025: aapnews.aap.com.au
Back to textWhat we took from it
The Ombudsman's site refused our automated request, so this rests on two press reports of the findings (second-hand; we have not read the report itself). Crozier, R., "Job seekers had payments cancelled unlawfully by gov IT system," iTnews, 6 August 2025: 2022 amendments gave Services Australia decision-makers discretion over cancellation that was never coded into the system; the error went undetected from April 2022 until September 2023, when external legal advisers raised it, and it took almost 10 months to pause cancellations in July 2024.
the Ombudsman found the cancellations "not lawful"; the article states that 1,009 JobSeeker recipients had benefits cancelled, and separately reports 964 recipients flagged for failing mutual obligation requirements and a further 45 cancelled after the pause was ordered; seven recommendations.
payments were automatically terminated for 964 people under the Targeted Compliance Framework between April 2022 and July 2024. The iTnews report gives no count.
- 5
Digital Transformation Agency, "Policy for the responsible use of AI in government," Version 2.1 (v2.0 in effect from 15 December 2025; minor updates 21 October 2026; v1.1 took effect 1 September 2024). digital.gov.au
Office of the Australian Information Commissioner, "New resources on transparency for use of AI and automated decision making," 30 September 2026: oaic.gov.au
Colley, A., "Feds move to regulate automated decision-making," iTnews, 20 July 2026: itnews.com.au
Back to textWhat we took from it
Non-corporate Commonwealth entities must meet requirements for accountable officials, transparency statements, AI use case accountability, internal use case registers, staff training and AI use case impact assessment.
from 10 December 2026, where a computer program makes, or does something substantially and directly related to, a decision that could reasonably be expected to significantly affect an individual's rights or interests using their personal information, APP entities must state in their privacy policy the kinds of personal information used and the kinds of decisions involved; resources include a supplementary fact sheet for government agencies.
the Attorney-General has been tasked with leading a framework to regulate automated decision-making in federal agencies; the report gives no timeline.
- 6
District Court of The Hague, judgment of 5 February 2020, case C/09/550982 / HA ZA 18-388, ECLI:NL:RBDHA:2020:1878 (English version of ECLI:NL:RBDHA:2020:865), NJCM et al. v. the State of the Netherlands (SyRI). biodiritto.org
Back to textWhat we took from it
The court held that the SyRI legislation (Section 65 SUWI Act and Chapter 5a SUWI Decree) has no binding effect because it fails to strike the fair balance required by Article 8(2) of the European Convention on Human Rights (the right to respect for private life), and in that respect its application is "insufficiently transparent and verifiable"; without insight into the risk indicators and risk model it is difficult to see how a data subject could defend themselves against a risk report; the human check for false positives and false negatives was deemed insufficient because the manner of the definitive risk selection is not public; SyRI had only been applied in so-labelled problem districts, giving a risk of links based on bias such as lower socio-economic status or an immigration background. The official database refused our automated request; we read the English text in this copy, hosted by BioDiritto.
- 7
Amnesty International, "Xenophobic machines: Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal," EUR 35/4686/2021, 25 October 2021 (advocacy organisation report, drawing on parliamentary documents, regulator findings and media reporting; not peer-reviewed). amnesty.org
Full report: amnesty.org
Back to textWhat we took from it
From 2013 the tax authorities used a risk classification model with self-learning elements; "Dutch citizenship: yes/no" was a parameter; applications with the highest scores went to a civil servant who could label them fraudulent but was given no information on why the system scored them high (the Dutch government argued, and Amnesty rejects, that considering citizenship did not make nationality a risk factor); parents were deprived of meaningful information about their results and unable to defend themselves; tens of thousands of parents, caregivers and children were affected.
- 8
National Audit Office (UK), "Tackling benefit overpayments due to fraud and error," HC 1336, Session 2024-26, 22 October 2025, paragraphs 3.4 to 3.16. nao.org.uk
Full report: nao.org.uk
Back to textWhat we took from it
Since May 2022 the Department for Work and Pensions has used a machine learning model to refer high-risk Universal Credit advance requests to a caseworker, who decides; estimated savings £4.4 million from 2022-23 to 2024-25, with no estimate of the model's costs; around three times more effective at identifying fraud risk than a randomised control sample; median payment delay of one day for referred requests. Its July 2025 fairness assessment found older claimants (45 to 54 and above) and non-UK nationals over-referred; analysis was possible on only one of nine protected characteristics (age) because equality questionnaire responses fell below a 70% threshold; DWP considers continued use reasonable and proportionate. DWP did not publish its early 2024 fairness results. The performance, delay and fairness figures are DWP's own analyses as reported by the NAO; DWP also analysed nationality, which is not a protected characteristic. The Committee of Public Accounts recommended in January 2025 that it share them, DWP briefed Committee members privately in June 2025 and published its assessment in July 2025. Officials told the NAO transparency is sometimes hard without helping fraudsters.