[Government](https://bfcs.ai/industry/government/) · [Better](https://bfcs.ai/better/)

# Can AI Improve Public-Service Quality?

When government guidance depends on who answers the phone, the answer people get can vary from right to badly wrong. An assistant grounded in published guidance can give everyone the same answer. The evidence shows that floor can rise with sustained testing, that untested tools fail confidently, and that the hardest questions still need a person.

Updated **October 2026** · 7 min read · Evidence-graded · 6 sources

## The BFCS verdict

Every article scored on the same four dimensions

| Dimension | Verdict | Why |
| --- | --- | --- |
| Better | Partly · Low confidence | Grounded assistants can raise the floor; independent measures are scarce · GDS reported GOV.UK Chat accuracy rising from 76% to 90% on its own strict, self-assessed measure. New York's chatbot gave wrong and inconsistent answers. Canada's Auditor General found the tax agency's chatbot accurate on 2 of 6 test questions. We found no larger independent audit of a government AI assistant, and no published Australian evaluation. |
| Faster | Partly · Moderate confidence | Answers in seconds, but only for in-scope questions · GOV.UK Chat averaged 10.7 seconds per answer, which GDS judged acceptable, though satisfaction rose when faster answers were simulated. Canadian tax callers waited an average of about 31 minutes for an agent in 2024-25, measured by the Auditor General. These are different services, not a comparison trial. |
| Cheaper | Unproven · Low confidence | No costed evaluation found · We found no published evaluation of the cost per accurate answer. New York's mayor said the city's chatbot cost around half a million dollars and was functionally unusable. |
| Safer | Conditional · Governance-gated | Only with grounding, testing and a route to a person · Users can over-trust an official badge, and partial answers look complete. Assistants need strict accuracy testing, a clear scope and a hand-off to a human for personal circumstances. |

## The problem, in numbers

When the quality of public guidance is measured, the results can be sobering. Between February and May 2025, Canada's Auditor General placed 167 test calls to the Canada Revenue Agency's contact centres. The auditors measured agents answering general questions about individual tax accurately only 17% of the time, and benefits and business tax questions accurately 56% and 54% of the time. (Source 1) Only 9% of an agent's performance evaluation rested on the accuracy and completeness of answers, against 45% for keeping to schedule and call handling time. (Source 1)

That is the floor this review is about. Our separate reviews cover [speed](https://bfcs.ai/articles/can-ai-make-government-faster), [unit cost](https://bfcs.ai/articles/can-ai-reduce-administrative-costs) and [eligibility decisions](https://bfcs.ai/articles/can-ai-make-benefits-decisions-fairer). Here we ask: does AI make the guidance people receive from government more accurate and more consistent? We found no published Australian evaluation of the accuracy of an AI assistant answering the public, so the evidence here comes from Canada, the UK and the US.

### The evidence, in four numbers

17% · of general individual tax questions answered accurately by Canadian tax agency staff · measured by the Auditor General of Canada, from 167 test calls across individual tax, benefits and business tax lines, February to May 2025

90% · accuracy reported for GOV.UK Chat, up from 76% at its earliest benchmark · self-assessed by GDS using subject experts and automated tools, reported March 2026

88% · of in-scope questions GOV.UK Chat now answers · self-reported by GDS; questions outside published guidance are declined by design

10 of 10 · Markup staff told wrongly by New York's chatbot that landlords need not accept housing vouchers · journalist testing, March 2024; the bot had told one reporter the opposite

## What improves, and why

In our reading, the case rests on one mechanism: a system drawing only on published guidance gives everyone the same source, at any hour, whoever picks up the phone. The Canadian audit shows the mechanism is not automatic. The auditors also put six tax questions to the agency's own chatbot, Charlie, which answered two accurately, while other publicly available AI tools answered five. (Source 1) Six questions are far too few to rank them. According to a Canadian Press report, the agency told MPs it would bring in standardised training and automation to improve accuracy. (Source 1)

Outside government, the large field study of AI help for customer support agents covered in our [small business customer service](https://bfcs.ai/articles/can-ai-improve-small-business-customer-service) review found the biggest gains among less experienced staff, a higher floor rather than a higher ceiling.

Plain-language letters are a second route. A Dutch workshop study tested ChatGPT on the six most complicated letters from a corpus of 200 government letters, combining automated readability metrics, a legal and linguistic expert review and a randomised test with 72 readers. (Source 5) As a workshop paper, we cite it for that method, not as proof of effect. In our reading, its lesson is that a simpler letter needs an expert check for legal accuracy, not just a readability score.

> Consistency is only a virtue when the answer is right. An assistant repeats its mistakes as faithfully as its facts.

## What the evidence actually says

**Grounded assistants can improve under sustained testing.** The UK Government Digital Service's first GOV.UK Chat experiment, a private pilot to which 1,000 users were invited, did not reach the accuracy GDS required. It produced a few hallucinations, mostly for ambiguous or inappropriate questions, and some users dismissed the risk of error partly because they trusted the GOV.UK brand. (Source 2) By March 2026, after two public pilots in which more than 10,000 users asked 26,000 questions, GDS reported self-assessed accuracy rising from 76% at its earliest benchmark to 90%. (Source 3) These are the team's own assessments, made by subject experts and automated tools, and an answer counts as accurate only if it meets all the standards of published content. (Source 3)

**Untested assistants fail confidently.** New York City's MyCity chatbot, built on Microsoft's Azure AI services, told business owners they could take a cut of workers' tips and that landlords need not accept housing vouchers. The Markup found it inconsistent as well as wrong: it told one reporter that landlords did have to accept vouchers, then told all ten staff who separately asked the same question that they did not. (Source 4) In January 2026 the new mayor called it "functionally unusable", and the city has since taken it down. (Source 4)

**Edge cases are where the floor gives way.** GOV.UK Chat declines questions outside published guidance by design and answers 88% of in-scope questions. (Source 3) Some users wanted an adviser who could see their personal circumstances, which it cannot do. Longer term, GDS plans to hand users to departmental customer support within the same interface. (Source 3) In our reading, the questions a general assistant cannot answer are often the ones that matter most to the person asking.

## Where it falls short

-   **The best accuracy figure is self-assessed.** The 90% GOV.UK Chat result is the team's own measurement. (Source 3) The only independent test we found, the Canadian auditor's six questions to the tax agency's chatbot, is too small to settle anything. (Source 1)
    
-   **An official badge invites over-trust.** Some GOV.UK Chat pilot users dismissed the risk of inaccuracy partly because of the site's credibility. (Source 2)
    
-   **Partial answers look complete.** Inaccurate answers often covered some aspects of a question but not all. (Source 3)
    
-   **A weak baseline is not a licence.** In our reading, beating agents who were right 17% of the time on individual tax would clear a low bar, not a high one. (Source 1)
    
-   **Cost per accurate answer is unknown.** New York's mayor put the chatbot's cost at around half a million dollars when he moved to end it. (Source 4)
    

## Where governance decides the outcome

Australian rules so far focus on registering and assessing AI, not on the route to a person. In NSW, the AI Operational Policy requires agencies to appoint an accountable official, register every AI use case on the AI Assessment Framework Platform (required for all new registrations from 30 September 2026) and refer high-risk and critical-risk use cases to the AI Review Committee. (Source 6) We found no requirement on the policy's page that members of the public be able to reach a person instead. In our reading, a public-facing assistant needs four things before launch: grounding in published guidance only, accuracy testing against a strict standard on real questions, a visible hand-off to a person for questions about individual circumstances, and an error log that feeds back into both the content and the system. Where an answer may shape a decision about entitlements, the safeguards in our [benefits review](https://bfcs.ai/articles/can-ai-make-benefits-decisions-fairer) apply.

## How to think about adoption

Measure the current floor first: test-call your own contact centre, as the Canadian auditor did, before claiming AI improves on it. Ground the assistant only in guidance you publish and maintain, because it can be no more consistent than the content beneath it. Score answers against a strict standard, as GDS does, and publish the method. Ask the same question many times, from different people, because inconsistency is itself an error, as the New York test showed. Design the hand-off to a person for personal circumstances, complaints and anything outside published guidance. When simplifying letters, have a legal reviewer confirm the meaning has survived. And treat a pilot's own accuracy score as a reason to commission an independent test, not as the test.

## How we read this evidence

BFCS.ai does not fabricate figures. The Canadian accuracy figures are measured by the Auditor General; the GOV.UK Chat figures are the team's own self-assessments; the New York findings come from journalist testing. Beyond the Canadian auditor's six-question test of the tax agency's chatbot, we found no independent audit of a government AI assistant's answers and no published Australian evaluation, and we grade confidence down for that reason.

## Sources

1.  Office of the Auditor General of Canada, "Canada Revenue Agency Contact Centres," 2025 Reports of the Auditor General of Canada to the Parliament of Canada, tabled 21 October 2025 (independent performance audit, read in full). [canada.ca](https://www.canada.ca/en/auditor-general/our-work/audit-reports/canada-revenue-agency-contact-centres.html)
    
    Office of the Auditor General of Canada, "Opening Statement before the Standing Committee on Public Accounts: Canada Revenue Agency Contact Centres," 23 October 2025: [canada.ca](https://www.canada.ca/en/auditor-general/media-room/osh-20251023-e-44763.html)
    
    Catherine Morrison, The Canadian Press, "CRA says AI will help boost call centre accuracy for taxpayers seeking info," Global News, 28 October 2025 (press report): [globalnews.ca](https://globalnews.ca/news/11499170/cra-call-centre-accuracy-ai)
    
    **What we took from it:** Auditors placed 167 calls between February and May 2025 (150 to general enquiry lines, 17 to the northern line), asking general, non-account-specific questions; agents' answers were accurate 17% of the time for individual taxes, 56% for benefits and 54% for business taxes (Exhibit 6); in a representative sample of 153 recorded calls, accuracy on account-specific questions was 71% to 98% by area (Exhibit 7); callers waited on average about 31 minutes for an agent in 2024-25; only 9% of an agent's performance evaluation was based on accuracy and completeness, against 45% for schedule adherence and call handling time; the agency's chatbot, Charlie, answered 2 of 6 tax questions accurately, while other publicly available AI tools answered 5 of 6.
    
    the agency placed greater importance on how closely agents adhered to their schedule for shifts and breaks.
    
    a CRA assistant commissioner told MPs about standardised training and said the agency planned to introduce automation to improve accuracy. The same report says the audit found taxpayers more likely to get an accurate answer from the chatbot than from an agent; the audit report itself gives only the 2 of 6 result, which we use instead.
    
2.  Government Digital Service (UK), Inside GOV.UK blog, "The findings of our first generative AI experiment: GOV.UK Chat," 18 January 2024. [insidegovuk.blog.gov.uk](https://insidegovuk.blog.gov.uk/2024/01/18/the-findings-of-our-first-generative-ai-experiment-gov-uk-chat/)
    
    **What we took from it:** A live private pilot invited 1,000 users; nearly 70% of 157 survey respondents found the responses useful and just under 65% were satisfied; answers "did not reach the highest level of accuracy demanded for a site like GOV.UK"; a few cases of hallucination, mostly for ambiguous or inappropriate queries; some users underestimated or dismissed inaccuracy risks, partly because of GOV.UK's credibility. Government's own evaluation.
    
3.  Sam Dub and Sharon McDonald, Government Digital Service (UK), Inside GOV.UK blog, "5 things we learned testing GOV.UK Chat: an AI assistant for government," 16 March 2026. [insidegovuk.blog.gov.uk](https://insidegovuk.blog.gov.uk/2026/03/16/5-things-we-learned-testing-gov-uk-chat-an-ai-assistant-for-government/)
    
    **What we took from it:** Two public pilots over 18 months; more than 10,000 users asked 26,000 questions; accuracy rose from 76% at the earliest benchmark to 90% at the latest, across all topics, assessed by subject matter experts and automated evaluation tools; an answer is rated accurate only if it meets all the standards of published content; inaccurate answers often address some but not all aspects of a question; Chat draws exclusively on GOV.UK guidance and will not answer outside its scope; 88% answer rate for in-scope questions; average answer time 10.7 seconds, which the team considers within acceptable bounds, though satisfaction rose when faster speeds were simulated; some users wanted an adviser with access to their personal information, which is not supported; longer term, the team plans to hand users to departmental customer support in the same interface. Self-reported by the team; not independently audited.
    
4.  Colin Lecher, The Markup (with Documented and THE CITY), "NYC's AI Chatbot Tells Businesses to Break the Law," 29 March 2024: [themarkup.org](https://themarkup.org/news/2024/03/29/nycs-ai-chatbot-tells-businesses-to-break-the-law)
    
    Colin Lecher and Katie Honan, The Markup and THE CITY, "Mamdani to Kill the NYC AI Chatbot We Caught Telling Businesses to Break the Law," 30 January 2026, updated 4 February 2026: [themarkup.org](https://themarkup.org/artificial-intelligence/2026/01/30/mamdani-to-kill-the-nyc-ai-chatbot-we-caught-telling-businesses-to-break-the-law)
    
    **What we took from it:** the MyCity chatbot, powered by Microsoft's Azure AI services, said employers could take a cut of workers' tips and landlords need not accept housing vouchers; it told one reporter landlords did have to accept vouchers, but ten separate Markup staffers asking the same question were all told no.
    
    the mayor called the bot "functionally unusable" and said it was costing around half a million dollars; the city has since taken it down.
    
5.  Beks van Raaij, N., Kolkman, D. and Podoynitsyna, K., "Clearer Governmental Communication: Text Simplification with ChatGPT Evaluated by Quantitative and Qualitative Research," Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context at LREC-COLING 2024, May 2024, pp. 152 to 178. [aclanthology.org](https://aclanthology.org/2024.determit-1.15)
    
    **What we took from it:** Workshop paper (abstract read), cited here for its method only: the six most complicated letters from a corpus of 200 Dutch government letters; ChatGPT compared with a naive approach and RoBERTa; automated metrics, a legal and linguistic expert review and a randomised controlled trial with 72 participants.
    
6.  Digital NSW (NSW Government), "NSW AI Operational Policy": [digital.nsw.gov.au](https://www.digital.nsw.gov.au/policy/artificial-intelligence/ai-strategy-and-operational-policy/nsw-ai-operational-policy)
    
    Digital NSW, "NSW AI Assessment Framework": [digital.nsw.gov.au](https://www.digital.nsw.gov.au/policy/artificial-intelligence/nsw-artificial-intelligence-assessment-framework)
    
    **What we took from it:** agencies must appoint an accountable official, register all AI use cases in the AI Assessment Framework Platform, refer high-risk and critical-risk use cases to the AI Review Committee and maintain accurate and complete records through the AI system lifecycle; compliance is mandated by Circular DCS-2026-02. The page states no effective date.
    
    the AIAF Platform is the mandatory platform for registering and assessing AI use cases, and from 30 September 2026 agencies must use it for all new registrations and assessments; assessment should continue from planning through to operation.
    

Putting this to work

## Working through this in your own organisation?

BFCS.ai is published by Inhouse CX. We help small and mid-sized organisations decide where AI earns its place, and put the governance in place that makes it safe to use.

[Talk to Inhouse CX →](https://inhouse.cx/engage/)

Continue the thread

- [Can AI Make Government Faster?](https://bfcs.ai/articles/can-ai-make-government-faster/): More in Government · GOVERNMENT · FASTER · 3 min · Read →
- [Can AI Make Benefits Decisions Fairer?](https://bfcs.ai/articles/can-ai-make-benefits-decisions-fairer/): More in Government · GOVERNMENT · SAFER · 7 min · Read →
- [Can AI Improve Financial Advice?](https://bfcs.ai/articles/can-ai-improve-financial-advice/): Also Better · FINANCE · BETTER · 7 min · Read →

---

Canonical page: https://bfcs.ai/articles/can-ai-improve-public-service-quality/
This is the Markdown version of the page, generated from its HTML at build time.
Site guide for agents: https://bfcs.ai/llms.txt · Every page: https://bfcs.ai/sitemap.xml