BFCS.ai
Small BusinessBetter

Can AI Improve Small-Business Customer Service?

The strongest evidence says AI makes the people answering customers better, especially the least experienced. The weakest evidence is for the thing most small businesses actually buy: a bot that answers alone. Under Australian law, whatever it says is still the business speaking.

On this page
The BFCS verdictEvery article scored on the same four dimensions
Better
PartlyModerate confidence
Strong for assisted staff; unproven for bots left aloneA large field study found AI suggestions lifted the least experienced agents most and made customers less likely to ask for a manager. There is no comparable evidence for unsupervised small-business chatbots.
Better: Note and confidenceModerate confidenceStrong for assisted staff; unproven for bots left aloneA large field study found AI suggestions lifted the least experienced agents most and made customers less likely to ask for a manager. There is no comparable evidence for unsupervised small-business chatbots.
Faster
YesModerate confidence
More issues resolved per hourThe field evidence on throughput is rigorous but comes from one large support operation. The direction is clear; the size of the gain in a five-person business is not.
Faster: Note and confidenceModerate confidenceMore issues resolved per hourThe field evidence on throughput is rigorous but comes from one large support operation. The direction is clear; the size of the gain in a five-person business is not.
Cheaper
PartlyLow to moderate confidence
Cheap to switch on, costly when wrongSubscriptions are inexpensive, but there is little independent small-business cost evidence, and one wrong answer can cost more than months of fees.
Cheaper: Note and confidenceLow to moderate confidenceCheap to switch on, costly when wrongSubscriptions are inexpensive, but there is little independent small-business cost evidence, and one wrong answer can cost more than months of fees.
Safer
ConditionalGovernance-gated
The business owns every answer the bot givesMisleading conduct needs no intent under the Australian Consumer Law. Safety comes from limiting what the bot may say, labelling it, and keeping a fast route to a person.
Safer: Note and confidenceGovernance-gatedThe business owns every answer the bot givesMisleading conduct needs no intent under the Australian Consumer Law. Safety comes from limiting what the bot may say, labelling it, and keeping a fast route to a person.

The problem, in numbers

In a small business, customer service is usually whoever is free: the owner between jobs, a casual on a Saturday shift, a phone that rings out after hours. The quality of an answer depends on who happens to pick it up. AI promises to fix both halves of that, by answering when no one can and by helping whoever does answer to answer well.

Customers are wary of the first half. In a 2025 survey of 1,031 Australians, commissioned by a customer-messaging vendor and run by YouGov, 49% said they would rather start with a human agent even if it takes longer. The vendor's Australian release adds that 84% of consumers said being able to move easily from an AI agent to a human was important.Source 1 78% of Australians had used AI tools such as chatbots, yet only 39% reported satisfaction with them, and the release says one in five consumers would stop using a service or switch to a competitor when automated service takes longer than expected.Source 1

What frustrates customers about AI serviceSource 1
Not being understood
52%
Repeating themselves
48%
Scripted or robotic answers
46%

Share of respondents naming each frustration, as given in the vendor's Australian release. Two of the three are failures of understanding, which is exactly where a fast route to a person matters most.

What improves, and why

There are two very different products sold under the same label. One is the assistant: AI that suggests a reply, finds the policy, or drafts the email, while a person decides what is sent. The other is the agent: a chatbot that answers customers on its own. Most small businesses meet the second first, because it is what website and messaging platforms switch on by default.

The quality gain is clearest for the first. An assistant spreads the habits of the best person on the team to everyone else. For a small business with high staff turnover or casual rosters, that is the real prize: a new starter who answers like someone with two years behind them. The bot answering alone can cover the hours no one is there, but it raises the floor only if it is held to a narrow, accurate script.

The evidence says AI makes the person answering better. It does not yet say the bot is better than the person. Small businesses are mostly buying the second claim.

What the evidence actually says

The best study is a field experiment, not a vendor deck. Brynjolfsson, Li and Raymond followed the staggered rollout of an AI conversational assistant to 5,172 customer support agents. In the published version, access raised issues resolved per hour by 15% on average, including a 30% improvement for less skilled and less experienced workers, while the most experienced saw only small gains in speed.Source 3 The authors also found that AI assistance improved customer sentiment and reduced attrition among newer workers.Source 2Source 3 An earlier working paper reported slightly different figures (14% and 34%); we use the journal's.Source 2

The peer-reviewed version adds the detail that matters for quality. Customers dealing with assisted agents were less likely to ask for a manager, and the most experienced and highest-skilled agents saw little productivity change and small declines in quality.Source 3 In other words, the tool lifted the bottom of the team towards the top, and did little for the top.

The evidence, in four numbers
15%
more issues resolved per hour
support agents given AI suggestions (Brynjolfsson, Li and Raymond, QJE 2025)
30%
gain for less skilled and less experienced agents
same study; small speed gains and small quality declines for the most experienced
84%
say an easy handover from AI to a human is important
Twilio-commissioned YouGov survey, Australian release, 2025
C$812.02
Air Canada ordered to pay over one chatbot answer
damages, fees and interest; the tribunal held the airline responsible

Two cautions keep this honest. The study covers one large support operation where every AI suggestion passed through a trained person; it is not evidence that an unattended chatbot on a florist's website performs the same way. And the survey numbers above come from a vendor-commissioned poll, which is useful for direction but not for precision.

Where it falls short

  • The evidence is for assistance, not autonomy. No study of comparable rigour shows unsupervised small-business chatbots improving service quality.

  • Experts gain little. An owner who already knows every answer may see almost no quality lift, and possibly a small dip if they lean on suggestions.Source 3

  • Confident errors. A bot that invents a refund rule or misstates a guarantee does damage at scale, out of hours, with no one watching.

  • Customers want a way out. Not being understood is the top frustration, and a bot with no handover turns a small problem into a lost customer.Source 1

Where governance decides the outcome

The cautionary case is Air Canada. Its website chatbot told a grieving customer he could apply for a bereavement fare retroactively, after booking; the airline's policy did not allow it. Air Canada argued it could not be held liable for what the chatbot said. The tribunal called that a "remarkable submission" and ordered the airline to pay C$812.02 in damages, fees and interest.Source 4

Australian law points the same way. Treasury's 2025 review notes that misleading or deceptive conduct can be contravened without fault, and that an AI system's tendency to hallucinate does not warrant re-examining that rule. Its own example is a small business paying $1,000 a month for an off-the-shelf support chatbot: if the bot misleads customers, the small business is liable, though it may have a remedy against the supplier.Source 5 Treasury's advice is that businesses should make the limitations of the technology clear to consumers.Source 5

Privacy is the second line. The OAIC recommends that public-facing chatbots are clearly identified as AI, that privacy policies explain how AI is used, and, as best practice, that personal and especially sensitive information is not entered into publicly available generative AI tools.Source 6 Safer, here, is earned by deciding in advance what the bot may never say, telling customers it is a bot, and putting a person within one step.

How to think about adoption

Start with the assistant, not the agent. Let AI draft replies, summarise the history and find the policy, and keep a person pressing send. That is where the evidence is strongest, and it gives the most to newer or casual staff. Measure it the way the field study did: how many enquiries are resolved, how often customers ask for the owner, and whether new staff reach a confident standard sooner. If those numbers do not move within a few months, the tool is not earning its place.

If a customer-facing bot is worth having, give it a narrow job: opening hours, booking, order status. Keep refunds, guarantees, pricing exceptions and complaints with a person, and make the handover obvious. Label it as AI. Review a sample of its conversations every week, and treat each wrong answer as a statement the business made, because in law it is one.

How we read this evidence

BFCS.ai does not fabricate figures. The strongest study here covers AI that assists human agents at a large firm; we do not treat it as evidence for chatbots that answer customers alone, and the customer survey is vendor-commissioned, so we treat it as directional.

Sources

  1. 1

    Vendor-commissioned survey (Twilio with YouGov, not peer-reviewed): twilio.com

    What we took from it

    "Decoding Digital Patience", press release, Sydney, 16 October 2025. 7,331 adults across seven Asia Pacific markets including 1,031 Australians, fieldwork 28 August to 4 September 2025.

    Back to text
  2. 2

    Brynjolfsson, E., Li, D. & Raymond, L. R., "Generative AI at Work", NBER Working Paper 31161, April 2023 (revised November 2023). nber.org

    What we took from it

    Working-paper version, reporting 14% on average, 34% for novice and low-skilled workers and 5,179 agents; the article uses the published journal figures (note 3) instead.

    Back to text
  3. 3

    Brynjolfsson, E., Li, D. & Raymond, L., "Generative AI at Work", The Quarterly Journal of Economics, vol. 140, no. 2 (May 2025), pp. 889-942, peer-reviewed published version. academic.oup.com

    What we took from it

    5,172 agents; 15% average gain in issues resolved per hour, 30% for less skilled and less experienced workers; reports effects by skill quintile, customer requests for a manager, and attrition.

    Back to text
  4. 4

    Daily Hive, "Air Canada ordered to pay customer after airline's chatbot misguided him about fares", 15 February 2024, reporting Moffatt v. dailyhive.com

    What we took from it

    Air Canada, BC Civil Resolution Tribunal decision of 14 February 2024 (2024 BCCRT 149): C$650.88 damages, C$36.14 pre-judgment interest and C$125 tribunal fees. Secondary report; the decision text on CanLII was not reachable when checked.

    Back to text
  5. 5

    Australian Treasury, "Review of AI and the Australian Consumer Law: Final report", 3 October 2025. treasury.gov.au

    What we took from it

    Discusses misleading or deceptive conduct by AI chatbots, the absence of a fault element, and a small-business chatbot example.

    Back to text
  6. 6

    Office of the Australian Information Commissioner, "Guidance on privacy and the use of commercially available AI products", 21 October 2024 (updated 17 January 2025). oaic.gov.au

    Back to text
Continue the thread
More in Small BusinessSMALL BUSINESS · FASTERCan AI Speed Up Admin for Owners?6 min · Read →More in Small BusinessSMALL BUSINESS · CHEAPERCan AI Make Small Firms More Competitive?6 min · Read →Also BetterMANUFACTURING · BETTERCan AI Improve Manufacturing Quality?6 min · Read →