[Education](https://bfcs.ai/industry/education/) · [Safer](https://bfcs.ai/safer/)

# Can AI Keep Students Safer Online?

Australian children meet harmful content, bullying and tracking online, and schools issue many of the devices and accounts they use. Filters and AI wellbeing monitors promise to catch risk early. The evidence shows monitors flag a great deal (in one US district, nearly two-thirds of alerts were deemed non-issues) and that they create new risks of their own: discipline, police contact and the outing of students who never chose to tell anyone.

Updated **October 2026** · 7 min read · Evidence-graded · 8 sources

## The BFCS verdict

Every article scored on the same four dimensions

| Dimension | Verdict | Why |
| --- | --- | --- |
| Better | Partly · Low confidence | Blocks listed sites; alert accuracy unproven · England requires filters to block listed illegal websites, though its standard says no filter is 100% effective. Monitoring may surface a student at risk, but in Lawrence, Kansas, nearly two-thirds of 1,200 alerts from the vendor Gaggle were deemed non-issues, according to records reported by the Associated Press, and we found no independently published accuracy data. |
| Faster | Partly · Low confidence | Alerts can be fast; help may not be · England's standard expects immediate reports for high-risk incidents. RAND's published summary says schools are often not sufficiently resourced to respond to youth mental health challenges, so in our reading speed of detection does not guarantee speed of care. |
| Cheaper | Partly · Low confidence | Licence fees are visible; the other costs are not counted · We found no independent study of what AI monitoring costs a school once staff time on false alarms, investigations and records requests is included, or what it saves. |
| Safer | Conditional · Governance-gated | Safer only with limits, transparency and a care-first response · Without independent outcome evidence, any safety benefit depends on what is scanned, who sees alerts, what goes to discipline or police, and whether students and families are told. US surveys record discipline, police contact and outing linked to monitoring. |

## The problem, in numbers

Many Australian children meet harm online. In eSafety's nationally representative survey of 3,454 children aged 10 to 17, 74% had at some point seen or heard content associated with harm, and 19% had seen material suggesting how a person can hurt or kill themselves (12% in the past year). (Source 1) Among the 2,629 children aged 10 to 15 in that survey, 52% had been cyberbullied and 23% had experienced non-consensual tracking, monitoring or harassment. (Source 1) Complaints to the regulator rose: eSafety received 3,688 under its children's cyberbullying scheme in 2025-26, up from 3,406 the year before. (Source 1)

Schools issue devices and accounts and run networks. England's Department for Education describes their two tools. Filtering is preventative: it identifies and blocks content, including AI-generated content. Monitoring is reactive: technical monitoring software views what users do on a device and generates alerts about harmful activity, including bullying. (Source 8)

### The evidence, in four numbers

19% · of Australian children aged 10 to 17 had at some point seen content suggesting how a person can hurt or kill themselves · eSafety survey of 3,454 children, December 2024 to February 2025 (12% in the past 12 months)

3,688 · complaints to eSafety's children's cyberbullying scheme in 2025-26 · up from 3,406 in 2024-25 (eSafety annual report)

29% · of LGBTQ+ US high school students said they or someone they knew had been outed by school monitoring · 19% of all students at schools using monitoring (Center for Democracy and Technology, 2023)

Nearly 2 in 3 · of 1,200 alerts from the vendor Gaggle were deemed non-issues in one Kansas district · ten months of alerts, public records request reported by the Associated Press, 2025

## What improves, and why

Filtering can stop students reaching listed illegal websites: England's standards require schools to use the Internet Watch Foundation and counter-terrorism blocklists, set so that no user can switch them off, while noting that no filter is 100% effective. (Source 8) RAND's summary of a preliminary interview study suggests AI-based suicide risk monitoring can help identify students at risk. (Source 6)

Australian regulation also pushes load onto platforms. The Basic Online Safety Expectations ask services likely to be used by children to treat their best interests as a primary consideration, and eSafety can compel reports on compliance. (Source 2) Since 10 December 2025, age-restricted social media platforms must take reasonable steps to stop under-16s holding accounts. On eSafety's initial figures, platforms removed access to about 4.7 million accounts identified as belonging to under-16s in the first half of December. (Source 2) eSafety's three-month follow-up of 803 children aged 10 to 15 found the share holding an account on an age-restricted platform fell from 52.4% to 42.1%, though 37% held one at both points. (Source 2)

> A filter decides what a student can see. A monitor decides what an adult gets to know about them. The second is the harder promise to keep.

## What the evidence actually says

The most detailed data we found comes from the US non-profit Center for Democracy and Technology (CDT). In its 2023 nationally representative surveys of 1,005 teachers, 1,018 parents and 1,029 high school students, 88% of teachers said their school monitored students' online activity, and 40% said it monitored students' personal devices. (Source 5) Among teachers at schools using monitoring, 67% knew a student who had got in trouble because of it, and 38% of teachers knew a student contacted by law enforcement after an alert. (Source 5) In 2022, 70% of teachers at monitoring schools said one purpose was detecting breaches of discipline policy, against 47% citing a possible mental health crisis. (Source 5)

In 2023, 19% of students at schools using monitoring said they or someone they knew had been outed by it, and 29% of LGBTQ+ students said the same. (Source 5) Filtering carries its own cost: 71% of students at schools that filter said blocking sometimes made assignments hard to complete, and 32% of teachers at those schools felt content about LGBTQ+ students was more likely to be blocked. (Source 5)

In Vancouver, Washington, an Associated Press and Seattle Times investigation reported that nearly 2,200 students, about 10% of enrolment, were the subject of an alert from the monitoring vendor Gaggle over one year. Officials answering the reporters' request accidentally released links to almost 3,500 unredacted student documents, and at least six students were potentially outed to school officials. (Source 7) In Lawrence, Kansas, a records request reported by the AP showed nearly two-thirds of 1,200 Gaggle alerts over ten months were deemed non-issues, including more than 200 homework assignments. (Source 7) Gaggle's own website claims more than 6 million students "kept safe"; that is a vendor claim, not an evaluated outcome. (Source 7)

Evidence on outcomes is thin. The Vancouver investigation reported no independent research showing these tools measurably lower student suicide rates or reduce violence. (Source 7) RAND's summary calls for data on how accurately the algorithms detect suicide risk and whether they improve student mental health. (Source 6) According to The 74, a 2022 investigation of four monitoring companies by US Senators Elizabeth Warren and Ed Markey found three did not directly alert students and guardians, and the companies had not checked whether their products disproportionately targeted students of colour or LGBTQ youth. (Source 6)

## Where it falls short

-   **Detection is not prevention.** We found no independent evidence that alerts reduce suicide, self-harm or violence. (Source 6), (Source 7)
    
-   **Safety tools drift into discipline.** Two-thirds of US teachers at schools using monitoring knew a student who had got in trouble because of it. (Source 5)
    
-   **False alarms have a cost.** In our reading, every non-issue alert takes staff time to clear. (Source 7)
    
-   **The records are a risk in themselves.** Monitoring captures intimate writing, and in Vancouver links to those records were released by mistake. (Source 7)
    

## Where governance decides the outcome

At least one Australian school system already discloses part of this. The NSW Department of Education's notice for NSWEduChat, its AI chat tool for years 5 to 12, tells parents that content-filter triggers may be disclosed to the school, so the principal or nominated staff would know the child's identity, and in very rare cases where the department is required to, to the Department of Communities and Justice. (Source 3) The OAIC's draft Children's Online Privacy Code, due to become law in December 2026, would require collection, use and disclosure of children's information to be consistent with their best interests. It will apply to online services, educational tools among them, run by organisations covered by the federal Privacy Act. (Source 4) England's standards spell out the mechanics: a named senior leader and governor, a review every academic year, a safeguarding lead who acts on alerts, users told that devices are monitored, and a data protection impact assessment. (Source 8) In our reading, before switching on AI monitoring, a school should settle in writing what is scanned, when, who reads alerts, what may go to police, and how families are told.

## How to think about adoption

Start with filtering, which in England has clear standards, and tune it so it does not block what students need, including health and identity information. Treat AI wellbeing monitoring as a separate, higher-risk decision. In our reading, limit it to school-managed accounts and school hours unless there is a documented reason, and send every alert first to trained wellbeing staff, not to discipline or police. Ask vendors for false-positive rates and independent evaluation. Track what follows each alert, including help given and harm done. Check that counsellors can absorb the alerts, since RAND's summary says schools are often not sufficiently resourced to respond. (Source 6) Tell students plainly what is watched. In CDT's 2022 survey, about half of US students at schools using monitoring said they were not comfortable expressing their true thoughts and feelings online when they knew they were monitored. (Source 5) Monitoring may find a crisis, but it can also change what students write.

## How we read this evidence

BFCS.ai does not fabricate figures. The most detailed data we found on student monitoring comes from US surveys and records obtained by journalists; Australian sources here describe harms, regulation and one state's disclosures, not measured outcomes of monitoring. Vendor claims are labelled as such. We found no independent study showing that AI monitoring of student devices reduces self-harm, suicide or violence.

## Sources

1.  eSafety Commissioner, "Keeping Kids Safe Online" survey. [esafety.gov.au](https://www.esafety.gov.au/research/the-online-experiences-of-children-in-australia/snapshot-content-associated-with-harm)
    
    "Digital use and risk: Online platform engagement among children aged 10 to 15" (last updated 16 June 2026): [esafety.gov.au](https://www.esafety.gov.au/research/the-online-experiences-of-children-in-australia/report-digital-use-and-risk-among-children-aged-10-to-15)
    
    eSafety Commissioner annual report 2025-26 (published with the ACMA annual report), Table 2.4 (p. 224): [esafety.gov.au](https://www.esafety.gov.au/sites/default/files/2026-10/ACMA-and-eSafety-annual-report-2025-26.pdf)
    
    **What we took from it:** "Snapshot: Content associated with harm": nationally representative survey of 3,454 children aged 10 to 17 between December 2024 and February 2025, before the social media age restrictions; 74% had seen or heard content associated with harm online at some point; 19% had seen things online that suggest how a person can hurt or kill themselves on purpose (12% in the past 12 months).
    
    same survey, with a subset of 2,629 children aged 10 to 15; of these, 52% had been cyberbullied and 23% had experienced non-consensual tracking, monitoring or harassment.
    
    3,688 complaints received under the Children's Cyberbullying Scheme in 2025-26, against 3,406 in 2024-25.
    
2.  eSafety Commissioner, "Basic Online Safety Expectations": [esafety.gov.au](https://www.esafety.gov.au/industry/basic-online-safety-expectations)
    
    eSafety, "Platforms restrict access to 4.7 million under-16 accounts across Australia," media release, 16 January 2026: [esafety.gov.au](https://www.esafety.gov.au/newsroom/media-releases/platforms-restrict-access-to-47-million-under-16-accounts-across-australia)
    
    eSafety, "Early days, early insights: Understanding experiences of social media age restrictions at the three-month follow-up," July 2026: [esafety.gov.au](https://www.esafety.gov.au/sites/default/files/2026-08/Early-days-Early-Insights-SMMA-Evaluation-Understanding-experiences-of-social-media-age-restrictions-August2026.pdf)
    
    **What we took from it:** the Expectations are a key element of the Online Safety Act; they include that children's best interests are a primary consideration in the design and operation of services likely to be used by children; eSafety can require providers to report on how they meet them, and the obligation to respond to reporting requirements is enforceable and backed by civil penalties.
    
    on initial figures gathered by eSafety, major social media companies removed access to about 4.7 million accounts identified as belonging to children under 16 in the first half of December; the minimum age obligation took effect on 10 December 2025; the Commissioner said it was too early to determine whether this constituted full compliance.
    
    follow-up sample of 803 children aged 10 to 15, surveyed March to April 2026; the share holding at least one account on an age-restricted platform fell from 52.4% at baseline to 42.1% at three months (statistically significant), and 37% of all children surveyed held an account at both points; the report says it is not an assessment of platform compliance.
    
3.  NSW Department of Education, "NSWEduChat" parental privacy notice (last updated 8 March 2026): [education.nsw.gov.au](https://education.nsw.gov.au/content/dam/main-education/about-us/educational-data/cese/NSWEduChat_Parental_Privacy_Notice.pdf)
    
    NSW Department of Education, "Privacy": [education.nsw.gov.au](https://education.nsw.gov.au/rights-and-accountability/privacy)
    
    **What we took from it:** students in years 5 to 12 can use the department's generative AI tool; the record of every question and response is collected; its inbuilt safety features are routinely monitored, which "may result in disclosures related to NSWEduChat's content filtering being made to the school," so the principal and/or nominated staff would know the child's identity; in very rare circumstances, where the department is required to submit information to the Department of Communities and Justice, this may involve information about the content filter being triggered; parental consent for online services is obtained at enrolment.
    
    the department monitors network activity to identify and respond to harmful, inappropriate or malicious activity.
    
4.  Office of the Australian Information Commissioner, "OAIC releases Exposure Draft of the Children's Online Privacy Code," 31 March 2026: [oaic.gov.au](https://www.oaic.gov.au/news/media-centre/oaic-releases-exposure-draft-of-the-childrens-online-privacy-code)
    
    OAIC, "Children's Online Privacy Code" (updated 5 August 2026): [oaic.gov.au](https://www.oaic.gov.au/privacy/privacy-registers/privacy-codes/childrens-online-privacy-code)
    
    **What we took from it:** collection, use and disclosure of personal information must be consistent with the best interests of the child; the Code is to become law in December 2026 and will cover games, streaming platforms and educational tools, not only social media.
    
    the Code applies to businesses or organisations covered by the Privacy Act 1988 (APP entities) that provide a social media service, relevant electronic service or designated internet service, as defined by the Online Safety Act 2021, likely to be accessed by children or primarily concerned with their activities, other than health services; the final Code must be registered by 10 December 2026.
    
5.  Center for Democracy and Technology (US non-profit), "Off Task: EdTech Threats to Student Privacy and Equity in the Age of AI," September 2023. [cdt.org](https://cdt.org/wp-content/uploads/2023/09/091923-CDT-Off-Task-web.pdf)
    
    CDT, "Hidden Harms: The Misleading Promise of Monitoring Students Online," summary, August 2022: [cdt.org](https://cdt.org/wp-content/uploads/2022/08/Hidden-Harms-The-Misleading-Promise-of-Monitoring-Students-Online-Research-Summary-Final-Accessible.pdf)
    
    The 2022 samples (1,008 teachers of grades 6 to 10 and 1,606 parents of grades 6 to 12) are as reported by Education Week, 8 August 2022 (second-hand). [edweek.org](https://www.edweek.org/technology/software-that-monitors-students-may-hurt-some-its-meant-to-help/2022/08)
    
    **What we took from it:** Online surveys of nationally representative samples of 1,018 parents (grades 6 to 12), 1,005 teachers (grades 6 to 12) and 1,029 students (grades 9 to 12), June to August 2023. 88% of teachers said their school uses student activity monitoring; 40% said their school monitors students' online activities on personal devices; 67% of teachers whose school uses it know a student who got in trouble; 38% of teachers know a student contacted by law enforcement because of an alert; 19% of all students whose school uses it said they or someone they know had been outed, and 29% of LGBTQ+ students; 71% of students whose school uses filtering agreed it is sometimes hard to complete assignments because of blocking; of teachers whose school uses filtering, 32% agreed LGBTQ+-related content and 37% that content about students of colour is more likely to be blocked.
    
    of teachers whose school or district uses monitoring, 70% said its purpose is to determine whether a student has violated disciplinary policy and 47% to determine whether a student is in a possible mental health crisis; approximately half of students at schools using monitoring are not comfortable expressing their true thoughts and feelings online when they know they are being monitored.
    
6.  Investigation by US Senators Elizabeth Warren and Ed Markey (second-hand; the senate.gov copy refused automated access): [the74million.org](https://www.the74million.org/article/senate-inquiry-warns-about-harms-of-digital-school-surveillance-tools-calls-on-fcc-to-clarify-student-monitoring-rules/)
    
    Ayer, L., Boudreaux, B., Welburn Paige, J. et al., "Artificial Intelligence-Based Student Activity Monitoring for Suicide Risk," RAND Health Quarterly 11(2):2, March 2024 (a preliminary RAND research study; only the abstract and key findings were read, on PMC). [pmc.ncbi.nlm.nih.gov](https://pmc.ncbi.nlm.nih.gov/articles/PMC10911757)
    
    **What we took from it:** M. Keierleber, "Senate inquiry warns about harms of digital school surveillance tools," The 74, 4 April 2022 (updated 5 April), reporting the senators' 30 March 2022 report on Gaggle, GoGuardian, Securly and Bark: "Three of the four companies indicated they do not directly alert students and guardians of their surveillance," and the companies had not attempted to determine whether their products disproportionately target students of colour or LGBTQ youth.
    
    Key findings: interviews with school staff, edtech company representatives, health care professionals and advocates "suggest" the tools "can help identify K-12 students who are at risk for suicide"; there is a need for data showing how accurately the algorithms detect risk and whether the tools improve student mental health; schools and communities "are often not sufficiently resourced to respond to youth mental health challenges."
    
7.  Second-hand reporting: [fortune.com](https://fortune.com/2025/03/12/ai-surveillance-schools-investigation-gaggle-safety-management-software)
    
    Associated Press, republished by WUSF, 7 August 2025: [wusf.org](https://www.wusf.org/education/2025-08-07/students-have-been-called-to-the-office-and-even-arrested-for-ai-surveillance-false-alarms)
    
    Vendor claim (self-reported, not independently verified): [gaggle.net](https://www.gaggle.net/)
    
    **What we took from it:** Bryan, C. (Seattle Times) and Lurye, S. (Associated Press), "'Why does my boyfriend hit me?' AI surveillance in schools is reading students' most personal thoughts," AP and Seattle Times investigation, published by Fortune, 12 March 2025. Vancouver Public Schools (Washington): between October 2023 and October 2024 nearly 2,200 students, about 10% of enrolment, were the subject of a Gaggle alert; responding to the reporters' records request, school officials accidentally provided links to almost 3,500 unredacted student documents (Gaggle screenshots of alert-triggering activity); at least six students were potentially outed to school officials; "There's no independent research showing it measurably lowers student suicide rates or reduces violence"; Owasso officials believe the tool has saved lives but do not track its efficacy.
    
    in Lawrence, Kansas, a public records request showed that nearly two-thirds of 1,200 Gaggle alerts over 10 months were deemed non-issues; one batch of false positives included over 200 student homework assignments.
    
    Gaggle's website states "6M+ students kept safe" and that human reviewers check alerts.
    
8.  UK Department for Education, "Filtering and monitoring: core standard," part of "Meeting digital and technology standards in schools and colleges" (last amended 16 September 2026). [gov.uk](https://www.gov.uk/guidance/meeting-digital-and-technology-standards-in-schools-and-colleges/filtering-and-monitoring-core-standard)
    
    **What we took from it:** Schools have a statutory responsibility under Keeping children safe in education to have appropriate filtering and monitoring; a senior leader and a governor are made responsible; provision is reviewed at least once every academic year; Internet Watch Foundation and Counter-Terrorism Internet Referral Unit blocklists must be implemented and cannot be disabled by any user; "No filtering system can be 100% effective"; filtering should not unreasonably impact teaching and learning; monitoring plans should include immediate reports for high-risk incidents; the designated safeguarding lead is responsible for safeguarding matters identified through monitoring; technical monitoring systems should notify users that the device is monitored; schools must conduct their own data protection impact assessment.
    

Putting this to work

## Working through this in your own organisation?

BFCS.ai is published by Inhouse CX. We help small and mid-sized organisations decide where AI earns its place, and put the governance in place that makes it safe to use.

[Talk to Inhouse CX →](https://inhouse.cx/engage/)

Continue the thread

- [Can AI Cut Grading Time?](https://bfcs.ai/articles/can-ai-cut-grading-time/): More in Education · EDUCATION · FASTER · 7 min · Read →
- [Can AI Personalise Learning Better?](https://bfcs.ai/articles/can-ai-personalise-learning-better/): More in Education · EDUCATION · BETTER · 6 min · Read →
- [Can AI Make Construction Safer?](https://bfcs.ai/articles/can-ai-make-construction-safer/): Also Safer · CONSTRUCTION · SAFER · 4 min · Read →

---

Canonical page: https://bfcs.ai/articles/can-ai-keep-students-safer-online/
This is the Markdown version of the page, generated from its HTML at build time.
Site guide for agents: https://bfcs.ai/llms.txt · Every page: https://bfcs.ai/sitemap.xml