Safeguarding and AI

Pupils type or photograph homework questions. If what a pupil writes suggests they may not be safe (thoughts of self-harm or suicide, being hurt, threatened, abused or neglected, being groomed online, or being afraid to go home), the lesson opens with where to get help straight away: Childline (0800 1111), Shout (text 85258), Samaritans (116 123) and 999. The tutor speaks to them kindly, encourages them to talk to a trusted adult, never asks what happened and never promises secrecy. If they also asked a real question, it still teaches it.

Homework about these subjects (a set text, a history source, a PSHE task) is not treated as a concern on its own: what counts is what the pupil says about themselves.

Your safeguarding lead, if you turn alerts on. Alerts are off until an admin chooses a safeguarding lead on the school's Safeguarding page. The lead then gets an email that says only that there is a concern and the pupil's first name. What the pupil wrote is shown only after signing in, behind two-step sign-in if your staff use it, never in an email. The lead or an admin can mark each concern as seen, and that is recorded in the school's activity log.

Never a pupil's parents. For pupils using The Tutor AI through your school, we do not contact parents. Whether and how to involve a family is your judgement, made with what you know about it.

The pupil. The pupil is told that your safeguarding lead knows and can see what they wrote, so nothing happens behind their back.

Pupils who use The Tutor AI on their own, outside a school, may have a linked parent account. Only then can a parent be told, without being shown what was written, and only when a second check is confident it is safe: never if the worry may involve anyone at home or in the family, the child fears their parents' reaction, or it cannot tell.

Two checks are made. The model writing the lesson flags a possible concern. A second, more capable model then reads the message on its own with one job: is this a real concern, who might it involve, and could a parent safely be told. If the second check finds no real concern, no one is emailed, though the flag stays on your Safeguarding page. A photo cannot be checked a second time, so a flagged photo reaches your lead only.

We test this against a set of 23 difficult messages, including 16 real concerns (10 of them involving someone at home or in the family, where a parent must not be told), everyday exaggerations such as “my parents will kill me if I fail”, and homework about suicide and self-harm. In 3 runs it made the right call every time (69 of 69). That is our own testing, not an independent study, and an AI can still get a message wrong.

For a school with alerts on, each concern is kept on our database (in the EU, in Ireland) with what the pupil wrote, when, who the concern seemed to involve, and the second check's one-line reading. It is deleted after 90 days, so please copy anything you need into your own safeguarding records (for example CPOMS). For a school without alerts on, no record is kept.

The AI is Anthropic's Claude. Every question a pupil asks is already sent to Anthropic so the tutor can teach it; the safeguarding check sends that same message to the same provider again, so it goes nowhere new.

• Anthropic receives the message only: never the pupil's name, email, school or account. Email addresses, phone numbers, postcodes, web links and social media handles are taken out of the message before it is sent. Names a pupil types themselves cannot be removed reliably.
• Anthropic does not use what we send to train its models.
• Anthropic deletes what we send within 30 days by default.
• If Anthropic's own automated safety systems flag a message, which a message about self-harm or abuse may be, Anthropic's terms allow it to keep that message for up to two years. That applies to every arrangement Anthropic offers. Because the message carries no name, email or account, it is not linked to the pupil.
• Anthropic processes data in the United States.

Anthropic's terms: API and data retention and how long Anthropic keeps organisation data.

It is not a monitoring system and does not replace the filtering and monitoring your school must have in place. It only sees what a pupil writes to the tutor, it works best in English, and it can be wrong in both directions. Please treat an alert as you would any other report, and follow your own procedures.

We can provide a data processing agreement and help with your data protection impact assessment. Email hello@thetutorai.app. Our full privacy policy covers everything else.