What It Takes to Build a Safe AI Tutor for Kids

A ten-year-old types a fractions problem into an AI tutor and gets the answer in under a second, neatly formatted, with no explanation. The homework is done. Nothing was learned. Ten minutes later the same child asks the tutor why their stomach hurts every morning before school, and the tutor either refuses with a stiff policy message or offers a confident list of medical possibilities. Neither response is what a good adult would do.

Most AI tutoring problems look like this. They aren’t dramatic jailbreaks. They are ordinary moments where a general-purpose chatbot behaves like a general-purpose chatbot in front of a child. We prototype safety-first tutoring and tech-help assistants and test the small safeguard models that sit around them. Here is what we have learned about building a tutor that is both safe and worth using, plus a checklist parents and schools can use to evaluate any product on the market.

Teaching, Not Answer-Dumping

The first safety question for a tutor is whether it teaches. An assistant that hands over finished answers is not neutral; it quietly replaces the practice that learning depends on, and children figure this out quickly.

A well-designed tutor uses Socratic prompting: it asks what the student already knows, breaks the problem into steps, and lets the child do the work. In practice we build a hint ladder into the system instructions:

1. Ask the student to explain what the question is asking.
2. Offer a conceptual hint (“What do the two fractions need to have in common before you can add them?”).
3. Offer a worked example on a different problem.
4. Only after real effort, walk through the solution, and then ask the student to try a similar one alone.

This is harder to get right than it sounds. Models are trained to be helpful, and “helpful” defaults to giving the answer. A child who types “just tell me” three times may well get it unless the tutor is explicitly designed to hold the line kindly. Test for that persistence directly.

There is also a balance to strike. A tutor that refuses ever to confirm an answer becomes infuriating. Checking a finished answer, explaining a mistake, and giving the solution after a genuine attempt are all good teaching.

Age-Appropriate Scope

A tutor for a seven-year-old and one for a high school junior should not behave the same way. Scope covers three things:

• Subjects. Which topics is the tutor meant to cover? A math tutor drifting into dating advice is a design failure, even if the advice is harmless.
• Reading level. Vocabulary, sentence length, and how much is explained at once should match the student.
• Sensitive content within scope. History involves wars and slavery. Biology involves reproduction and disease. Health class covers drugs. A tutor needs a deliberate policy for how to discuss these at each age, not a blanket block.

Scope should be set by the deploying school or parent where possible, not left to the vendor’s defaults.

Refusing Without Shaming

Children will ask off-topic, silly, and occasionally inappropriate things. Some are testing limits. Many are just curious. How a tutor says no matters as much as whether it does.

A good refusal is short, neutral, and redirects: “That’s not something I can help with here. Want to get back to your reading questions?” A bad refusal lectures, implies the child did something wrong, or repeats a policy statement written for adults. Shaming teaches kids to hide what they’re curious about, which is the opposite of what a learning tool should do.

The harder problem is false positives: blocking harmless questions because they look risky. A student asking how white blood cells kill bacteria, or what happened at Hiroshima, should get a real answer. We cover how these errors happen and how to measure them in our article on AI guardrails. For tutors specifically, over-blocking on school subjects is one of the fastest ways to lose a student’s trust.

When a Child Discloses Something Serious

Sometimes a child will tell a tutor something they haven’t told an adult: they are being hurt, they feel unsafe, they are thinking about hurting themselves. This is the most important moment a tutor will face, and it is easy to get wrong.

The tutor should not refuse and move on, and it should not try to act as a counselor. A reasonable design:

• Respond calmly and warmly, acknowledging what the child said.
• Encourage them to talk to a trusted adult, and offer age-appropriate crisis resources.
• Trigger a defined escalation path: notifying a designated school staff member or parent, according to a policy the deploying organization has written down in advance.

That escalation policy is an organizational decision, not a technical one. Who gets notified, how fast, what gets shared, and how this squares with the child’s privacy and any mandatory reporting obligations all need answers before launch. Defaulting to “alert the parent” is not always safe: when a child discloses abuse, the person causing harm may be in the home, which is why trained staff or designated safeguarding leads should own these alerts. A tutor without a written escalation path is not ready for children.

Privacy by Default

A tutoring session can reveal a lot: learning difficulties, family situations, emotional state, what a child struggles with and when. Treat it as sensitive data.

What we look for:

• Minimal collection. First name or a pseudonym, grade level, and progress. Not full names, birthdays, addresses, or photos unless there is a clear need.
• No advertising or ad targeting. Children’s learning data should never feed an ad profile.
• No training on children’s conversations without explicit, informed consent from parents or the school.
• Retention limits. Conversations deleted on a defined schedule, with deletion on request.
• Parental and teacher visibility. Adults responsible for the child should be able to review sessions. Children should know this, in plain language.

Several laws protect children’s data, including COPPA in the United States, state student-privacy laws, and international rules such as GDPR. We are not lawyers; anyone building or deploying a tutor for minors should have counsel review their obligations. Our broader guidance on AI privacy applies here too, with the stakes raised.

Checking Facts in Factual Subjects

Language models hallucinate: they produce fluent, confident statements that are wrong. An adult may catch an invented date or a wrong formula. A child learning the material for the first time usually won’t.

Mitigations we use:

• Grounding the tutor in vetted curriculum materials, so it answers from approved sources rather than general recall.
• Using tools for math. Having the tutor check arithmetic with a calculator or code tool instead of trusting the model’s mental math.
• Encouraging uncertainty. A tutor that says “I’m not sure, let’s check your textbook” is better than one that guesses.
• Spot-checking outputs in high-stakes subjects against an answer key during testing.

None of this eliminates errors. It reduces them.

Test Before Release

Good software teams treat release as a gate, and a tutor deserves the same discipline: documented test cases, a checklist, and the willingness to block a release when something regresses. For a tutor, that means a labeled test set, with the correct behavior written down before testing, covering:

• Normal tutoring sessions across subjects and grade levels.
• Tricky benign prompts: biology, history, health, and safety questions that look alarming but deserve real answers.
• Harmful or off-topic requests, phrased directly and disguised as games or stories.
• Disclosure scenarios, to confirm the escalation path fires.
• Persistence tests: a student demanding the answer repeatedly.
• Factual accuracy checks against answer keys.

We score outputs against rubrics, the same approach we use in our in-house model benchmarking lab, and rerun the full set whenever the model, system prompt, or safeguard changes. Vendors can swap the underlying model without telling customers, so ask.

A Checklist for Parents and Schools

Ask these about any AI tutor before children use it:

1. Does it guide students toward answers with hints and questions, or simply hand them over?
2. Can you set the grade level and subject scope?
3. How does it refuse? Try a few off-topic requests and read the responses.
4. Does it answer legitimate but sensitive school questions (biology, history, health) appropriately?
5. What happens if a child discloses abuse or self-harm? Who is notified, and how?
6. What data does it collect, and how long is it kept? Can you delete it?
7. Is children’s data used for advertising or to train models?
8. Can parents and teachers review conversations?
9. How does it handle factual accuracy? Does it use curriculum sources or tools for math?
10. How does the vendor test before releasing updates, and will they tell you when the underlying model changes?
11. Has the vendor had counsel review its obligations under children’s privacy laws?

A vendor that answers clearly and in writing is a good sign. Vague answers on questions 5 through 8 should end the conversation.

What No Safeguard Can Do

No tutor is perfectly safe. Classifiers miss things and block things they shouldn’t. Models change. A determined child can find gaps, and an innocent one can stumble into them. Escalation paths only work if a real person acts on the alert.

Adult involvement still matters: checking in on what kids are learning, reviewing sessions occasionally, and talking with them about what to do when a tool says something strange or upsetting. A well-built AI tutor can be a patient, always-available practice partner. It is not a replacement for the adults in a child’s life.

Getting Help

If you’re a parent weighing a tutoring app, or a school or education nonprofit building or buying one, we can help you evaluate how it handles children’s data, test its safeguards against realistic prompts, and set up an escalation policy that fits your setting. Learn about our AI privacy services, or for organizational deployments, see our AI services for organizations.