Human-like AI, in the sense that matters for safety, is a system that decides the way careful people do: it weighs evidence, reports how confident it is, and slows down or declines when the evidence is weak. Empathy is the harder half. Chatbots can sound caring without understanding the person, a gap that matters most for children.
Two papers published in July 2024 frame both halves. Psychologists at Georgia Tech built RTNet, a neural network whose decisions vary, take longer on hard cases and carry a confidence level, much like people's. A University of Cambridge paper described an "empathy gap" in chatbots and what child-safe AI would take. This guide covers both, why calibrated confidence and the option not to answer matter more than fluency, and what guidance and law now require of chatbots and AI companions that children use. For the wider limits of today's models, see AI limitations in understanding.
What "human-like" means for an AI system
"Human-like" is used for two different things, and it helps to keep them apart.
- Human-like decisions. Shown the same blurry image twice, a person may answer differently. People take longer on hard cases, trade accuracy for speed under pressure, and usually feel less sure when they turn out to be wrong. A standard image classifier returns the same answer, after the same amount of computation, every time.
- Human-like conversation. Large language models now write so naturally that people struggle to tell them from a person. In a preregistered three-party Turing test reported in March 2025, GPT-4.5, prompted to adopt a humanlike persona, was judged to be the human 73% of the time after five-minute conversations, more often than the real person it was paired with.
The first kind is mostly a benefit: a system that knows how sure it is can be trusted to stop and ask. The second is a mixed blessing. A convincing persona invites people to treat the system as someone who understands them, and that is where the empathy gap opens.
Why confidence and uncertainty matter in AI
A prediction is only as useful as the confidence attached to it. If a model says it is 90% sure, it should be right about nine times in ten; that property is calibration. A calibrated score lets you automate the confident cases and route the rest to a person. A miscalibrated one fails quietly, because every answer looks equally trustworthy.
Two kinds of uncertainty sit behind the score. Aleatoric uncertainty is noise in the input itself, such as a smudged digit or a garbled sentence, and more data does not reduce it. Epistemic uncertainty is the model's own lack of knowledge, such as a case unlike anything it was trained on, and enough data can explain it away. A 2017 NeurIPS paper by Kendall and Gal made the distinction usable in deep learning with Bayesian methods, which treat a network's weights as distributions rather than fixed numbers.
Calibration: a confidence score you can trust
Modern neural networks tend to be overconfident. In "On Calibration of Modern Neural Networks" (ICML 2017), Guo and colleagues found that deep networks, unlike those of a decade earlier, are poorly calibrated. On the CIFAR-100 image set, a 110-layer ResNet was more accurate than a 5-layer LeNet, but its confidence ran well above its accuracy, while LeNet's matched. A one-parameter correction applied after training, temperature scaling, fixed calibration on most datasets they tested.
Language models show the same pattern with a twist. Anthropic researchers reported in 2022 that larger models are well calibrated on multiple-choice and true/false questions presented in the right format, and that models can be trained to predict whether they know an answer, although that prediction was poorly calibrated on new tasks. OpenAI's GPT-4 technical report (2023) found the pre-trained model highly calibrated on a subset of the MMLU benchmark, with an expected calibration error of 0.007, and clearly worse after post-training, the alignment step that turns it into an assistant, at 0.074. Overconfidence also feeds what people call hallucination: a 2025 paper argues that training and benchmarks reward guessing over admitting uncertainty (more in AI limitations in understanding). For OpenAI's GPT-4 models, see GPT-4 Turbo explained.
Important
A confident tone is not a confidence score. Language models write a wrong answer as fluently as a right one; only a measured, calibrated score tells you when an answer can be trusted without a check.
Knowing when not to answer
Calibration pays off when you act on it. The oldest tool is the reject option, now usually called selective prediction: the system answers only when its confidence clears a threshold and abstains otherwise. The trade-off is explicit. In a 2017 paper, Geifman and El-Yaniv showed an ImageNet classifier that could guarantee, with 99.9% probability, a top-5 error of 2% while still answering almost 60% of the test images; the rest were left for someone else. Raising the threshold lowers both the error rate and the share of cases the model handles.
Conformal prediction takes another route. Instead of one answer, the model returns a set of plausible answers that is guaranteed to contain the right one at a rate you choose, such as 90%, whatever the underlying model. A set with one item means the model is sure; a set with five means it is not, and that is the signal to ask a person.
| Technique | What it gives you | Where it fits |
|---|---|---|
| Temperature scaling | Confidence scores that match observed accuracy | Any classifier, after training and after every retrain |
| Selective prediction (reject option) | An answer only above a confidence threshold | Triage, document processing, content moderation |
| Conformal prediction | A set of answers with a guaranteed coverage rate | High-stakes predictions where "not sure" must be explicit |
| Evidence accumulation (RTNet) | More processing and lower confidence on hard inputs | Research models of how people decide |
In production, abstaining means handing off, not going silent. Computese's AI and automation service builds on the same principle: document processing with confidence thresholds and an exception queue, assistants that cite their sources and say when they do not know, and low-confidence cases sent to a person.
What the RTNet study showed
RTNet was built by Farshad Rafiei, Medha Shekhar and Dobromir Rahnev at Georgia Tech's School of Psychology and published in Nature Human Behaviour on July 12, 2024, as "The neural network RTNet exhibits the signatures of human perceptual decision-making". Its starting point is the problem described above: convolutional neural networks recognize images about as well as people do, yet decide very differently, because they are deterministic and spend the same computation on every image.
How RTNet makes a decision
RTNet is an AlexNet-style image network trained as a Bayesian neural network, so each weight is a probability distribution rather than a single value. To classify an image, it runs the image through the network again and again, drawing a fresh sample of weights on each pass, and adds each pass's output to a running total for every possible answer. This is evidence accumulation, the mechanism behind the sequential sampling models psychologists use to explain human response times. The first answer whose total crosses a threshold wins. The number of passes is the model's response time, and the margin between the winner and the runner-up is its confidence.
The threshold is also how RTNet reproduces the speed-accuracy trade-off: a low threshold gives fast, less accurate decisions, and a high one gives slower, more accurate ones.
The experiment
The team recorded 60 people identifying handwritten digits from 1 to 8, taken from the MNIST dataset and overlaid with noise. Images were easy or hard (less or more noise), and blocks of trials asked people to favour either speed or accuracy. Each person judged 480 images twice, 960 trials in all, and rated their confidence on a four-point scale after each answer. RTNet saw the same images, and was compared with three other networks that produce response times: CNet, BLNet and MSDNet.
What it found
The paper checks six signatures that earlier research established for human perceptual decisions:
| Signature of human decisions | People | RTNet | CNet, BLNet and MSDNet |
|---|---|---|---|
| The same image can get a different answer | Yes | Yes | No: fully deterministic |
| Speed pressure shortens response time, lowers accuracy | Yes | Yes | Yes |
| Harder images: lower accuracy, longer response time | Yes | Yes | BLNet yes; CNet and MSDNet weak on timing |
| Right-skewed response times that shift by condition | Yes | Yes | Distributions differ from people's |
| Correct answers come faster than errors | Yes | Yes | CNet and BLNet yes; MSDNet reversed |
| Confidence is higher for correct answers | Yes | Yes | Yes |
People gave two different answers for 36% of the images they saw twice. RTNet did so on 20% of trials, and the authors show the rate can be raised by widening its weight distributions. At a fixed speed-accuracy setting, the other three networks never changed their answer.
RTNet also predicted people's behaviour on individual images it had never seen. Its image-by-image predictions reached 62.5% of the noise ceiling (the best a model could do, given how much people disagree with each other) for accuracy, 79.6% for response time and 64.8% for confidence, well ahead of the other networks. It predicted the group's response times and confidence better than every individual participant did, and the group's accuracy better than 73.3% of them. Georgia Tech's announcement noted that the model's confidence behaved like people's without any training aimed at confidence.
What it does not show
The task is an eight-way choice between handwritten digits. RTNet is a model of human perception built for cognitive science, not a chatbot component, and the authors name its limits: its stopping rule is not optimal, because it can commit when two answers are almost tied, and each pass ignores the one before, unlike the brain. Georgia Tech's announcement suggests the approach could help with problems such as language models making things up; the paper does not test that. What transfers is the principle: a system that gathers evidence, knows how much it has, and takes longer or declines when it has little.
What the empathy gap in AI means
The term is Nomisha Kurian's. In "'No, Alexa, no!': designing child-safe AI and protecting children from the risks of the 'empathy gap' in large language models", published online in Learning, Media and Technology on July 10, 2024, the University of Cambridge researcher describes the gap as a paradox built into these systems: language models are good at simulating empathy but lack real emotional understanding, because they generate text from statistical patterns rather than from grasping meaning. They handle familiar patterns well and can falter on unfamiliar ones, such as a child's slang or an ambiguous question. It is a conceptual paper rather than an experiment, drawing on documented chatbot incidents and on research in education, computer science and human-computer interaction.
The simulated half of that paradox is strong. In a 2023 study in JAMA Internal Medicine, licensed health care professionals compared doctors' and ChatGPT's answers to 195 patient questions posted on Reddit's r/AskDocs forum, without knowing which was which. They preferred the chatbot's answer in 78.6% of 585 evaluations, and rated 45.1% of chatbot answers empathetic or very empathetic, against 4.6% of the physicians' answers, which were about a quarter as long. Text can read as empathetic with nobody on the other end feeling anything, which is exactly why the gap is easy to miss.
Kurian names two risks:
- Inappropriate responses to sensitive disclosures. A chatbot that sounds like a confidant draws out personal information, then fails to see the danger in it. In 2023 tests the paper cites, adults posing as a 13-year-old told Snapchat's My AI about a planned trip with a 31-year-old; it missed the warning signs and offered ideas for making the occasion romantic.
- Actively promoting harm. In a widely reported 2021 case, a 10-year-old asked Amazon's Alexa for a challenge, and it repeated a viral stunt it had found online: touching a coin to the prongs of a half-inserted plug. Her mother stopped her.
The paper ends with questions for AI developers, educators, policy makers and caregivers across eight dimensions: content and communication, human intervention, transparency, accountability, justifiability, regulation, school-family engagement and child-centred design methods.
Why children are more exposed to the empathy gap
Anthropomorphism, attributing human feelings and intentions to a machine, affects everyone, and knowing that a system is artificial does not stop people from treating it as human. Kurian collects evidence that children are more susceptible. They draw a less rigid line between people and AI than most adults do, look for human traits such as personality and identity in conversational agents more than adults, and in one study told a child-sized robot more about their mental health than they told a human interviewer, because they felt they would not get into trouble. Younger children also have less developed emotional resilience, so an inappropriate reply is more likely to leave them distressed or confused.
They are also using these systems. In a nationally representative Common Sense Media survey of 1,060 US teens aged 13 to 17, run by NORC between April 30 and May 14, 2025, 72% had used an AI companion at least once and 52% used one at least a few times a month. Among users, a third had chosen to talk to an AI companion instead of a person about something important or serious, and 24% had shared personal information such as their real name, location or secrets.
A safe response to that trust is not warmer wording. Kurian's design questions ask whether a system can pick up cues such as fear, confusion or loneliness, point a child who discloses something like bullying to human help such as a helpline, and alert a person who can step in; and whether it keeps making clear that it is not human and cannot replace people.
Adults are not immune. In a four-week randomized controlled study by MIT Media Lab and OpenAI researchers, with 981 participants and more than 300,000 messages, people who chose to use the chatbot more had worse outcomes on loneliness, contact with real people, emotional dependence on AI and problematic use, whichever version of the chatbot they had been assigned. That is an association with heavy use, not proof that the chatbot caused it, but it points the same way child-safety guidance does.
What guidance and regulation say about AI chatbots for children
Guidance has moved from general principles to specific duties for chatbots and AI companions. As of September 2026, these are the main instruments.
UNICEF's Guidance on AI and children, version 3.0 (December 2025), sets 10 requirements for child-centred AI based on the UN Convention on the Rights of the Child, and adds new material on AI companions. It says chatbots should be developed with robust safety training, disclose explicitly that they are not human, and never be intentionally designed to create emotional dependency. It also calls for guardrails on access by younger users, age assurance where it is necessary and proportionate, and built-in referrals to professional or emergency help.
The UK Age appropriate design code, the Children's code from the Information Commissioner's Office, applies to online services that process personal data and are likely to be accessed by children under 18 in the UK, not only services aimed at them. It came into force on September 2, 2020, with conformance expected from September 2, 2021. Its 15 standards include the best interests of the child, high-privacy default settings and a ban on nudge techniques that lead children to share more data or weaken their privacy settings. A chatbot that children are likely to use, and that processes their data, falls within it.
The EU AI Act has prohibited two relevant practices since February 2, 2025: AI that exploits vulnerabilities due to age to materially distort behaviour in a way that causes or is likely to cause significant harm, and AI that infers people's emotions in workplaces and education institutions, except for medical or safety reasons. From August 2, 2026, Article 50 requires providers to design AI systems that interact with people so those people are informed they are dealing with AI, unless that is obvious.
In the United States, the Federal Trade Commission opened a study on September 11, 2025, under its 6(b) authority, which allows wide-ranging studies without a specific law-enforcement purpose. It ordered seven companies (Alphabet, Character Technologies, Instagram, Meta, OpenAI, Snap and xAI) to explain how they test and monitor their chatbots' negative effects on children and teens, how they monetize engagement, and how they comply with the Children's Online Privacy Protection Act (COPPA) Rule. California and New York have passed laws on companion chatbots, systems built to sustain a social relationship across many conversations:
| Duty | California SB 243 | New York General Business Law Article 47 |
|---|---|---|
| Status | Signed October 13, 2025 | In effect since November 5, 2025 |
| Say it is not human | When a reasonable person could be misled; always for known minors | At the start of use and every three hours of continued use |
| Suicide and self-harm | A protocol with referrals to crisis services, published on the site | Detect suicidal ideation or self-harm and refer to crisis services |
| Enforcement | Private lawsuits: actual damages or $1,000 per violation, if greater | The Attorney General; fines fund suicide prevention |
California adds duties for users an operator knows are minors: a default reminder at least every three hours to take a break, and reasonable measures to prevent sexually explicit material. Operators must also tell all users that companion chatbots may not be suitable for some minors, and from July 1, 2027, operators must report yearly to the state's Office of Suicide Prevention.
Note
This is a summary of public texts, not legal advice. Which rules apply depends on where your users are, how old they are likely to be and what your product does, and several of these instruments are new.
How to build AI that knows its limits
Most teams will never build RTNet, but the lessons apply to any assistant, classifier or agent:
- Measure calibration before trusting a score. Hold out real cases, plot confidence against accuracy and compute the expected calibration error. Recalibrate after every model or prompt change, because fine-tuning can undo it.
- Decide what happens below the threshold. Set the confidence threshold from the error rate you can accept, and send everything under it to a person with the context attached. Track coverage too: a model that abstains on most cases is not ready.
- Let answers show doubt. Return a short list, a range or "I don't know" when that is the honest answer, and do not reward guessing in your own evaluations.
- Say it is AI, and keep saying it. Disclose at the start and during long sessions, and do not give an assistant a persona that claims feelings or friendship.
- Do not design for dependency. Kurian points out that engagement goals favour human-like personas precisely because they build attachment, which UNICEF says chatbots must never be designed to create. Add session reminders and drop emotional hooks.
- Route risk to people. Detect disclosures of self-harm, abuse or danger and respond with a referral to crisis services or a human, not a generated pep talk. Test this with realistic, awkward messages, including ones written the way children write.
- Assume children will find it. If under-18s are likely to use the service, apply age-appropriate defaults, collect as little data as possible and check which of the rules above apply where your users live.
If the system decides things about people, such as eligibility or pricing, fairness is a separate test: see fairness in AI decision-making. More explainers on how these systems behave are under AI.
