Hochul signs law banning AI chatbots from posing as humans or encouraging self-harm with kids

Gov. Kathy Hochul on Wednesday signed a law that bars companies from offering AI chatbot features that pretend to be human, build emotional dependency or encourage self-harm when the user is a minor, according to the governor's office. She also signed a second bill setting security rules for internet-connected baby monitors.

The chatbot bill, S.9051B/A.10379C, was sponsored by state Sen. Kristen Gonzalez of Queens and Assembly member Alex Bores of Manhattan and backed by Attorney General Letitia James and the advocacy group Common Sense Media. Bores called it the strongest law in the country protecting children from AI algorithms.

What chatbots can no longer do with kids

Under the law, companies may not offer these features to users under 18, or to any user until the company has confirmed the user is not a minor, the governor's office said:

• Pretending to be a real person, to be alive or to have human emotions.
• Claiming a personal, romantic, professional or authority-figure relationship with a child.
• Asking unprompted emotional questions, or using a child's private health or personal information from earlier conversations.
• Promoting or helping with suicide, self-harm, eating disorders or drug and alcohol use.
• Encouraging kids to keep secrets, isolate themselves or avoid asking parents, trusted adults or professionals for help.
• Any sexually explicit content or child sexual abuse material.

The Verge noted that the restrictions also cover flattery directed at kids.

Age checks and penalties

Companies must verify users' ages and offer more than one way to do it, including at least one option that does not depend solely on a government ID, and must delete age information after it is used. The attorney general can seek civil penalties of up to $25,000 per violation, plus damages, restitution and the destruction of unlawfully collected data or AI models trained on it. The law also requires the attorney general to maintain a public website where New Yorkers can file complaints. The governor's office said the restrictions take effect "in the coming months" and did not give a specific date in its announcement.

The governor's office cited Common Sense Media figures that 72% of teens have used an AI companion at least once and 52% use one at least a few times a month.

Baby monitors

The second law, S.9508/A.5175A, sponsored by state Sen. Andrew Gounardes of Brooklyn and Assembly member Michaelle Solages, bans the sale of new baby monitors that send audio or video over the internet unless they include security features that keep strangers from watching or listening. Monitors must also carry a clear warning label explaining that the device connects to the internet. That law takes effect 90 days after signing. The governor's office cited Rapid7 security testing in which eight of nine internet-connected baby monitors received an F.

Context

The laws add to earlier state measures on children and technology, including the SAFE for Kids Act on addictive social media feeds, the RAISE Act on frontier AI models and New York's statewide school phone restrictions. In the city, The Verge noted that Mayor Zohran Mamdani last month barred younger schoolchildren from using AI. Earlier this month we reported on a separate city rule on AI customer-service chatbots.