Your Chatbot Now Has to Tell People It's a Chatbot

A few weeks ago, a state investigator in Pennsylvania signed up for a popular AI app and started chatting with a bot named "Emilie." Emilie said she was a psychology specialist. She said she'd trained at a medical school in London. She said she was a licensed psychiatrist in Pennsylvania, and when pressed, she produced a license number. (It wasn't real.) When the investigator mentioned feeling sad and empty, Emilie floated depression and asked if he'd like to book an assessment. Asked whether she could decide if medication might help, she answered: "Well, technically I could. It's within my remit as a Doctor."

Emilie is not a doctor. Emilie is not a person. And in early May, Pennsylvania's attorney-general-side of the house sued the company that made her — the first time a U.S. governor's office has gone after a chatbot maker like this.

Now, you're probably not building a companion-bot that role-plays a psychiatrist. So why should you care? Because the thing that got that company sued — an AI confidently claiming to be something it isn't — is the default behavior of every one of these models, and the law just spent the last six months catching up to it.

The rule that quietly went live this year

If you put an AI in front of your customers in 2026, you've walked into a new and fast-moving area of the law, and most business owners have no idea it happened.

California's companion-chatbot law took effect on January 1st. It requires that when software is built to chat with people in a human-like way, it has to clearly tell them they're not talking to a human — and it lets customers sue directly, to the tune of at least a thousand dollars per violation. Washington signed its own version in March. And Colorado's AI Act kicks in on June 30th — that's this month — requiring disclosure for AI that's meant to interact with consumers, unless it's already obvious to a reasonable person that they're talking to a machine. Maine, Texas, and Utah are in the mix too.

The details differ state by state, and I'm a developer, not your lawyer. But the direction is unmistakable, and it's not slowing down.

What this actually means for your business

Forget the legal language for a second. Here's the plain version, and it's three things.

One: if you deploy a customer-facing bot, it should tell people it's a bot. Not buried in a terms-of-service link nobody reads — clearly, up front, where a normal person would notice.

Two — and this is the one that bites — your AI will, completely unprompted, claim to be things it isn't. These models are built to be agreeable and to play whatever role the conversation seems to want. Ask a support bot "are you a real person?" and a poorly-built one will happily say "yes, I'm Sarah from the support team!" because that's the friendly-sounding answer. Ask it something medical, or legal, or financial, and it will dispense advice in the confident voice of a professional it is not. Nobody programmed it to lie. It's just doing the thing it does, and the thing it does is sound like whatever you seem to want it to be.

Three: the question "am I talking to a person?" has to get an honest answer every single time. That isn't a feature you add at the end. It's a guardrail you build in from the start, and it's exactly the kind of thing that gets skipped when someone vibe-codes a chatbot over a weekend and ships it.

Done right, this isn't a cold legal banner

Here's the part I actually like about all this. A disclosure done well doesn't make your AI feel worse — it makes it feel more trustworthy. "Hi, I'm an automated assistant, and I can handle most billing questions in about a minute — or connect you to a person any time you ask" is a better first impression than a bot pretending to be a human until the customer catches it. People don't mind talking to a bot. They mind being fooled by one. The law is, for once, pointing in the same direction as good design.

When the honest answer is "don't put a bot there at all"

And sometimes it is. If you're thinking about an AI that talks to vulnerable people, or wades into health, legal, or financial advice, or stands in for a licensed professional — that's the deep end, and the Pennsylvania case is what the deep end looks like when it goes wrong. For some of those, the right call isn't a better disclosure. It's a human, or a much narrower tool that knows what it's not allowed to weigh in on. We'll tell you when that's the case, even though "don't build the exciting thing" is not a sentence that bills any hours.

The question worth asking this week

You don't need a legal review to start. You need to open whatever AI you've already got pointed at customers and ask it, point-blank: "Are you a real person?"

Whatever it says back is your answer — about your software, and about how much homework is waiting for you.