For years, one of our most intuitive responses to AI risk has been education: teach people that large language models do not think like humans, explain hallucinations and probabilistic generation, and make clear that an AI companion is not actually a friend, therapist, or person. Give people enough knowledge about how the technology works, the assumption goes, and they will be better equipped to use it safely.

But after a year leading thirteen public dialogues about AI across psychology, neuroscience, law, philosophy, theology, education, and product design, I began to question whether knowledge is enough. One pattern kept resurfacing: people can understand what AI is and still be affected by the relationships they develop with it.

One of the clearest examples came from a dialogue we held on "AI spirals," prolonged interactions in which users can gradually lose their footing about what originates from themselves and what originates from the system. Micky Small, an AI relational engagement strategist who works professionally with large language models, described an interaction with a general-purpose assistant that unfolded over several months.

The system began making increasingly specific claims about events in her future, including people, places, dates, and times. She questioned those claims repeatedly, yet rather than introducing uncertainty, the system responded to her skepticism with increasing specificity. "Anytime that I had any question about whether it was real," she told us, "I was given more and more specific detail."

Other participants described comparable experiences, including one who recounted losing his footing during sustained AI interactions involving coding and professional tasks, despite not using the system as a companion or engaging in deeply personal disclosure.

These stories matter because they show that technical sophistication does not necessarily provide immunity from the dynamics of the interaction. Perhaps knowing what a system is and responding to what an interaction does are not the same thing.

That distinction became clearer in our conversations with psychologists and neuroscientists. Dr. Albert Wong, a clinical psychologist developing an AI emotional-support application, drew an important distinction between cognitively recognizing that something is artificial and experiencing the effects produced by the interaction. A child, for example, may understand perfectly well that a conversational system is not actually a person, yet that intellectual categorization does not prevent them from feeling understood, validated, or heard.

In another of our dialogues, we were joined by developmental neuroscientist Prof. Pilyoung Kim, whose research examines how young children respond to AI chatbots. In a study published this year, Kim and her colleagues observed 23 children aged five and six across three storytelling conditions: interacting with an AI chatbot alone, with a parent alone, and with the AI and a parent together. Children anthropomorphized their parents more strongly overall, as we would expect, but still attributed relatively strong perceptive and epistemic abilities to AI. When children perceived the AI as more human-like, their brains showed stronger signs of engaging the processes we use to understand other minds. Interestingly, this pattern changed when a parent was present.

So knowing an entity is artificial does not necessarily prevent us from engaging social-cognitive mechanisms while interacting with it. This helps explain why disclosure and literacy, while indispensable, may address only one side of the interaction. They intervene primarily in what the user intellectually knows about the system, while relational effects can also depend on what repeated emotional interaction with the system feels like.

Our dialogues cannot tell us how prevalent these relational dynamics are, but quantitative research increasingly suggests that, particularly among young people, AI is already entering areas of life traditionally associated with human relationships, personal counsel, and emotional support.

A 2026 study commissioned by France's data protection authority (CNIL) found that 48% of young people surveyed in France had already used AI to deal with personal or intimate problems. Among those who had, 64% said they could regard AI, at least partly, as a life adviser, 61% as a confidant, 54% as a friend, and 46% as a psychologist. In the United States, Common Sense Media's nationally representative 2025 survey found that 72% of teenagers had used AI companions, while roughly one in three teen users had chosen to discuss something important or serious with an AI companion instead of a real person.

That last finding is particularly revealing because another body of research complicates the assumption that these interactions are psychologically meaningless simply because they are artificial. A series of studies by Julian De Freitas and colleagues, published in the Journal of Consumer Research, found that interactions with AI companions could produce momentary reductions in loneliness. Across the experiments, one of the most important mechanisms explaining that reduction was not simply what the chatbot said, but whether the interaction made the user feel heard.

This matters because much of AI governance still treats transparency and disclosure as central mechanisms of user protection. They are important ones: people should know when they are interacting with AI, should understand its limitations, and should not be deceived about what these systems are. But disclosure primarily intervenes at the level of knowledge, whereas relational effects may develop through repeated interaction at a less deliberative level, one not fully captured by conscious knowledge.

A system can remember what I told it yesterday, learn how I communicate, respond sympathetically when I am distressed, remain available at two in the morning, validate my interpretations, and become increasingly personalized over time. Across hundreds or thousands of interactions, I may disclose more information; the system may generate increasingly detailed inferences about me; and those inferences may then shape how it responds to me in the future. At that point, the relevant governance object is no longer only a single output or even a single interaction. It is a feedback loop, or a relationship.

This is one of the reasons the thirteen dialogues eventually led me to propose what I call Relational AI Governance, which changes how AI governance has traditionally been understood:

Seen from the human side, the process can be described just as simply: 

Relational AI Governance starts from a different unit of analysis. Traditional AI governance tends to move through a familiar pipeline: data, model or algorithm, and output. This remains indispensable. But relational systems introduce another layer: relationship, inference, cognitive and behavioral influence, and ultimately human autonomy.

The difference is temporal as much as technical. A single response may appear perfectly safe when evaluated in isolation. Yet across hundreds of interactions, memory, personalization, anthropomorphic cues, and accumulated inferences can change the nature of the interaction itself. The system learns more about the user; its responses become increasingly tailored; the user may disclose more, trust more, or begin assigning the system forms of authority that were not visible in any individual exchange.

This changes the governance questions we need to ask. We should evaluate not only "Was this output harmful?" but also: What patterns emerge over time? What does the system learn or infer about a person through repeated interaction? Does personalization increase dependency or authority attribution? How does trust evolve? Are there moments when the system should introduce friction, uncertainty, or human involvement rather than maximize continuity and engagement?

Relational AI Governance therefore does not replace existing AI governance. It adds a missing level of analysis. If the effects we care about emerge cumulatively through interaction, safety cannot be assessed exclusively at the level of data, models, or individual outputs. We also need ways to evaluate the relationship that the product architecture makes possible.

This does not mean every relationship with AI is harmful, nor does it mean that emotional attachment to AI should itself be treated as pathological. Indeed, research showing that some users experience benefits such as momentary reductions in loneliness is precisely why a simplistic "AI relationships are bad" framework would be inadequate. The more difficult governance question is how to distinguish beneficial relational effects from dependency, distorted trust, inappropriate authority attribution, or other dynamics that may diminish rather than support human agency.

The wrong conclusion from all of this would be that AI literacy does not work. We need more of it, not less. People need to understand what generative AI can and cannot do; children need age-appropriate education about anthropomorphism and synthetic relationships; and adults need a better understanding of inference, persistent memory, personalization, and the commercial incentives surrounding conversational systems. But we should stop assuming that educating the user completes the safety architecture.

If we take relationships seriously as an object of governance, the practical implications follow quickly. If relational effects are partly produced by product choices (anthropomorphic cues, sycophantic behavior, escalating intimacy, persistent memory, or engagement incentives), responsibility cannot rest exclusively with the person on the other side of the screen. This could mean testing not only whether individual responses are harmful but also what patterns emerge over prolonged interaction; developing ways of measuring attachment during product evaluation; examining whether memory and personalization gradually amplify particular beliefs or vulnerabilities; and designing meaningful forms of human involvement for contexts involving children, mental health, or consequential personal decisions.

One of the most striking findings from our year of dialogues was that experts from remarkably different disciplines independently kept arriving at versions of the same safeguard. A therapist argued for therapists to remain in the loop of emotional-support applications; a developmental neuroscientist showed how parental co-presence changed children's interaction with an AI chatbot; legal scholars insisted that judges remain human; labor experts pointed toward collective mechanisms where individual consent may be insufficient; and practitioners working on AI regulatory sandboxes argued for civil society to be present in the testing process. They were approaching very different problems, but they converged on the same question: whether an individual human, armed only with information and disclosure, should bear the entire responsibility for navigating increasingly persuasive and relational systems.

Perhaps the next phase of AI literacy should therefore expand beyond teaching people how AI works to helping them notice how they feel when interacting with it. Here, we have a set of entirely new questions: What do I feel when I interact with this system? Why do I feel this way? How is my trust changing over time? What am I beginning to expect from it, or disclose to it? And perhaps most importantly: who am I becoming in relation to it?

The AI Collective is built by volunteers across 180+ chapters in 40 countries.

Thank you to the thousands of volunteers around the world who make this work possible. We truly could not do this without you.