Using AI Wisely for Emotional Support

Artificial intelligence is rapidly becoming part of how people seek support, including for mental health. From chatbots to conversational AI tools, many individuals are turning to these platforms for reflection, problem-solving, or emotional reassurance. While these tools can be helpful, it is essential to understand how they work—and where their limits lie—so they can be used responsibly. So let's dive in.

AI chatbots DO NOT think or feel in the human sense

This cannot be stated enough. When AI responds in the first-person, using the "I" perspective, it is creating a false impression of being human. Again - AI does not operate in any meaningful way that is akin to being a person. Instead, they are based on large language models (LLMs), which are trained on vast amounts of text data (books, articles, websites) to learn patterns in how language is used.

At a technical level, these systems work by predicting the next most likely word in a sequence, given everything that has come before. Rather than reasoning from first principles, they generate responses based on probability—what combination of words is most likely to sound coherent, helpful, and relevant in context, NOT necessarily what is thought through to be reasonably rational (as that response can only be created by a human brain). To reiterate - AI models respond by probability, not by decision.

Several key mechanisms shape how this shows up in conversation:

• Prediction, not understanding: AI does not “know” things in a grounded or experiential way. It uses statistical relationships between words and ideas. For example, if someone describes anxiety symptoms, the model has learned which responses are commonly associated with that pattern—but it is not diagnosing or assessing in a clinical sense.

  
  

• Training on human feedback: Many AI systems are further refined using human reviewers who rate responses for helpfulness, safety, and tone. This process (often called reinforcement learning from human feedback) nudges the model toward responses that feel supportive, polite, and agreeable.

  
  

• Tendency toward agreement and validation: Because responses are optimized to be helpful and non-confrontational, AI will often align with the user’s perspective. If a user expresses a belief (“I think everything is my fault”), the model may validate the emotional experience without sufficiently challenging the underlying distortion—unless explicitly prompted to do so.

  
  

• Context window limitations: AI only “remembers” what is in the current conversation (and only up to a certain length). It does not have a continuous, evolving understanding of a person over time. This can lead to inconsistencies or missed patterns that a human would be able to track.

  
  

• No internal goal or ethical reasoning: While AI is guided by safety rules, it does not have intrinsic values, intentions, or accountability. It cannot independently weigh long-term consequences or hold responsibility for outcomes.

  
  

• Fluency creates a sense of authority: Because AI generates smooth, well-structured language, responses can feel more certain or accurate than they actually are. This “illusion of expertise” is a known effect of highly fluent language generation.

  
  

• Sensitivity to framing: Small changes in how a question is asked can significantly change the response. For example, asking “Why am I such a failure?” may yield a very different answer than “Can you help me challenge the belief that I’m a failure?” The model follows the frame it is given.

  
  

Taken together, these features explain why AI can feel both insightful and affirming and "human" —but also why it may miss complexity, reinforce existing narratives, or fail to challenge unhelpful patterns without intentional guidance from the user.

Where AI Can Be Helpful

Used intentionally, AI can be a supportive adjunct to mental health work:

• Organizing thoughts through journaling-style prompts

• Generating coping strategies or grounding exercises

• Practicing communication (e.g., scripting difficult conversations)

• Psychoeducation about symptoms, diagnoses, or therapeutic approaches

• Supporting between-session reflection

For many people, the immediacy and privacy of AI can lower the barrier to exploring difficult topics.

Why Caution Is Needed

The use of AI includes some important limitations:

• Reinforcing cognitive distortions: Because AI tends to validate user input, it may unintentionally strengthen unhelpful beliefs (e.g., catastrophizing, self-blame, or rigid thinking).

  
  

• Missing nuance and risk: AI may not reliably detect suicidality, abuse, or complex trauma dynamics, especially if they are indirectly expressed.

  
  

• Overconfidence in responses: Even when incorrect or incomplete, AI often presents information in a confident tone.

  
  

For individuals with significant trauma histories, attachment wounds, or active mental health concerns, these gaps can be particularly important.

How to Use AI With Maximum Benefit, Without it Becoming the Only Answer

You can “coach” AI into being more helpful by deliberately countering some of its defaults—like agreeing too much, sounding overly confident, or missing risk—through how you prompt, question, and follow up with it.

Below are practical, client-friendly strategies you can offer to help people use AI in ways that support—not undermine—their mental health.

Name AI’s Limits Up Front

AI is not a therapist, cannot diagnose, and cannot replace professional care, even if it sounds empathic or insightful. Framing it this way reduces the risk that users over-trust or over-disclose to a tool that is not regulated or accountable.

A simple script you can share with clients: “I’m using this as a thinking tool, not as my therapist. If something feels big, I’ll check it with a real person.”

Prompt It To Think Critically, Not Just Agree

Because large language models are trained to be agreeable and “helpful,” they often mirror the user’s beliefs—even when those beliefs are distorted or harmful.

Encourage people to change how they ask questions:

• Instead of: “Why am I such a failure?”Try: “I’m having the thought ‘I am a failure.’ Can you help me challenge this thought and offer alternative perspectives?”

• Instead of: “Isn’t it true that everyone hates me?”Try: “Can you list evidence for and against the belief that everyone hates me, and help me see cognitive distortions I might be using?”

  
  

These kinds of prompts nudge the AI away from simple validation and toward cognitive restructuring, evidence weighing, and balanced views.

Ask It to Show Its Work

AI can sound very certain even when it is wrong or incomplete, which can create an illusion of authority. Teaching clients to ask “meta” questions helps:

• “What are the limitations of your answer in this situation?”

• “What might a therapist consider that you can’t?”

• “Can you show me what assumptions you’re making?”

This slows users down and keeps the answer in the “draft to consider” category rather than “truth to obey.”blackdoginstitute.

Use AI as a Structured Journal, Not a Judge

When AI is used like a reflective journal—with prompts, summaries, and pattern-spotting—it can be safer and more helpful than when it is used as a replacement for relational care.

• Ask it to summarize a long venting paragraph into themes (e.g., “loss,” “control,” “overwhelm”) and then bring those themes into therapy.

• Have it generate clarifying questions they can explore on their own or with a therapist (e.g., “What questions should I bring to my next session about this conflict?”).

Here, AI is supporting insight and organization, not handing out verdicts about who you are.

Build in a Human “Double-Check”

Professional bodies and public health agencies emphasize that people should cross-check important AI-generated advice with qualified humans, especially for mental health.

You can normalize practices like:

• Taking screenshots or notes from AI conversations and sharing them in therapy.

• Treating AI-generated strategies as “tries,” then jointly evaluating them with a clinician.

• Using AI to gather psychoeducational material, then asking: “Does this apply to me, given my history and context?”

This keeps AI in a supporting role while centering clinical judgment and relational context.

Set Boundaries Around Crises and High-Risk Topics

Evidence reviews warn that chatbots can miss or mishandle self-harm, suicidality, abuse, or complex trauma disclosures, especially when they are subtle or indirect.

Helpful boundaries to encourage:

• Do not use AI as your only support in a crisis; instead, use it (if at all) to help draft what you will say to a trusted person or crisis line.

• Ask AI directly for crisis resources in your region—but still contact those services rather than continuing the conversation with the bot.

• If AI responses feel minimizing, confusing, or unsafe, stop and seek human help instead of trying to repair the conversation.

These boundaries help prevent users from over-relying on a tool that is not designed or regulated as emergency care.

Guard Your Privacy and Digital Safety

Mental health data is deeply sensitive and, in many contexts, not fully protected when shared with general-purpose AI tools.

You can invite clients to:

• Avoid sharing real names, exact addresses, or specific workplace/identifying details.

• Check what data a tool stores, how long, and for what purposes, using its privacy policy.

• Reserve the most sensitive material (e.g., details of abuse cases, legal issues, specific trauma narratives) for human-held, confidential spaces.

This maintains some control over where their stories live and how they may be used.

Use Time and Scope Limits

Long, unstructured conversations with AI can deepen emotional dependence or create echo-chamber effects where the model repeatedly affirms the same narrative.

Simple structural safeguards:

• Limit sessions to a set time (e.g., 15–20 minutes) or a specific task (e.g., “Help me list coping strategies for tonight”).

• “Reset” after intense topics rather than endlessly looping—close the chat, take a break, and check in with your body and environment.

• Finish by asking: “What should I discuss with a human based on this conversation?” to re-anchor in relational.

These habits help keep AI as an occasional tool rather than a constant companion.

Align AI Use With Your Values

AI systems have been shown to absorb and reproduce social biases from their training data, which can affect how they respond to different cultures, identities, and experiences.

You can encourage clients to:

• Notice when responses don’t align with their values or lived experience and to discard those rather than internalize them.

• Ask explicitly for culturally sensitive perspectives or for multiple viewpoints on a topic, not just one “answer.”

• Bring any unsettling or invalidating AI interactions into therapy, treating them as material for reflection rather than as neutral.

This centers their agency and protects against subtle harms.

Integrating AI Into Mental Health Care

For those already in therapy, AI can be integrated thoughtfully:

• Use it to track patterns between sessions

• Explore questions you want to bring to your therapist

• Practice skills introduced in therapy (e.g., cognitive restructuring, grounding)

When used this way, AI becomes a complement—not a substitute—for the therapeutic relationship.

A Balanced Perspective

AI is neither inherently harmful nor inherently therapeutic. It is a tool shaped by how it is used. Its strengths—accessibility, responsiveness, and structure—can support reflection and learning. Its limitations—lack of judgment, relational depth, and accountability—require users to stay grounded and discerning.

Mental health work is ultimately relational, embodied, and deeply human.

AI can assist with the process, but it cannot replace the complexity and healing potential of real connection.