The Most Dangerous Thing About AI Isn't What You Think

Image generated by Gemini (Google), 2026. Prompt by Damian K. ten Bohmer

I've spent years working with and coaching leaders across 40 countries. I hold an ICF PCC credential. I've delivered over 3,200 coaching hours. I've designed award-winning leadership programs that measure real impact through competency frameworks and ROI dashboards, and I've received the ICF Global Coaching Impact Award for Organisations.

And I've watched ChatGPT convince thousands of people they're being coached when they're actually handing over their deepest professional struggles to a general-purpose language model that stores everything on US servers and uses it to train future versions of itself, and that scares the shit out of me. 🤨

The danger isn't that AI is obviously bad. The danger is that it feels so helpfully convincing that people don't notice what they're handing over or what they're receiving in return.

Let me be clear about what concerns me most: the confusion this creates for people trying to develop themselves, and the damage it does to a profession that many people spend hundreds of hours studying to serve with integrity.

People get disgruntled and disappointed with real coaches. They balk at investing money in their development because they've already tried "coaching" with ChatGPT and it didn't work.

But what they tried wasn't coaching at all.

What You're Actually Handing Over

When you type a question into ChatGPT, you probably think it's a private conversation.

It's not. 😯

By default, consumer prompts on ChatGPT Free and Plus are used for model training unless you manually opt out in settings. Even paying $20 per month doesn't protect your data by default.

Everything you type gets saved. Every prompt, question, response, and uploaded file is stored as "Content" on OpenAI's servers. Researchers found that about 63% of ChatGPT outputs contained personally identifiable information from users.

Only 22% of users were actually aware of ChatGPT's data-sharing settings.

Think about what you've typed into ChatGPT in the last month. Career struggles. Team conflicts. Performance reviews. Salary negotiations. Leadership challenges you haven't told anyone about.

All of it is stored on US-region infrastructure, which means US privacy laws apply regardless of where you live. As of May 2025, a court order in NYT v. OpenAI requires OpenAI to preserve consumer ChatGPT logs indefinitely until litigation concludes. That includes chats you've deleted.

Security experts now warn: "treat anything you type as potentially visible to public" and never share personal, financial, or corporate information in a chat.

When you work with a professional, credentialed coach, everything you share is confidential. It's protected by ICF ethics standards. It's never used to train anything. It's never stored anywhere except in the coach's secure notes.

When you work with ChatGPT, you're handing over your career and personal development data to a company that uses it to improve a commercial product.

That's not coaching. That's data extraction dressed up as support.

Image generated by Gemini (Google), 2026. Prompt by Damian K. ten Bohmer

What You're Actually Receiving

ChatGPT can sound incredibly helpful. It asks thoughtful questions. It reflects back what you've said. It offers frameworks and suggestions.

But it has no ethical framework. No ICF accreditation. No coaching methodology. No accountability.

The International Coaching Federation introduced AI Coaching Standards in 2025 to address ethical risks, including bias and confidentiality issues. The standards emphasise that "AI should augment human judgment, not replace it" and that AI coaching must clearly distinguish between coaching and therapy.

The ICF's 2025 Code of Ethics now requires coaches to disclose the use of AI tools and ensure clients' interests are protected. Coaches cannot abdicate responsibility to technology providers and remain accountable for their choice of platforms and the resulting data privacy implications.

Privacy, trust, and bias mitigation are identified as critical concerns in AI coaching.

A general-purpose AI like ChatGPT doesn't follow any of these standards. It doesn't know when to stop. It doesn't know when a conversation has moved from leadership development into mental health territory. It doesn't know when its advice is culturally tone-deaf or legally risky.

I've seen leaders follow ChatGPT's advice on difficult team conversations and make things worse. I've seen people use it to process trauma that needed a therapist, not a language model. I've seen executives share confidential business strategy and not realise they've just fed it into a training dataset.

Real coaching is built on trust, presence, and ethical boundaries. It's designed to help you think more clearly, not to generate plausible-sounding text based on pattern recognition.

The Wider Damage

The AI coaching market is projected to reach $2.4 billion by 2028. 73% of professionals are willing to try AI-powered coaching solutions, and corporate AI coaching adoption has increased 156% year-over-year.

That sounds like progress until you look at what's actually happening.

There are now over 340 active AI coaching providers worldwide, with an average of 15 new platform launches every month. Most of them have no connection to professional coaching standards. Many are just ChatGPT wrappers with a new interface.

AI coaching costs 80% less than traditional executive coaching. Traditional coaching ranges from $200 to $600 per hour. AI platforms charge $50 to $200 per month for unlimited access.

This democratises access, which sounds good. But it also devalues professional coaching expertise and creates a market flooded with tools that look like coaching but deliver something fundamentally different.

Security researchers warn that many companies are quick to incorporate AI agents and off-the-shelf models into their systems, often without fully understanding where these tools fit or how to protect users' personal data. This rush for convenience could lead to serious data breaches.

The main thing that concerns me is the confusion this creates for people looking to develop themselves. When someone tries an AI tool labelled "coaching" and it doesn't help, they assume coaching doesn't work. They don't realise they never actually experienced coaching in the first place.

Then, when they encounter a real coach, they're sceptical. They've already spent time with an AI that asked insightful questions and provided them with frameworks. Why would they pay hundreds of dollars for something they think they've already tried?

This damages the profession and leaves people worse off than if they'd had no coaching at all.

What Ethical AI Coaching Actually Looks Like

I'm not against AI in coaching. I've built an AI coaching platform myself.

But I built it the right way.

My AI coach runs on CoachVox, a platform designed specifically for professional coaches. It's built on my ICF PCC-accredited coaching expertise. It reflects the frameworks I use with real clients. It adheres to the same ethical boundaries I maintain in my practice.

And here's the critical difference: it does not use client conversations for model training.

The CoachVox platform is GDPR-aware. Conversations are private. Client data is kept separate from model training. I can link to my own privacy statement. The infrastructure is designed for coaches who take confidentiality seriously.

OpenAI's Enterprise Privacy commitments state: "By default, we do not use your business data for training our models" and "You own and control your data." But these protections only apply to Enterprise, Business, and API customers. Not consumer accounts.

Professional coaching platforms built for coaches can provide GDPR-compliant infrastructure, with data stored in secure locations such as Frankfurt, Germany. Client data stays separate from model training.

The ICF standards emphasise responsible technology use, transparency and trust, and addressing concerns about data leaks and risks associated with machine learning. This creates a framework that allows coaches and clients to navigate AI integration with confidence.

ICF CEO Magdalena Nowicka Mook states: "As we explore AI's possibilities, ICF is committed to embracing its possibilities as well as maintaining the highest standards of coaching competencies" to ensure technology makes "coaching more impactful and accessible than ever."

That's the standard I'm holding myself to.

Image generated by Gemini (Google), 2026. Prompt by Damian K. ten Bohmer

Try Something Built for Coaching, Not Data Extraction

If you've been using ChatGPT for coaching conversations, I'm not judging you. It's easy to see why people do it. It's free, it's fast, and it sounds helpful.

But now you know what you're actually handing over and what you're actually receiving in return.

If you want to try AI coaching that's built on professional coaching expertise, designed to protect your privacy, and grounded in ICF ethical standards, I've built that.

It's called Coach Damian AI. It runs on Coachvox. It doesn't use your conversations for model training. It's GDPR-aware. It's built on years of coaching experience across 40 countries and over 3,200 hours of professional coaching.

It won't replace a real coach. But it's a legitimate tool for reflection, clarity, and growth between coaching sessions or when you need support outside of scheduled time.

You can find all the information here: https://www.worldcoach.me/damian-ai

And if you want to work with me directly as a human coach, you can reach me through World Coach Performance or drop me a DM on LinkedIn.

Either way, please stop treating ChatGPT like a coach. It's not. And you deserve better.

#AICoaching #DataPrivacy #Coaching #LeadershipDevelopment #AIEthics