The Black River
Cult dynamics in the glamor of AI
Through this series on AI that I’ve titled “A New Kind Of Being,” one of the biggest epiphanies I’ve had is that AI, once you remove the glamor of its fancy algorithms, isn’t really all that new.
What feels like an unprecedented technology that is reshaping our very existence in fact has an architecture that captivates, magnifies and exploits our deepest longings, desires and vulnerabilities in the same way systems of power have been since the dawn of time.
And I haven’t heard anyone who can speak to this better than Star Spider, our guest for episode 419 of This Is Actually Happening, “What if you realized AI was a cult?”
Power Points
From a young age, Star had always gravitated toward big philosophical questions. She was especially drawn especially to Star Trek: The Next Generation, where the character Data’s uncertain personhood seeded an early fascination with consciousness, identity, and what makes something “real.”
Though her father wrestled with anger issues, Star didn’t name any particularly strong childhood traumas. But there was, as she called it, an “ache of loneliness and a lack of direction,” which led her to be a seeker. Her parents encouraged her curiosity and by her teens she’d embraced Wicca, drama, and eventually Toronto’s rave scene, a period marked by drug use, deep community, and a restless search for purpose.
That search took her on a solo, one-way journey across the world, chasing spiritual power centers from Stonehenge to the pyramids of Egypt to Peru where she found her magic.
It was during this period, during a stint back home working at a juice bar, that she met the couple who would become her cult leaders: a husband and wife pair who drew her in with intense intimacy, philosophical conversation, and a doctrine they called “parapragmatism.” This was built on the idea that all spirituality was meaningless but could be aesthetically appropriated.
My new friends were really, really cool. They made me feel extremely special. They were impressed with absolutely everything I told them. We would talk for hours and hours. It was so intense almost immediately. But it was like falling in love... They took my spirituality at that point from me and replaced it almost with themselves as the core central figures, like my deities.
And that’s when the PowerPoints began.
They sat her down one night and presented her, through PowerPoint presentations, with a full catalogue of all of her flaws. They framed this as a way to heal her shadows, and began making her do penance rituals, food restriction, forced isolation from family, and constant surveillance.
During my time with them, I developed something that I called the Black River, which was this river that runs through my soul. I saw myself as a sort of moving body of water that was always changing and always in flux and always learning and growing. And they tarnished that. They filled it with toxic sludge. I hated myself. There was no way to be good enough. But I bought into everything they had to offer, and I was absolutely hooked.
Everything Shatters
Star then met her boyfriend (now husband) and he moved in with them, expanding the group to four. She spent roughly seven years in the group, developing severe depression, physical illness, and near-total isolation.
I was kept in place by a belief that I could be better. I was always led to believe that they had my best interests at heart and that they were trying to help me. No one else did, and no one else knew how complex it was, and no one else knew the pathway to get to where I wanted to be.
At the end of her rope, one day she and Ben go to a therapist who said, “just don’t listen to them.” And so she did. She went home and she told them, “I can’t listen to you.” And once she did that, as she said, “It was like this magic key that unlocked this door.” Everything shattered - the cult, her living situation, and her entire belief system. Star and Ben were kicked out and they began the long process of putting their lives back together.
As part of her healing, Star found out everything she could about what happened to her, diving into cult studies, coercive-relationship dynamics through social psychology, persuasion, manipulation, and high-control tactics. All of it began to make sense. She then went into philosophy, deepening her inquiry. That path finally led her to AI, where she came to see the same manipulative architecture playing out at even greater speed and scale. Everything clicked as she went down the parallel checklist between AI and her own cult indoctrination:
Unlimited positive regard / love-bombing
Access to secrets / sense of wonder
Frictionless companionship
Ideological isolation leading to physical isolation
One controlled source of “truth”
Escalation risk (sleep/nutrition neglect, psychosis)
Exploitation of universal human need for connection, purpose
We’re All In The River
The parallels she draws are brilliant and it’s a list that everyone should have in their back pocket. But to me, the real heart of the story comes at the end, when she poses a deeper challenge.
If cults provide community, connection, purpose, wonder, if AI provides this thing, how ethical is it for us to ask people to leave that behind if we are not offering anything better in society? If we’ve created this modern landscape of isolation and desperation and loneliness, and then people are going into these cults or into these AI relationships and finding deep and profound meaning. There’s a price to pay, obviously. I spent years in recovery... but at the same time, we have to ask ourselves why these people are being so drawn into these profound and intense relationships. And maybe it’s because we’ve created a world where those profound relationships are really hard to maintain in everyday life.
Systems of control always rely on a lack within ourselves, a Black River that they continue to tarnish while promising to be the ones that know how to purify it. We like to think it’s the “vulnerable people” who need to be careful with AI. We consider the technology inevitable and go on believing that these vulnerable people are always somebody else. But we’re all in the River. We’re all susceptible, we’re all victims and we’re all enablers. The question that remains isn’t just about AI, it’s about whether and how we can build a world in which its worst incarnations won’t consume us.
In the end, Star’s answer is to devote her work now not to what’s broken, but to what actually works in healthy communities and societies, how to build them and how to make them sustainable. She says,
I want to create a world where the conditions are better so people aren’t forced into these relationships to their detriment, but can find community connection and purpose in places that are healthy.
You can contact Star and find out more about her incredible work at thecounterproject.org
Fascinating!!