One of the dangers of generative AI is what happens when its work becomes self-referential. When an AI model is trained on its own output, it can “drift away from reality, growing further apart from the original data that it was intended to imitate,” as reported in a recent piece in the New York Times, “When A.I.’s Output Is a Threat to A.I. Itself.”
In a paper published last month in the journal Nature, a group of researchers in Britain and Canada showed how this process results in a narrower range of A.I. output over time — an early stage of what they called “model collapse.” …
If only some of the training data were A.I.-generated, the decline would be slower or more subtle. But it would still occur, researchers say, unless the synthetic data was complemented with a lot of new, real data…
“The model becomes poisoned with its own projection of reality,” the researchers wrote of this phenomenon.
This also can happen with religions.
Religious stories are fictional tales that have meaning for the tellers. But when those stories become a substitute for actual knowledge, they take on a life of their own. When we don’t allow our beliefs to be confirmed or disconfirmed by external observation, we find ourselves becoming progressively more unmoored from reality and normalcy. This especially happens when we adopt a fundamentalist mindset. Our beliefs become more important to our identity—and to us—than life itself. We start thinking up self-righteous self-justifications to confirm those beliefs, even when they are incorrect and even harmful. When faced with evidence that a belief is wrong, rather than change the belief, we double down on it. Then we rely on that belief, that story, to inform future conceptualizations about truth and fantasy, good and bad, right and wrong.
That’s what it was like when I lived in the Evangelical world.
Our belief system included memes that protected it from outside challenges. As TheraminTrees put it in a video from 2015, “Critics don’t stand a chance against belief systems that are so psychologically ring-fenced and can wield such gargantuan carrots and sticks.” And so the religion’s model of truth collapses into an upside-down world where down is up, wrong is right, and suffering is joy. It’s a world ruled by a malignant narcissist bigger than the whole universe, and you are expected to worship him just because he’s so perverse. It’s a world in which this is normal and healthy, and anyone who challenges that norm is branded as a pervert, just because they’ve taken control over their own life rather than submitting unconditionally to the religion’s god and his deeply held insecurities.
Once you leave that world, it’s like leaving a cult. You gain some distance from the dysfunction, and you see how sick it is. After a number of years, you can hardly believe you used to think that was okay. But you did, and you understand that you were trapped in a twisted fantasy of the religion’s making.
“And yet,” TheraminTrees continues, “critics do get through.” Maybe not enough to save the religion itself, but maybe enough to save a few souls trapped in its clutches.