on paternalism, or: against scaling smoothly, exponentially and uneventfully through superintelligence
what shakespeare, oppenheimer, aristotle and I have to say about OpenAI CEO's latest essay
In the past seven days, 374,621 people have had their fortune read to them by a stranger on the Internet.
We all find it conceptually difficult to understand such large numbers, so let’s break this down. On the same day Sam Altman published The Gentle Singularity, a group of people large enough to pack out Michigan Stadium read it. By day two, 250x the population of Vatican City had devoured it. By day three, the number was equivalent to the entire population of Iceland. By the end of the month, the number of people who will have read Altman’s latest essay will likely be equivalent to the amount of people employed by the Federal Government of Brazil. Talk about influence.
But as soon as I caught sight of the phrase “The Gentle Singularity” hurtling through the devastating vortex of news from around the world, a well of fury started building in my stomach. To understand why, please join me for a momentary literary detour.
In Romeo and Juliet, Shakespeare famously wrote:
“What's in a name? That which we call a rose
By any other name would smell as sweet.”
For fear of slipping down (and potentially disappearing in) a metaphysical wormhole, let’s keep this simple. What Juliet was trying to say is that Romeo would have the same attractive qualities even if he were not a Montague, just as a rose would have the same sweet smell if it were called a sunflower.
But to get just a teensy bit metaphysical for a second, let’s agree that if a rose were not a rose, it would not be a rose. It would not evoke the same amorous emotions nor generate the same idyllic images in our minds. As Shakespeare knew, it turns out that naming is rather important after all.
on the unbearable heaviness of naming
The very first gift most of us receive is our name. Our name cradles our emerging sense of self as we make our abrupt, wailing entrance into the world; it chisels our unique legacy into the hearts of those we love long after our death. It is precious. And so the split second in which our new colleague’s eyebrows furrow upon hearing our name spotlights the disconnect between the stereotypes they associate with our physical appearance and the reality of our culture, class and ancestry.
So, what therefore is in a name? Well, I’ll tell you, my sweet comrade: everything is in a name. Because naming is not just writing; it is priming.
In titling his essay The Gentle Singularity, Altman, with the nib of his pen bleeding ink of molten power, has knowingly swaddled one of the most contentious socio-technical topics of the past five years in a spool of cotton candy-floss. The juxtaposition of the familiar adjective “gentle” cushions the existential connotations of a deeply technical, uncontrollable concept that is foreign to most laypeople. Altman primes his reader for an essay that drapes a veil of casual indifference over a Munchian painting of a society desperately gasping for air by declaring that there is no emergency.
If Altman were an octogenarian husband at the bedside of his ailing wife, he would be promising her that “Everything is fine, dear, I’m here.” Except when the octogenarian husband does it, it’s soothing, because they both know how this ends and have been preparing for this eventuality for some time. When Altman does it, it’s callous and disrespectful, because we don’t know how this ends and neither he nor we have been preparing for it for any time at all.
In dubbing the Singularity–this grand, omnipotent, unfathomable thing–gentle, Altman is pouring icy water on the existential concerns of a generation of students who are calling into question the future of their working lives when “whole classes of jobs [are] going away”. Altman is undermining the expert opinions of hundreds of AI safety researchers around the world who are working day and night to align the super powerful technology whose regulation he is directly incentivised to oppose. Worst of all, standing high on his gilded pedestal as CEO of one of the most valuable AI companies in the world, he is manipulating the opinions of every policymaker, parent, teenager and jobseeker reading his blog, quietly but forcefully weaponising language to deepen their trust in him as some kind of paternalistic overlord.
the poisonous voice of paternalism
The honeyed deception of the essay’s name oozes through every subsequent paragraph. I will leave it to the futurists and AI experts to analyse Altman’s claims about the Singularity; the point I want to make has nothing to do with AI. It has, of course, to do with language.
Paternalism speaks in a voice that camouflages condescension in a canopy of care.
The paternalist voice uses basic words in the name of accessibility, generalising them to the extent that they become as unintelligible as esoteric language.
“There will be very hard parts like whole classes of jobs going away, but on the other hand the world will be getting so much richer so quickly that we’ll be able to seriously entertain new policy ideas we never could before.”
DING DING DING excuse me Sir, I have a few questions please:
“very hard parts” for whom? For the people who have historically and continuously been disenfranchised and othered by the same systems of power that have enabled you to build this technology unregulated in the first place?
“the world will be getting so much richer quickly” meaning what, exactly? That the wealth the global economy will go on to generate as a result of superintelligence (ASI) will continue along the same trend line we’ve experienced since 2020, after which the richest 1% have captured almost two-thirds of all new wealth, nearly twice as much money as the bottom 99% of the world’s population? Not sure that really constitutes the world getting richer…
“new policy ideas”…are you talking about universal basic income (UBI)? Now, I’m no policy expert (neither is he), but the last time I looked there was zero direct correlation between the world getting richer and the serious entertainment and implementation of new policy ideas such as UBI.
The paternalist voice drips with manipulation thinly veiled in a cloak of protection.
“We do need to solve the safety issues, technically and societally, but then it’s critically important to widely distribute access to superintelligence given the economic implications.”
Okay, now I do appreciate you dedicating 11 words in the 22nd paragraph of this deeply problematic essay to the alignment problem…before immediately skating over its significance with an invalid presupposition at best–that global access to ASI will repair some of the systemic economic inequality in our society–and an ambiguous stream of words at worst.
The paternalist voice talks about the world in broad brush strokes, weaving a web of invisible silk that cocoons your mind until it strangles your oxygen supply.
“People still die of disease…the future can be vastly better than the present…scientific progress is the biggest driver of overall progress…”
These statements can be written without any pushback because they are not falsifiable. When powerful people make outlandish, non-falsifiable statements and distribute them to a global audience, it is very difficult to hold them to account. And those that could are disincentivised from doing so.
from Aristotle to Altman
Most toxic is the way the paternalist voice perpetuates the same socioeconomic power structures that enable it: “It’s hugely exciting to think about how much more we could have”. Is it really? As Robert Oppenheimer, creator of the atomic bomb, famously said: “When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success.”
This wilful paternalistic ignorance in the name of technological progress is no longer excusable. At some point, as economists and anthropologists like Jason Hickel, Mariana Mazzucato and Kate Raworth have advocated for, we need to stop and ask: do we not have enough already? Are we not drowning in an excess of wants, desires, things that we simply do not need? What might happen if we question whether capitalist consumption is the best model for an economy that is tearing our planet apart at the seams? What if, instead of maximising resource and capital accumulation, we stop to ask ourselves how we might redistribute the abundance enjoyed by the few?
Paternalism finds its origins way back in Aristotle’s Politics, in which he asserts that the state is a teleological outgrowth of the household, which is itself a natural society. In other words, the nuclear household was thought of as the foundational social institution. Etymologically, the Greek pater meaning "father" and arkhein meaning "to rule" denote the sociopolitical hierarchy inherent within the patriarchy: rule by fathers. In failing to publicly ask the difficult questions and instead adopting the wilfully ignorant voice of paternalism, Altman is carving out his own role as an autocratic father figure of society. What he is crucially forgetting, however, is that society never asked him to create nor promote himself into this deific role.
By replicating the same structure of Star Wars’ quasi-religious adage: “may we scale smoothly, exponentially and uneventfully through superintelligence”, Altman, our lord and saviour, abandons the responsibility that comes with his protector status, sugarcoating reality in reassurance. In Star Wars, characters utter the blessing “may the force be with you” when acknowledging that someone is facing overwhelming odds and mortal danger: it suggests that they have no choice but to rely on some divine force for their survival. By ending his essay with an adaptation of such a culturally iconic phrase, Altman is excusing himself of the hardest task of all: putting his so-called plan to “solve the alignment problem” into action. In effect, he’s saying: “Good luck when it comes, guys! We’ve done our part, I’m sure you can figure out the rest 🙂”. Oh, you mean our collective survival as a species? Cool. Thanks for all your hard work, father Altman 🙂
Altman is avoiding uncomfortable truths about the sheer existentialism and scope of the challenge that threatens the very creation that would ensure him a gold-plated statue in the museum of civilisation. But Altman cannot afford to display wilful ignorance when he is both creating and reading the fortunes of 374,621 people around the world. He has a duty to face up to the level of responsibility he has created through his own accumulation of power, question the fortunes he is creating, and treat his readers not as loyal followers or incompetent children, but as fellow global citizens, themselves the parents of all future generations.
I suppose all that’s left to say is this: may we collaborate slowly, intentionally and wisely to overcome paternalism.
drops from the eaves
In this section, I pay tribute to ingenious sentences and words that spotlight something untranslatable: an emotion, a way of being, a moment in time. Sometimes they’re already memorialised in print; often they’re overheard in cafés; always they’re rough diamonds I’ve eavesdropped.
sometimes the most important thing we can do is let ourselves stay blurry. to be unbranded, unpositioned, undone for a while. because real self-knowledge is a slow, unmarketable process. and the lost feeling might just be the first sign that you’ve finally stopped performing someone else’s idea of a good life.
from why feeling lost might mean you’re finally doing it right by , a beautiful Substack I recently found in exactly the right moment
Perhaps the desire to create closed systems and keep time going in a straight line is the reason for Second Peoples’ obsession with creating fences and walls, borders, great divides and great barriers. In reality we do not inhabit closed systems, so why choose the second law of thermodynamics to create your model of time?
from Sand Talk: How Indigenous Thinking Can Save the World by Tyson Yunkaporta, a revelationary look at how indigenous thinking can help us untangle some of our most complex problems such as climate change
new words as windows
In this section, I coin new words to help plug gaps in our understanding of ourselves, each other and the world around us. If you have ideas or submissions you’d like me to include, just hit reply!
ostricharchy n.
[oss-trik-ar-kee]
Etymology: From English ostrich (Middle English, from Old French ostruce, ultimately from Latin avis struthio meaning "ostrich bird") + Greek -arkhia (rule, government), from arkhein (to rule, command).
Definition: A form of governance or leadership characterised by the deliberate avoidance of inconvenient truths or mortal danger to society, while maintaining authoritative control over others' decisions and futures. Ostricharcal leaders use reassuring language to maintain their authority while failing to address the very issues their position of power obligates them to confront.
Thank you, as always, for reading!