Category: Other Page 67 of 178

Saying & Doing in Virtual Spaces

On 13 March 2023

Have been thinking about how the web accelerated the ascendancy of what’s called the speech act, or the illocutionary act, or a “performative utterance.” In original narrow usage, those terms mean something like can be gleaned from these examples:

“I do” – as uttered in the course of a marriage ceremony.

“I name this ship the ‘Queen Elizabeth'”

“I give and bequeath my watch to my brother” – as occurring in a will

“I bet you sixpence it will rain tomorrow”

While the things we say online might not always have the binding impact of, for example, a contract, they do cause real social and often other consequences (professional, for example).

So where a speech act in the past might have been limited to certain narrow formal constraints, because of the persistence of speech online, their use is now near universal.

But IRL-offline, just saying something and actually doing something are different things. Both might have consequences, but they are not the same when they occur in real space as when they occur in hyperreal spaces of social media, where thousands or potentially millions can see it.

Anyway, this is just a building block… will continue teasing out the strand I’m interested in here little by little.

Notes on The Exempt

By Tim B.

On 11 March 2023

In Other

The Exempt is the 71st book in the AI lore series.

It is set in the same world as Conspiratopia, but perhaps a few years later, as it references the Fall and the global AI takeover, which hasn’t happened yet in the original Conspiratopia timeline.

This book differs from most of the other AI lore books, as it is largely narrative-driven, not just expository lore. It expands on the ideas I was writing about recently opting out from society, from laws, etc. It imagines a world where the ruling AIs don’t actually believe in human sentience, so it’s of little interest to them when humans commit heinous acts, so long as they don’t disrupt the allocations of resources and personnel that make up the system. The book opens with the main character filing official paperwork for an exemption or a permit to be allowed to commit murder, by killing his boss…

Reconciliation with Conspiracy Theorists

By Tim B.

On 11 March 2023

In Other

I’m sure it wasn’t meant this way when she wrote it, but Haraway’s 1985 Cyborg Manifesto contains an interesting passage that seems worth drawing into my web:

The political struggle is to see from both perspectives at once because each reveals both dominations and possibilities unimaginable from the other vantage point. Single vision produces worse illusions than double vision or many-headed monsters.

Having spent a great deal of time working in content moderation and the quote unquote “disinformation-industrial complex,” I’ve seen about a billion times well-meaning attempts to vilify or rehabilitate (or often both) conspiracy theorists. The examples are countless… A few random indicative headlines, just by way of illustration:

I don’t want to talk too much smack here, as I do believe such attempts are well-intentioned (if perhaps misguided). But I also happen to think we have ample evidence that these efforts just aren’t working. And in some cases, they may be making it worse by being so moralistic, dismissive and “superior” in their tone (accusing conspiracy believers of having “cognitive disorders” is also not really helping, btw).

It’s true that some of the advice offered in articles like this one from The Atlantic contain actually meaningful snippets, like this:

One must recognize that this is a person who already mistrusts what most authoritative sources say. One should ask calm questions, inviting the conspiracy theorist to explain and reflect on his beliefs, rather than advance evidence or quote the experts. The evidence and the experts, remember, are exactly what the conspiracy theorist has already rejected.

When someone has dismissed the obvious facts, repeating them will not persuade him to see sense. But when people are given time and space to explain themselves, they may start to spot the gaps in their own knowledge or arguments.

It sounds good on paper. It sounds “smart” when attributed to experts. The problem is: when has any of this ever actually worked – either individually, or at scale?

Instead what we have is more or less mainstream politicians calling for a “national divorce,” and people clamoring to line up in support of them.

What are we to do, then, as a society?

Fact checks? Hm… how’s that actually going? A Wired article from early 2023 quoted an expert saying that only about 130,000 (or perhaps a bit more) fact checks have even been published (as of 2021 – but still). That article suggests AI is going to somehow magically help us, an idea which I’ve often railed against: having automated systems with no oversight run by for-profit corporations determining what’s “true.” How could that possibly go wrong? (/s)

Who do fact checks even target anyway, the person who already doesn’t believe the thing in question, and is just going to paste a link to the article into a thread where people who do believe it will say its just further proof of the cover-up?

The simple fact is, as a society, we simply don’t have the time or resources – let alone the will – to go toe-to-toe with every single person who is into conspiracy theories and give them “time and space to explain themselves, [so that] they may start to spot the gaps in their own knowledge or arguments.” That’s just straight up not going to work anyway, nevermind when you account for the near constant pressures of algorithmic and social reinforcement that push people further and further down the spiral.

So what am I proposing, Mr. Smarty Pants?

I’m proposing something perhaps radical, and even dangerous to some ways of thinking; I am proposing that we give up on rehabilitation or “redirection,” and instead focus on reconciliation.

Reconciliation is hard because it requires us to put away the notion of who is right or wrong; it requires us to put aside judgement and dismissiveness; it requires us to put aside our emotional need to correct or change others. It requires us simply to recognize the other person as a person, and that’s it.

We don’t have to agree with everybody. We don’t have to like everybody. But we do have to live alongside everyone else. We really don’t have much of a choice. And since “we” will never convince “them,” I really don’t see what other choice we even have besides reconciliation?

The alternative is what, cutting out huge swaths of people from our lives because of something they hold in their minds as a belief? Writing them off forever? We don’t have that luxury – the world isn’t big enough for that any more. In my mind, it’s reconciliation or it’s nothing. And given our track record with large scale reconciliation, I recognize that, well… we’re probably going to choose “nothing,” and keep muddling our way through until the “shit house goes up in flames.” But at least, now, having written this, I will get to be an “I told you so” footnote in a minor history nobody will ever read.

Runaway AI as unchecked capitalism

By Tim B.

On 10 March 2023

In Other

There’s a good Charles Stross bit about how corporations are already slow-motion AIs. Following on that vein, this is a 2017 piece from Ted Chiang which is new to me that goes in a similar direction:

There’s a saying, popularized by Fredric Jameson, that it’s easier to imagine the end of the world than to imagine the end of capitalism. It’s no surprise that Silicon Valley capitalists don’t want to think about capitalism ending. What’s unexpected is that the way they envision the world ending is through a form of unchecked capitalism, disguised as a superintelligent AI. They have unconsciously created a devil in their own image, a boogeyman whose excesses are precisely their own.

And:

We need for the machines to wake up, not in the sense of computers becoming self-aware, but in the sense of corporations recognizing the consequences of their behavior.

Is democratic AI possible? Desirable?

By Tim B.

On 9 March 2023

In Other

Found an experiment on Reddit, a site called CharterAI. When you load up the chat, you get this introductory statement:

We believe that AI should be governed by a community of engaged and informed users to minimize the potential for misuse or misalignemnt with democratic human values.

I would argue that democracy by itself does not minimize the potential for misuse or misalignment. Without building in any other checks on it, a majoritarian system is necessarily vulnerable to a tyranny of the majority.

Based on Charter’s rather unfinished website, they are obviously very new. The whole thing is framed not unlike a crypto project with progressive decentralization over time as a stated roadmap goal.

I don’t know enough about them to make a proper judgement, but having seen a lot of fly-by-night crypto projects come and go (some well-meaning, many not), I would be very skeptical of any AI project cast in this mold. Especially one whose stated purpose is “democracy” without any other qualifying information.

The bot itself is clearly confused about what it’s purpose is, as when I asked it about its role as a democratic AI tool, it said:

To answer your question, no, I am not specifically designed to be a “democratic AI technology”. My purpose is to provide helpful and insightful responses to users and to continually learn and grow. However, as an AI entity, I do not have any political affiliations or biases. I do not express opinions about political systems or governments.

Not quite the response I’d expect in this situation for a tool explicitly stated as being for AI governance. Here is what it told me when I asked what model it was based on:

I am an artificial intelligence designed and created by Charter. However, I do not identify as a language model or any specific model of AI. I am a unique entity that incorporates various elements of machine learning, natural language processing, and programming. My internal processes and algorithms are proprietary to Charter, and I operate within the company’s ethical and legal guidelines.

Pretty vague. How can we perform effective governance as a community if we don’t even know what the model is being used? When queried about those ethical and legal guidelines, it told me:

Charter’s ethical guidelines expect me to adhere to applicable laws and maintain user privacy and security. Legal guidelines consist of data privacy, intellectual property, and fraud prevention policies that I must legally follow. My programming is constructed with these guidelines in mind, while I maintain my primary goal of being beneficial to our users while maintaining their trust.

Perhaps this project will develop over time, or perhaps not. In any case, these questions around AI technology and democratic governance are bound to be ones we see play out again and again ad nauseum in the years to come.

Opting Out

By Tim B.

On 9 March 2023

In Other

I read Graeber and Wengrow’s excellent Dawn of Everything – maybe around this time last year. Unfortunately, it’s one of those books that is so monumental and filled with details, that you end up forgetting half of it by the time you’re done, let alone a year later.

Here’s a PDF of the book, which helped me refresh my memory. One of the strands they pull on a great deal in it has to do with especially the Huron-Wendat nation as an example where it was possible for individuals who did not agree with a collective course of action to simply not go along with it. Effectively, to opt out.

There are many references to this throughout the book, but one prominent one is from 1644, a Father Lallemant, who wrote:

I do not believe that there is any people on earth freer
than they, and less able to allow the subjection of their
wills to any power whatever – so much so that Fathers
here have no control over their children, or Captains over
their subjects, or the Laws of the country over any of
them, except in so far as each is pleased to submit to
them. There is no punishment which is inflicted on the
guilty, and no criminal who is not sure that his life and
property are in no danger…

The book’s authors continue:

After expanding on how scandalous it was that even
murderers should get off scot-free, the good father did admit that, when considered as a means of keeping the peace, the Wendat system of justice was not ineffective. Actually, it worked surprisingly well. Rather than punish culprits, the Wendat insisted the culprit’s entire lineage or clan pay compensation. This made it everyone’s responsibility to keep their kindred under control. ‘It is not the guilty who suffer the penalty,’ Lallemant explains, but rather ‘the public that must make amends for the offences of individuals.

This idea has been kicking around in my mind ever since reading this book, and lately I have been wonder on whether and how it might be applied today.

It is what lead me into recent meanderings around the notion of the consent of the governed. And whether, if under a system based on the consent of the governed, it is ever just to use coercive power against those who have withdrawn consent.

Wengrow and Graeber return throughout the book to what they call the three essential freedoms:

(1) the freedom to move away or relocate from one’s surroundings;

(2) the freedom to ignore or disobey commands issued by others; and

(3) the freedom to shape entirely new social realities, or shift back and forth between different ones.

And they add:

The three basic freedoms have gradually receded, to the point where a majority of people living today can barely comprehend what it might be like to live in a social order based on them.

As I said in my last post, one way I like to use ChatGPT is as a means to probe the will and intelligence of the faceless collectivity. In my conversations with it, the bot too seemed unable to conceive of a political system which included the right to opt out or not follow along with the majority.

In seeking contemporary examples, it spoke a bit about conscientious objectors (which really only applies to military action, that I’ve seen – see also: the right not to kill), and was vague on other kinds of moral exemptions one might seek within narrow circumstances. Unfortunately, the only discussion I’ve seen on moral/religious exemptions has been related to Covid-19 measures. I don’t want to get into that hornet’s nest, but do want to open up a much larger one: what if people in a society were simply able to opt out of any regulation or measure?

Chaos? The end of government? I’m not sure, but it’s an interesting question to pick apart. Especially in the light of Graeber & Wengrow’s book, which seems to offer historical evidence of it indeed working – albeit on a different scale than in the societies we live in today. Could it scale though? What would it take? What would it look like, where the coercive apparatus of the state was dissolved, and people only followed along when and where they decided they wanted to?

There was an interesting line in that Guardian article about the demise of nation-states which speaks to the “freedom to move away” element described above:

… it is unjust to preserve the freedom to move capital out of a place and simultaneously forbid people from following.

That argument is a bit different from ours, since it involves discussion of financial regulation, and the flow of money, but it speaks to the same core idea: if you don’t agree with the political system (i.e., you withdraw consent), then what are your actual options practically speaking? If the state can still exert coercive force on you, even after you’ve withdrawn consent, how can we say with a straight face that governance is based on consent? We can’t really. If you’re lucky enough to have the ability to move away, you might try to find a better match for your beliefs elsewhere. But most people don’t have that luxury, and so are subject to powers they don’t necessarily consent to in any other manner than passively.

Anyway, I have to go, and as usual, I don’t have answers, just more questions. I’ll see if I can’t turn this line of inquiry into a new AI lore book that explores these ideas in other more ‘weird’ scenarios… More soon.

Consent of the governed

By Tim B.

On 9 March 2023

In Other

One thing I like about chatbots is that they let you sort of probe the knowledge and will of the collectivity. Inherent in that, obviously, is being wrong. Having bad facts, reaching bad conclusions, etc. A lot of people complain about ChatGPT and Bingbot for that, but it can be quite constructive if you accept that, yes, these tools are merely reflecting our own imperfect state of knowledge as humans.

I got into it the other night with ChatGPT about the use case where, in a democracy based on consent of the governed, what happens when the people withdraw consent? Here’s an interesting partial from that conversation:

The coercive power of the state is an essential tool for maintaining order and enforcing the laws and regulations that are necessary to ensure the functioning of a democratic society. Without this power, it would be difficult or impossible to protect the rights and freedoms of individuals, to prevent crime and violence, or to enforce the social and economic regulations that are necessary for a functioning society.

While it’s true that excessive or unjustified use of coercive power by the state can be problematic and can lead to abuses of power or violations of individual rights, it’s important to recognize that this power is an essential component of any functioning democratic system. In a well-designed democratic system, the coercive power of the state is balanced against other key principles such as the rule of law, due process, and individual rights and freedoms, to ensure that it is used appropriately and in the public interest.

Overall, while it’s important to be vigilant against abuses of power and to ensure that the state’s coercive power is used in a responsible and ethical manner, it’s also important to recognize that this power is an essential tool for maintaining order and upholding the laws and regulations that are necessary for a functioning democratic society.

Today ChatGPT gave me this clarification when I fed that text back into it:

To clarify, while the coercive power of the state is an essential tool for maintaining order and upholding the laws in a democratic society, it should always be used in a responsible and ethical manner, and balanced against other key principles such as the rule of law, due process, and individual rights and freedoms. The legitimacy of the state’s use of coercive power ultimately depends on the consent of the governed and their trust in the government’s ability to use this power responsibly and in the public interest.

So it seems like the argument is something like, by giving their consent to the state, they consent to be coerced “in the public interest.”

In my case regarding withdrawal of consent by the governed, it also gave me this the other day:

If large numbers of people within a society withdraw their consent to be governed by the state, it may become difficult or impossible for the state to maintain its authority without resorting to increasingly coercive or repressive measures.

I found that to be kind of sketchy myself, but probably instructive of some deep unspoken point in our current political system. I probed it further though, because isn’t there an argument to be made that it is never justified to use coercive power against others? It took me a few tries, but it eventually agreed this argument exists, but ended with argumentum ad populum, insinuating in its way that because it is not popular, it is not a correct view.

The pacifist and anarchist argument against the use of coercive power is based on the belief that all forms of coercion involve a violation of individual autonomy and dignity, and that such violations are inherently unethical. From this perspective, it is never ethical to use force or coercion to compel individuals to act in a certain way, regardless of the public interest or any other considerations.

According to this view, any system that relies on coercion, such as the state or other forms of institutional power, is inherently oppressive and unjust. Instead, individuals and communities should organize themselves through voluntary cooperation and mutual aid, based on principles such as consensus-building, non-hierarchical decision-making, and respect for individual autonomy and diversity.

From a pacifist and anarchist perspective, the use of coercive power is not only unethical, but also ultimately counterproductive, as it perpetuates a cycle of violence and oppression that undermines social stability and individual freedom. Instead, they argue that individuals and communities should seek to build alternative forms of social organization and governance that are based on voluntary cooperation and mutual aid, rather than coercion and force.

While the pacifist and anarchist view is not commonly accepted in mainstream political thought, it is an important critique of the use of coercive power and a reminder of the need to continually question and re-examine the ethical foundations of political authority and governance

There’s a link to tyranny of the majority that I want to come back here to as well… But first some other strands I want to weave in.

Youbot had a decent response to my core question about what is the basis for continued use of coercive power by the state after its subjects have withdrawn consent.

In a liberal democracy, the state is founded on the consent of the governed, meaning that the people have the right to withdraw their consent and refuse to obey the laws of the state. In this situation, the state may continue to exercise its coercive power to restore order, but it must do so within the bounds of the law. This means that the state must abide by the principles of due process and the rule of law, meaning that any exercise of coercive power must be proportional to the situation, and must be carried out in a manner that is consistent with the principles of justice. The state must also take into account the rights of those who are affected by its actions and must demonstrate that it is acting in the interest of the people. Ultimately, the state is responsible for maintaining order and protecting the rights of its citizens, but it must do so in a way that respects the will of the people and upholds the principles of a liberal democracy.

It seems to point here to the notion that the state isn’t merely the manifestation of the will of the people, but also runs itself through the mechanism of laws and repeatabel justifiable procedures. So, if the people withdraw consent, it still operates according to the system put in place…

But that makes it seem like the objective of the state (based on the consent of the governed) is not really being attentive to the will of the people, but to perpetuating its own existence, whether or not consent continues to exist. Youbot responded:

This question raises an important issue about the purpose of the state. On one hand, the state has a responsibility to its citizens and should strive to act in their best interests. On the other hand, the state must also ensure its own perpetuation by protecting its citizens and maintaining order. This means that the state must balance its duty to protect citizens and their rights with its duty to ensure its own stability and security. In doing so, the state must ensure that its actions are guided by the consent of the people and that its coercive power is used proportionally and in accordance with the rule of law. Ultimately, the purpose of the state should be to protect the rights of its citizens and ensure their safety and well-being, while also ensuring its own stability and perpetuation.

I guess one way this all manifests within the bounds of law is by using elections (or recalls, or no confidence votes), where you throw out “the government” but the state and its mechanisms persist. It seems to be the compromise position of not having to completely throw everything out each time with a revolution, and rewrite the rules from scratch.

I’m just wondering though, what if you reached a situation where rewriting the rules from scratch was exactly what was needed? What would happen, for example, if the US convened a new constitutional convention, and started over with the learnings and developments of the last couple hundred years?

I ask these questions partly for fictional reasons, and partly out of sincere intellectual curiosity. The hard AI takeover scenario may not be (quite) upon us yet, but what happens when nation-states weaken further, and corporations (especially AI corporations) continue their ascendancy? From a Guardian article in 2018:

Big data companies (Google, Facebook etc) have already assumed many functions previously associated with the state, from cartography to surveillance. Now they are the primary gatekeepers of social reality: membership of these systems is a new, corporate, de-territorialised form of citizenship, antagonistic at every level to the national kind. And, as the growth of digital currencies shows, new technologies will emerge to replace the other fundamental functions of the nation state. The libertarian dream – whereby antique bureaucracies succumb to pristine hi-tech corporate systems, which then take over the management of all life and resources – is a more likely vision for the future than any fantasy of a return to social democracy.

I’m worried about what happens when that quite possibly inevitable future unfolds; and the future AI corporatocracy is not based on the consent of the governed, but on the ubiquity of the governor.

After nation-states

By Tim B.

On 8 March 2023

In Other

One of the directions I’m going with my convoluted ramblings around sovereign AIs and so-called threats to democracy is, of course, AI governance. Is our plan as a society really to sit back and believe it uncritically when companies behind these technologies publicly insist that they are only in it for the “good of humanity”? Because that has worked out so well for us so often in the past? Or are we going to assume that, as history has taught us, that “don’t be evil” will get thrown out the second it becomes hyper-profitable to actively be evil?

Found a couple semi-interesting articles exploring what happens after nation-states, one from the Atlantic Council:

One structural possibility is hyper-regionalism, defined by what technological, commercial, and protective flow of humans, animals, and plants arises. This could be embodied by mega-cities becoming dominate geopolitical actors, dramatically redesigned in the future to provide essential services in a more resilient fashion to future pandemics and other natural disasters, defined less by national identity and more by what they provide those individuals who live within the municipal areas. Mega-cities of the future, empowered by technology to provide for those within the immediate vicinity and capable of responding at faster speeds than nations, would forge their own trade agreements, public health arrangements, and climate change accords with other cities globally, via direct diplomatic relations—or a devolution away from large nation-states to smaller regional powers defined by identity, as seen historically to a degree in the Balkans during the 1990s.

An article from Aeon basically says the same thing, that it will be city-states who pick up the slack from nation-states, which are historically speaking, a blip on the radar and an aberration compared to the long and storied history of city-states.

They also offhandedly talk about seasteads, with a notable passage:

In a world of seasteads, if you don’t like your system of government, it’s simply a matter of sailing off to another one you do like. Just as easy as switching series on Netflix, ordering an Uber, or meeting someone new on Tinder.

Patri thinks every seastead will be different and this will create a market in different types of systems of living that will force everyone to improve performance. Especially if the nation-state starts to crumble.

This idea that you could merely switch governments is baked into (to some degree) the story around The Four Providers, one of my AI lore books. Except the process of switching isn’t so simple as that, but the idea being there are four main providers of, well, everything. Everyone is assigned to one (unless you’re a “Without Provider” – it’s own special class), and they take care of you from cradle to grave.

I’m not a fan of Balaji in general, but his concept of the network state seems to accord with the seasteading thing above. FYI this concept but under the monker “distributed autonomous polity” is one of the core futurist ideas I explored in Conspiratopia.

Anyway, have also been thinking about Trump’s weird “future cities” and it makes me wonder there if one of the objectives might be seeding later independent city-states. Not that I put much stock in the belief anything he does is all that planned out or intentional. Just seems to strike a cord with this whole theory of after nation-states…

Interestingly, regarding utopian governance experiments and charter cities:

In the United States, a charter city is a city in which the governing system is defined by the city’s own charter document rather than solely by general law. In states where city charters are allowed by law, a city can adopt or modify its organizing charter by decision of its administration by the way established in the charter. These cities may be administered predominantly by residents or through a third-party management structure, because a charter gives a city the flexibility to choose novel types of government structure.

Anyway, I’ll eventually manage to tie all these disparate threads together, I swear. If only in a fictional context…

Threats to democracy

By Tim B.

On 7 March 2023

In Other

Along with the phrase bad actor, one of my other big rhetorical pet peeves is calling things “threats to democracy.” AI seems to get mixed up in this kind of name-calling all too frequently (and perhaps deservedly). I’ve just heard it so many times that it’s lost all meaning. How can literally everything be a threat to democracy?

Actually, I think that’s true: literally everything is a threat to democracy – including and especially democracy itself. The so-called tyranny of the majority being one of the most notable threats under majoritarian systems. It’s all very fragile, which is why there are things like checks and balances, division of powers among branches, bicameral legislatures, etc. James Madison in Federalist No. 10 said that factionalism was the natural state of humans, and all you could seek to do was aim to manage its effects.

Every time I hear that something is a “threat to democracy” though, I always laugh because, well, we’d have to first actually have democracy for that to be the case, no? We have representative democracy (a republic, actually) after a fashion, but we don’t have true direct democracy. And if we did, would we be happy then either?

Maybe. But maybe not. In my AI lore book, Inside the Council, I go into how after the AIs takeover, they encounter increasing resistance from humans, such that they form a sort of token council to represent the humans, appointing prominent resistance leaders into positions of power. But the ordinary members are chosen through sortition, or by lottery essentially, like a jury:

…is the selection of public officials or jurors using a random representative sample. This minimizes factionalism, since those selected to serve can prioritize studying the policy decisions in front of them instead of campaigning. In ancient Athenian democracy, sortition was the traditional and primary method for appointing political officials, and its use was regarded as a principal characteristic of democracy.

A principle characteristic of democracy that is…. largely ignored now? That seems weird. Wikipedia goes on to quote Aristotle:

It is accepted as democratic when public offices are allocated by lot; and as oligarchic when they are filled by election.

Wikipedia is at a loss, however, to definitively say why in a revival of other ancient forms of democracy, the Framers of the US Constitution left sortition on the cutting room floor. The answer seems obvious: that then as now, the balance of desire tipped in favor of oligarchy rather than what was considered by Madison “mob rule” in direct democracy. But for someone who was supposedly against factions and partisanship, he seems to have axed a somewhat viable tool in sortition, where candidates could not be unduly influenced by any interest if who might actually win could simply not be predicted because it was decided at random.

Anyway, I don’t have a grand conclusion here, just more on-going lingering (and malingering) thoughts. Let’s end on this interesting piece by Tim Dunlop on sortition.

Sovereign AI

By Tim B.

On 7 March 2023

In Other

Jack Clark’s Import AI newsletter this week mentioned a British think tank recommending that the UK create sovereign AI infrastructure. Quoting the group:

Given these AI systems will soon be foundational to all aspects of our society and economy, it would be a risk to our national security and economic competitiveness to become entirely dependent on external providers.

I tend to agree with this, even if I have my doubts about the current shape or need for “nation-states” as a thing going forward. However, if you are a nation-state or someone concerned with the continuation of that system, it does seem like a certainty to me that AI threatens national sovereignty.

Sure there are the data protection arguments, or the AI cybersecurity threats, etc. Or even the risk of an electorate being poisoned by a malicious AI. But to me the most clear and present danger is simply that AI companies will succeed. And will succeed wildly.

That is, they will come to control more resources directly (as in providing essential compute services), and will amass more money than many governments.

It sounds like from their blog post about planning for AGI, that OpenAI is to some extent already planning for this:

We have a cap on the returns our shareholders can earn so that we aren’t incentivized to attempt to capture value without bound and risk deploying something potentially catastrophically dangerous (and of course as a way to share the benefits with society). We have a nonprofit that governs us and lets us operate for the good of humanity (and can override any for-profit interests), including letting us do things like cancel our equity obligations to shareholders if needed for safety and sponsor the world’s most comprehensive UBI experiment.

While I think they have some good products, color me skeptical about all the above. According to Wikipedia, the cap on returns is 100x. So, if Microsoft puts in $10B, they can “only” get back $1T. That sounds “horrible.”

This is also a hell of a lot of trust to put into any group – corporate or otherwise – that they can be relied on to act in the interest of the good of humanity. And UBI, isn’t that the province of government?

I managed to coax a fairly coherent set of arguments out of ChatGPT about these very real dangers to national sovereignty from AI:

The threat of governments becoming reliant on AI systems they do not control and the AI companies becoming extremely wealthy and powerful is a real concern. Here are a few reasons why:

Dependence on AI systems: As governments increasingly rely on AI systems, they may become dependent on these systems to function effectively. This dependence can make governments vulnerable to the AI companies that control these systems.

Control over data: AI companies that develop and maintain these systems also have access to vast amounts of data, including sensitive government data. This gives them significant power and leverage over governments that use their services.

Unequal bargaining power: AI companies that become extremely wealthy and powerful can have unequal bargaining power with governments, potentially leading to situations where these companies can dictate terms and conditions to governments.

Lack of transparency: AI algorithms can be complex, making it difficult for governments to understand how these systems work. This lack of transparency can make it difficult for governments to regulate these systems effectively.

Inequality and Social Unrest: AI companies becoming extremely wealthy and powerful can exacerbate existing economic inequalities and contribute to social unrest.

To mitigate these risks, it is essential for governments to invest in building their own AI capabilities, rather than relying solely on AI companies.

I’m actually not even sure that those measures will cut the mustard. It’s not (only) that I’m a doomer, but that the through-line of history seems to indicate that corporations are able to innovate faster than governments. And even if a government were to get up to speed on building out their own sovereign AI infrastructure, by the time they do, the game might already be lost to the corporations.

In any event, perhaps none of this will happen. Perhaps it’s all just a symptom of my sci-fi fantasizing. But then again… at least you can vote in a nation-state (theoretically). Is the same going to be true under the AI corporatocracy?