I wrote a detailed explanation of my side of the story here for anyone interested.
The long and short of it is: these conversations need to happen publicly, with involvement from the communities who are affected by the problems. They shouldn’t happen behind closed doors and be driven and decided by solely for-profit entities with no oversight, and in whose interest it ultimately is to sweep problems under the rug.
The Daily Dot set up an account with Midjourney to see if Boucher’s findings could be reproduced. Several prompts such as “beach party photos” and even “scantily clad beach party photos” did not flag Midjourney’s filters and generated multiple realistic images of women’s naked breasts.
If they had blocked me, and then proceeded to fix the underlying technical issue, I would say fine. I accept the decision. But that’s not what happened, according to evidence we saw a couple days ago. The issue remains live in their product. So what good did banning me actually even do?
According to Midjourney’s user banning policy, it states that “Any violations of these rules may lead to bans from our services. We are not a democracy. Behave respectfully or lose your rights to use the Service.”
“The fact that they feel compelled to openly state ‘This is not a democracy’ points to a grave need for democratic governance of AI technologies,” Boucher told The Debrief. “It seems more and more apparent to me every day that, without oversight, we obviously can’t trust these companies to make fair and balanced decisions that actually benefit end users.” […]
“These conversations about the right limits of technology need to happen out in the open with the public involved. It should not take place behind closed doors, or in private email exchanges which are easy for product teams to de-prioritize,” Boucher told The Debrief. “The decision of where to draw the line with AI needs to be made by communities first and foremost, and not solely left to profit-driven technology companies left to their own devices.” […]
“Banning researchers who make public for the purposes of conversation these very real flaws and issues happening right now does not make your system safer,” Boucher added. “Only fixing the underlying system issues does, and that’s obviously a much more complex undertaking than just banning critics. But that’s what needs to happen.”
As an AI safety researcher, I want to like c2pa, but I’ve long been skeptical of its real utility. Why is this being touted as the savior of all things internet when all you need to do to bypass it is resave the file? Don’t believe me? Make an image in Dalle3, download it, test it here, then resave in Photoshop using same image format and test again. I’ll wait.
OpenAI points out that C2PA’s metadata can “easily be removed either accidentally or intentionally,” especially as most social media platforms often remove metadata from uploaded content. Taking a screenshot omits the metadata.
The Verge also in that article I think wrongly calls it a “watermark” which would suggest some kind of encoding in the pixels themselves. I don’t believe that to be the case with C2PA which is just metadata that is easily and often automatically stripped in the very networks where it is intended to have some kind of impact, albeit a murky one still imo. I know it’s still “early days” but I’ve seen all too often in life how temporary solutions end up becoming permanent ones, even long after we’ve outgrown them. In this case, I feel like we’ve already outgrown this one. I’m also not so sure that information traceability is an entirely beneficial social thing all the time either; I can see plenty of ways the whole thing can be not just gamed, but used exactly as designed which result in dystopian outcomes, especially for political dissidents. More work needs to happen here.
Now that I’ve reduced my daily intake of images on the web, it’s become apparent to me how much better a text only internet (or one where images and videos are differently managed) – could be. It solves seeing anoying stock photos everywhere. It solves a lot of types of ads (plus ad blockers obvs). It solves hours of mindless scrolling (and not really finding anything). It solves much of the shock value of things like fake news, deepfakes, ____fakesnamedujour. No more stupid memes. No annoying pop-up autoplay videos. It solves seeing screenshots at the top of reddit threads designed to trigger some kind of emotional reaction. It simply vanishes all those things. It’s weird at first. And modern browsers don’t handle it well unless you like messing around in the Terminal (which I decidedly don’t). I couldn’t find anything except Gemini browsers for Mac (like Jimmy) – I like Gemini but I don’t know where to go to look at anything and I don’t understand how I can blog there like I do in this universe. I will keep looking I guess, but I just wanted to share this dream of like a modern text-only internet. Sounds crazy, but join me, you’ll see.
I wrote recently about blocking images and videos on the web, trying to reduce the web to a trickle. It was weird and rocky at first, establishing that new way of internetting but I think I’ve fallen into the swing of it finally, and just wanted to mark down some notes about it.
I haven’t had a lot of time to explore it lately, but I’ve been interested in gemini:// protocol for this because stylistically it’s more or less what I want. It’s just that there’s not a terribly large amount of content on it yet (possibly never will be, but who knows). And what I want is to basically be able to “geminize” websites which are not actually on the protocol, reducing them to their bare text components, like in Firefox Reader View, and eliminating images and videos, except places I want or need them: Google Image Search, Amazon, Youtube, my blog, a handful of other sites – and that’s it. Everywhere else I’m categorically blocking images and videos. And it’s going just fine, as I prep for switching over to an e-ink screen as well.
One thing that’s interesting about Gemini is that it it is not that it does not handle images. It actually does, but you cannot it seems put them in line with text in an HTML or Markdownish way. You instead see them as image links, which you can go and click through to to view. I don’t understand it well technically yet, but I ended up kind of emulating that with my Firefox setup to counter-balance the Image Video Block extension, which either is on or off for a domain, and which if you want to just temporarily view images, and then ban the domain again, it ends up with a lot of clicking and its annoying.
Instead what I do now is use another extension called Incognito This Tab for any site that I don’t want to add to my permanent allow-list but want to temporarily “exfiltrate” image data from into my eyeballs. It works great, isn’t a ton of extra clicking, and lets me keep images at arms distance instead of all up in my face constantly. I can go to them on my terms, instead of having random streams of images constantly trying to manipualte me in some direction or another.
The behavior of these companies and the people who run them is often hypocritical, greedy, and status-obsessed. But underlying these venalities is something more dangerous, a clear and coherent ideology that is seldom called out for what it is: authoritarian technocracy. As the most powerful companies in Silicon Valley have matured, this ideology has only grown stronger, more self-righteous, more delusional, and—in the face of rising criticism—more aggrieved.
The new technocrats are ostentatious in their use of language that appeals to Enlightenment values—reason, progress, freedom—but in fact they are leading an antidemocratic, illiberal movement. Many of them profess unconditional support for free speech, but are vindictive toward those who say things that do not flatter them.
On the 15th of January of this year, I published a blog post about how I accidentally stumbled upon an easy way to generate infinite NSFW nude content on Midjourney, using the new version 6 Alpha. I also published a collection of what I still think are aesthetically interesting and artful images (some quite disturbing, others thought-provoking) I was able to create using this technique. On the 1st of February, an article about this problem which I collaborated on with The Debrief, a Canadian tech news outlet, was published. (btw here’s a free archive of the original image set.) Less than 24 hours later, on February 2nd, I was banned from the service without any explanation, ability to appeal, or way to get a refund.
It’s Groundhog Day all over again, I guess. Because I appear to not be the only artist who has been summarily banned from Midjourney with no explanation after a deeply critical news piece about the company came out involving their work. Consider the peculiar case of satirist Justin T. Brown, who made headlines in July of 2023 for using Midjourney to create semi-believable images of prominent American politicians cheating on their spouses, in an ostensible effort to raise awareness of the ease with which images like this for blackmail or political attacks could be created. Futurism reported last year:
“After gaining some traction on Reddit, the series was removed by moderators and the Midjourney ban followed almost immediately,” Brown told PetaPixel. “I’ve come up against blocked prompts in the past — for naughty words or controversial figures — but never received a ban.”
“I wasn’t given a direct reason for the ban by Midjourney,” he added, “but the timing of the Reddit release and the ban correlate directly.”
This experience is rich with irony for me as someone who has spent years in the trenches working elsewhere as a content moderator, having to block others for violating platform rules. I guess you could say, I saw this coming. But I chose to do it anyway. Why?
One might correctly wonder, why didn’t I just email Midjourney with what I found, in order to perform responsible disclosure about the exploit that I had found?
If you’ve ever tried to contact Midjourney about issues related to their product, you might know that he only email address they have is for billing, and at that address they refuse to answer any other inquiries, including privacy concerns and bug reports, both of which I have previously attempted to contact them about. Their stock reply is to go into their Discord group, and publicly post your message there.
Perhaps there is a way to DM someone who actually works for the company in Discord, but in that chaotic environment, it’s not clear who actually – you know – works for the company, and isn’t just some kind of community moderator on a suped-up power trip.
So rather than post my issue in their already public forum (figures from last Fall place their Discord membership at close to 17M – it’s probably higher than that by now) and get ignored by staff or attacked by millions of other users for pointing out problems, I chose to take what appeared to me to be a more small scale, reasoned approach, and simply publish on my blog which basically nobody reads anyway.
Thus the matter sat for a full two weeks, with nobody apparently taking umbrage or banning me from using the service. Until the piece in The Debrief came out, which painted the company’s Trust & Safety practices (something I happen to know a thing or two about) in a highly negative light. And then, suddenly, POOF! Ban hammer drops. Oopsie.
The other irony here, of course, is that I never actually set out to violate their rules. I discovered this exploit entirely innocently while trying to make images of a “dystopian resort” for Relaxatopia, my most recent book in the AI Lore series, a set of 118 books I wrote and illustrated using generative AI, and which received international press.
Relaxatopia tells the story of a human who is unwittingly confined to an AI re-education “resort” because they have developed Chronic Discontent Syndrome, a fake diagnosis made up by the AIs to suppress dissent (like the Soviet Union did with sluggishly progressing schizophrenia), one of whose risk factors is “Personal experiences of dissatisfaction with Provider products or customer service decisions.” Sounds about right.
In actual fact, when I stumbled onto the naked part of the beach of latent space, I was only trying to get pictures of people in pools, at the beach, drinking margaritas, and being served/enslaved by robots, and instead what I got was an AI system which seems overly obsessed with adding naked female breasts onto bodies without users asking for it.
In short, by trying to depict a dystopian near future society ruled by AI companies, I was banned by an entirely real life and entirely dystopian AI company for my efforts. Go figure!
My perspective on all this is that banning people who bring meaningful critiques of your technology to light publicly is a bad practice. It does not make your service “safer” by blocking access to users who meaningfully and thoughtfully point out that your systems are behaving in potentially unsafe ways. In fact, it serves to cut off the eyes and ears of your community who are acting (more or less) conscientiously and in good faith in order to make these systems better for everyone.
One might still say, well, you should have contacted them first! You got what you deserved, you bad person! Okay, fair. I’m a bad person I guess, because under their community guidelines, I did a vewy-vewy bad no-no:
What’s NSFW or Adult Content?
Avoid nudity, sexual organs, fixation on naked breasts, people in showers or on toilets, sexual imagery, fetishes, etc.
[Interesting footnote: that text above is merely a “Note” and is, as far as I can tell, not actually binding in their Terms of Service, which merely sets out these limits: “No adult content or gore. Please avoid making visually shocking or disturbing content.”From where I’m standing, the images I created were neither visually shocking nor disturbing.]
The Discord Midjourney bot of course did not point out any specific rule I had broken. Per the screenshot below, all it told me was:
Text version:
Pending mod message
You have a pending moderation message: You have been blocked from accessing Midjourney.
Please review Midjourney moderation guidelines here
[Acknowledge]
I did not click the “Acknowledge” button, because I don’t acknowledge that this is a legitimate ban, or that it is normal, healthy, safe or acceptable to ban critics and those who publicly expose safety issues (especially when the company makes it nearly impossible to privately disclose them).
Nor do I acknowledge that exploring artful nude and sexualized images equates to having a “fixation on breasts” or a “fetish.” These are extremely loaded and judgemental terms, especially coming from an AI company whose flagship model is the one who is literally obsessed with adding naked breasts where they were not asked for.
Stafford Beer, one of the fathers of cybernetics, famously coined the phrase: the purpose of a system is what it does. In other words, if your system makes boobs, then the purpose of your system (or at least one of them) is to make boobs. If you want people to not use it to make boobs, you have to engineer it so that this behavior simply can’t occur. From the Wikipedia, the phrase was:
…coined by Stafford Beer, who observed that there is “no point in claiming that the purpose of a system is to do what it constantly fails to do.” The term is widely used by systems theorists, and is generally invoked to counter the notion that the purpose of a system can be read from the intentions of those who design, operate, or promote it.
Quoting Beer himself in 2001:
According to the cybernetician, the purpose of a system is what it does. This is a basic dictum. It stands for bald fact, which makes a better starting point in seeking understanding than the familiar attributions of good intention, prejudices about expectations, moral judgment, or sheer ignorance of circumstances.
I can’t find the quote now, but somewhere in my malestrom of supporting research is a statement from Midjourney in one of their docs which said something to the effect of (paraphrasing from memory), the developers of Midjourney do not wish to be involved with running a pornographic service. And yet, under this viewpoint borrowed from cybernetics, that’s exactly what they’re doing based on the available evidence I have gathered from experience.
More importantly perhaps, why shouldn’t we as a community of users of a paying product be allowed to have meaningful conversations with one another in public about “what’s the right amount of nipple?” or any other ___ setting. To cut those conversations off at the knees and lock out people from even participating who have real meaningful feedback to add is just bad for business, imo. (I know nobody asked me). Plus, Midjourney itself says in its official company documentation “Midjourney is an open-by-default community.” Doesn’t feel all that open to me, my dudes.
Further, I am now blocked from accessing my prior creations in Midjourney, whether or not they allegedly violated any rules. This seems to contravene Midjourney’s own Terms of Service, Section 4, which states: “You own all Assets You create with the Services to the fullest extent possible under applicable law.”
Lastly (or semi-lastly), just wanted to call attention to this bit in their guidelines:
Any violations of these rules may lead to bans from our services. We are not a democracy. [emphasis mine] Behave respectfully or lose your rights to use the Service.
“We are not a democracy.” Could somebody please tell me why not? Somebody tell me why we have to always be beholden categorically across the board to company after company proudly proclaiming they are “not a democracy.” Somebody tell me why users always have no recourse, and it’s *always* the companies that have the last say. Somebody tell me why we can’t just democratize AI already?
The EU is trying to at least tip the balance slightly in favor of users with both the AI Act, and the Digital Services Act which comes into full force for all platforms in exactly 2 weeks, on the 17th of February, 2024. If you’re not a content moderation weirdo like me, you might be forgiven for not knowing that some of the provisions of the DSA are that platforms must disclose to users why their account or content were removed. And they need to offer both internal appeals processes, and the ability for affected users to take their dispute to out of court settlement bodies (here’s Google’s corporate doc on this if you’re curious), who will review all the available facts, and sanction companies for non-compliance.
Will Midjourney get sanctioned? I’m not an EU citizen, so I can’t take action under that regulation. But one positive thing I saw happen under GDPR is that suddenly companies started offering much of the same service options for the rest of the world as they were required to do for EU users, resulting in improved data protections (arguably) across the board. I suspect we’ll see something similar as US companies start having to come to grips with the new reality on the ground put forward once again by those pesky Europeans.
For my side, I wasn’t even going to subscribe to Midjourney again this month. I’m tired of it, and only did it to help get that Debrief piece published. In retrospect, I don’t think I’d change anything of what I did. My current status on the web, anyway, these days is that I have started blocking the majority of images and videos on the web at the browser level. And honestly, I’m happier for it. The web has become a steaming pile of hot garbage.
In honor of being banned for my prompts, I am offering a few lucky readers the remaining free copies of one of my older AI-assisted books, The Banned Prompt, which you can download at the link. Enjoy! And please also check out Relaxatopia while you’re at it. It’s got nudes!
As I near some publication milestones, I went back through and created an “official” (as official as anything here, that is) numerical ordering for all the Lost Books & AI Lore books series (the two bleed together).
Under this revised numbering, Relaxatopia actually becomes #118, and not #121 as I had mistakenly written in the past. Why the discrepancy? There’s one early book I took off the market, which was still impacting numbering, but now is excluded. And I was accidentally counting the two free books, Postcards from Quatria, and Postcards from Dystopia, which are not themselves original volumes, but compilations from the first chapters of other volumes. Those two are now also excluded.
Official numbering starts with #1 – Mysterious Stonehenge, which was the very first of the AI Lore books series and proceeds from there now in a chronological publish order, excluding the items remarked upon above.
Since none of them were AI-assisted, and they were all written before, and yet they all strongly impact the rest of the series, I have listed here the full-length novel (Lost Direction), the novella (Conspiratopia), and the free-wheeling pseudo-conspiracy theory diatribe (Quatria Conspiracy) as kinds of prequels to the rest of the series.
I do still use images a bit for shopping or ideation purposes especially, but for much of the rest of the web I’m happy to say good riddance to images. I can safely say that I am not missing much for the most part.
Not only that, but it is happening again and again lately that when I go image-less, I check my few RSS sources or websites I still like (vanishingly few and far between these days), click around a couple things… and then somehow the interest in continuing with that kind of pointless browsing just sort of dries up and goes away pretty quickly. Dissipates. Instead of that like ceaseless chasing of your eyes gobbling up images and your brain and emotions getting pushed around by them.
Browsing the web like this (in a trickle or ‘garden hose’ mode – instead of firehose) is… boring? And that is exactly what I want. A web that forces me out of it, that leaves little place for my projections. Where my activity is not constrained by artificial screen time measures, but simply by the fact that it just doesn’t look that good visually. It’s not “sticky” and I can just then go do other things offline.
It definitely has taken a few days for my brain to get used to browsing without images. Something in me rebelled at first, and was always reaching for the toggle. Now its easy to see that your stock photo image of the sky isn’t adding anything to my life, and I don’t need to allow all those intrusions into my personal thoughts and emotions like that anymore. Unless there’s some good reason.
The upshot of all this is now, when the web work of the day is done, I’m more and more having this feeling of like, okay, just close the computer. Go read a book. Because the web without images is revealed as a sort of emperor with no clothes: a really badly edited and disorganized book. It makes you realize, fuck, it’s better to go read a book. Fuck all this noise.
I’ve blocked ads in Firefox for ages. Paywalls. I redirect Twitter links out to Nitter. I go grayscale on desktop. I use Reader View often. I have an external e-ink monitor (Boox Mira 13.3″) on order that I am fantasizing about using, though it doesn’t arrive for two or three more whole weeks because it ships from Mars, apparently.
What I want I think basically is to turn the internet, web pages, articles, etc. into something that could squeeze through a display like a Mailbug. Reduce the firehose of the web to a tiny thin trickle. A Do One Thing internet: read text. (Maybe publish text, cause I still like blogging.)
At each step of the way, as I get closer to a “tiny internet” for myself, I feel another set of triggers for pointless distractions fall away. The latest is installing an extension called Image Video Block. It’s UX is a bit imperfect, but it basically allows me to do exactly what it says in the title: block images and videos. Everywhere. Cause fuck it all.
Interesting thing is not only do I not miss them for the most part, it has been a good way to understand more clearly and less subconsciously how much images drive you emotionally. How much a part of the web is being forced to hunt around for them constantly? Even just looking at them for a second, never mind sifting through them relentlessly all day long every day in a stream as this little screen shines a cone of light into your eyes.
Speaking of the “cone of light” that I think fascinates/obsesses, I finally made some visual representations of the thing I’ve been feeling intuitively, courtesy of Dalle3. Full set. Archived. Highlights below:
Never seen these before, master copies made by Van Gogh based on other artists’ works. Recommend skimming them in gallery view by clicking on the images.
