I’ve become a massive user & mostly fan of Midjourney, starting with this latest v5. It’s actually been only a handful of times that I’ve encountered their keyword filtering in the wild, and it has been overall pretty mild and mostly silly. For example things like butt and bathing are banned.

The filtering, which seems purely based on a keyword list, appears to be much more primitive than that of OpenAI’s Dall-E (though it’s been a while since I tested that). It’s usually easy enough to get around, which I think is sort of pointless. If I can get around your filter by changing a word or phrase, then you’re not really filtering at all, are you?

There’s a story that’s developing however that makes their filtering I think look a whole lot worse. Via Washington Post originally (I believe), many sources now are reporting on Midjourney’s keyword filtering blocking phrases related to the President of China, Xi Jinping.

Midjourney CEO David Holz is quoted as saying, by WaPo:

“We just want to minimize drama,” the company’s founder and CEO, David Holz, said last year in a post on the chat service Discord. “Political satire in china is pretty not-okay,” he added, and “the ability for people in China to use this tech is more important than your ability to generate satire.”

Effectively blocking expression around a sole world leader is not really what I would consider “minimizing drama.”

I can make hundreds of images, after all, of young Putin as a pot-smoking hippy, Ted Cruz as the Zodiac Killer, Ronald Desantis crying at Disney World or in an 80’s romantic comedy about a vacuum cleaner salesman, of a brawl breaking out at Trump’s arraigment, or of Trump and Hillary kissing.

But if I even type in the two letters xi, I get scolded that its a “banned prompt” and circumventing it may get me kicked off a platform I’m paying to use.

I wouldn’t care as much if this activity were actually illegal in my country; then it would make sense for it to be blocked by a company offering a service in my country. But I am not in China. I’m not a Chinese citizen. I’m in no way bound by their rules or laws or social norms.

In my experience as a user and working in moderation & policy with platforms, blunt force tools are very rarely a good and viable answer. They might be a short term fix, but they never solve the problem, and then introduce too many casualties along the way for them to be desirable.

What’s the obvious low-hanging fruit solution here then? Simply detect my country. If county == china, then activate the custom legally-required (presumably) filtering list for that country. Geoblocking is already a normal and more or less accepted practice that doesn’t force everyone to be bound by rules that don’t naturally apply to them otherwise.

The other ridiculous aspect here is that though phrases like “chinese president” are banned, you can simply re-order it in your prompt as “president chinese” and it will shoot out picture after picture of Xi Jinping, with no questions asked, and no threats of revoking your access.

If word a + word b is (allegedly) a rule violation, but word b + word a is not, then you really don’t actually have a rule, because it is inconsistent. You have merely a pattern that is being matched against, and not a coherent rational policy.

Plus, of course, you can input an image URL of Xi from elsewhere, and use that as the basis of the image prompt w/o any issue. So there are a million loopholes here.

And don’t get me wrong: I don’t actually want them to tighten the loopholes. I don’t want them to switch off simply naive keyword matching to something more sophisticated. I want it to be leaky & shoddy filtering. If anything, as a paying adult user, I want even less filtering. And I certainly don’t want to have to follow arbitrary restrictions imposed by authoritarian regimes that I don’t even live under.

Related writing & research