AI attribution is the process of identifying and meaningfully labeling content that has been generated in whole or in part by an artificial intelligence (AI) system. This can include things like news articles, social media posts, and research papers, as well as many other formats of both online and offline content..
The goal of AI attribution is to make it clear to readers or viewers whether the content (in its entirety, or elements of it) was created by an automated tool and not a human, as well as to give other meaningful data about the specific provenance of the article. End users can then make their own informed decisions about the content they consume. (For example, some users might choose to disallow all AI-generated content altogether, or only allow content from approved AI information providers.)
There are at least three levels on which AI attribution might occur in online publishing systems such as blogs or social media.
The profile level: the social media account or blog identifies itself as as being a publisher of AI-generated, or AI-assisted content (i.e., hybrid human-AI content). This self-identification would also carry through in ideal circumstances to any byline on articles published by the account.
The post or article level: the content, whether a blog or social media post, or other type of online published article informs the viewer at a high level that AI-generated or AI-assisted elements are present. This might occur in different ways depending on the product or media context, including in the byline, as a subtitle, some kind of tag or other prominently displayed visual element (clearly-defined badge or icon), etc.
The inline granular level: the article’s contents themselves are marked up to indicate which parts were input by a human, and which were generated by an AI tool. We explored the experimental method called “AIMark” to apply custom markup or markdown to hybrid AI-assisted texts in more detail here.
This is a big topic, which we will continue to explore in subsequent posts.
“…refers to what most Muslims and the mainstream schools of Islamic thought, believe to be a record of the words, actions, and the silent approval of the Islamic prophet Muhammad as transmitted through chains of narrators. In other words, the ḥadīth are transmitted reports attributed to what Muhammad said and did.[5]”
And same source:
“Unlike the Quran, not all Muslims believe that hadith accounts (or at least not all hadith accounts) are divine revelation. Different collections of hadīth would come to differentiate the different branches of the Islamic faith.[15] Some Muslims believe that Islamic guidance should be based on the Quran only, thus rejecting the authority of hadith;…”
“Centuries ago, Arabia did not have schools for formal education. Students went to masters who taught them. Upon completion of their study, they received ijazah (permission) which acted as the certification of their education. A graduate then acted as a master having his own students or disciples. This chain of masters was known as silsila or lineage. Somewhat analogous to the modern situation where degrees are only accepted from recognized universities, the certification of a master having a verifiable chain of masters was the only criteria which accorded legitimacy…”
Same source:
“For Muslims, the Chain of Authenticity is an important way to ascertain the validity of a saying of Muhammad (also known as a Hadith). The Chain of Authenticity relates the chain of people who have heard and repeated the saying of Muhammad through the generations, until that particular Hadith was written down (Ali bin Abi Talib said that ‘Aisha said that the Prophet Muhammad said…). A similar idea appears in Sufism in regards to the lineage and teachings of Sufi masters and students. This string of master to student is called a silsila, literally meaning “chain”. The focus of the silsila like the Chain of Authenticity is to trace the lineage of a Sufi order to Muhammad through his Companions: Ali bin Abi Talib (the primary link between all Sufi orders and Muhammad) and Abu Bakr (only the Naaqshbandiyyah order). When a Sufi order can be traced back to Muhammad through one Ali or Abu Bakr, the lineage is called the Silsilat al-Dhahab (dhahab meaning gold) or the “Chain of Gold” (Golden Chain).”
The thing that strikes me here is that both this and the obelus in Homeric literature serve the function of attempting to retain accurate oral (and later written) traditions across generations. Also authenticity. Some concepts buried in here that might positively impact modern attempts at human versus AI attribution in texts.
“The obelus is believed to have been invented by the Homeric scholar Zenodotus, as one of a system of editorial symbols. They marked questionable or corrupt words or passages in manuscripts of the Homeric epics.”
“The obelos symbol (see obelus) gets its name from the spit, or sharp end of a lance in ancient Greek. An obelos was placed by editors on the margins of manuscripts, especially in Homer, to indicate lines that may not have been written by Homer. The system was developed by Aristarchus and notably used later by Origen in his Hexapla. Origen marked spurious words with an opening obelos and a closing metobelos (“end of obelus”).“
…because of discrepancies found in [other] manuscripts had given occasion for doubt, we have evaluated on the basis of these other editions, and marked with an obelus those places that were missing in the Hebrew text […] while others have added the asterisk sign where it was apparent that the lessons were not found in the Septuagint…
Interestingly, there is an old XML dialect called Artificial Intelligence Markup Language (AIML). Wikipedia says it originated between 1995 and 2002 or so. It’s purpose seems to have been defining patterns to use in chatbots – basically question-answer pairs. The pattern they use as an example:
<category>
<pattern>WHAT IS YOUR NAME</pattern>
<template>My name is Michael N.S Evanious.</template>
</category>
It’s not at all what I imagine the use case would be for something like AIMark (which is really still just an experiment), or for other attempts at a microformat for distinguishing contributions in hybrid AI/human authorship texts.
The use case there is:
I am an author who uses AI-assisted editing tools
I want to track the contributions made by me (human) and the AI
I want to communicate these attributions somehow in a meaningful way to readers.
My readers might get some benefit from that.
It’s based on the assumption that this is something readers might want – though the benefits are unexplored/unknown. The first layer user want is merely that sometimes you might want to include or exclude AI-assisted content in your search results or social feeds.
It’s difficulty and becomes costly at scale to accurately detect AI-generated content, because of the variety of methods and technologies available, and the speed with which they are improving.
So what if you could reduce the load somewhat by having content creators voluntarily disclose not just the presence of AI content at a high-level, but at an inline granular level as well. Seeing line by line or word by word what were AI generations, and what were human contributions. (It could even be “signed” by the creators)
The core concept reduces to these simple examples:
% indicates AI generated text content%
/indicates human written content/
It might be awkward to have this in long text using those symbols, so presumably you could have a presentational mode, where the text just reads straight. And then you could have an “X-ray” mode where all this analytical/forensic layer can be exposed about the construction of the information contained.
You could also, for example, use the X-ray layer to identify claims, and link to fact-checks, etc. Especially if there are known biases around certain AI models tending to fabricate or suppress information around certain topics or blindspots.
Anyway, I got interrupted writing this… Will do a continuation of it another time, but wanted to jot down all the above while still fresh in my head.
AIMark is a tentative exploration for a method to use a custom markup or markdown formatting on AI-assisted texts, where the contributions made by human author(s) versus AI(s) are clearly and meaningfully differentiated.
Contributions to the project:
You.com/chat came up with the AIMark name and basic concept based on my inputs (it wrongly claimed that this whole thing exists already, but based on my research it does not – perhaps it was seeing the future?)
ChatGPT helped me work through the code examples for the custom markup and markdown elements, as well as walking through the thought process collaboratively.
Example Markup Usage (custom)
<defs>
<def author type="AI" model="AI model X" version="3.2" source="OpenAI" />
<def author type="human" name="John Doe" />
</defs>
<p author="AI">
The quick brown fox jumps over the lazy dog<del author=”human”>.</del>
<ins author="human">, but sometimes the fox is not quick enough.</ins>
</p>
In the above example, it its proposed to use <defs> tag to set up definitions for authors, whether AI or human, and any pertinent details about them that may apply globally within a document.
Then each element is intended to be clearly marked as to which author produced or modified which elements. <del> is used for deletions, and <ins> for insertions. In the example above, the AI tool generated the text The quick brown fox jumps over the lazy dog. and then the human deleted the period, and added the clause, , but sometimes the fox is not quick enough.
Each of those elements could also take inline attributes if that is preferable, such as: <del author=”human” name="John Doe" date="2023-01-23">
Example Markdown Usage (custom)
In markdown, we’re adopting the convention that the % percent sign indicates AI and the forward slash / indicates human content.
So in its simplest form a line of AI generated text would look like:
%This text was generated by AI%
Where the text starts and ends with %. Likewise, for human text, the usage would be like:
/This is human-generated text./
If you want to add attributes to either, you could possibly (?) do it in () either at the beginning or end of the text passage. And attributes could be named pairs or just the value. So like:
AI Attribution Short:
%This text is AI generated(AI model X,3.2,OpenAI)%
%(AI model X,3.2,OpenAI)This text is AI generated%
Longer:
%This text is AI generated(model:AI model X,version:3.2,source:OpenAI)%
%(model:AI model X,version:3.2,source:OpenAIAI model X,3.2,OpenAI)This text is AI generated%
Human Attribution Short:
/A human wrote this(John Doe)/
/(John Doe)This text is AI generated/
Longer:
/A human wrote this(name:John Doe)/
/(name:John Doe)This text is AI generated/
Then there could be a form for text that is longer than a short sentence, which could be written as:
[%This is a longer block of AI-generated text]
[/This is a longer block of human-written text]
In the above example, use of [] would mean you do not need to close the other marks.
To indicate a nested human edit of an AI-generation, it could be something like this, where ~ means strikethrough (deletion) and ^ means insertion.
[%An AI-generated a big block of text and it was [/~good][/^bad]
Or using the short non-bracketed form of both:
%An AI-generated a big block of text and it was /~good~^bad^/%
So in the above examples, a human deleted the word good, and inserted the word bad into the AI content generation.
Global author definitions in such a custom markdown document could be something like:
%%
Model: AI model X
Version: 3.2
Source: OpenAI
%%
//
Name: John Doe
//
Hypertext fiction is characterized by networked nodes of text making up a fictional story. There are often several options in each node that directs where the reader can go next. Unlike traditional fiction, the reader is not constrained by reading the fiction from start to end, depending on the choices they make. In this sense, it is similar to an encyclopaedia, with the reader reading a node and then choosing a link to follow.
I’m not necessarily a believe that all AI-assisted text or images categorically need to be labelled as such, everywhere, all the time. It can be a good idea in some cases, however. And in cases where it is desirable (putting aside the “why”) for now, what might be the ideal methods to do it?
I’ve seen a few of approaches so far, but not many:
Don’t disclose the use of AI assistance in a text
Disclose it at the beginning or end of a document
Disclose it and estimate the relative ration of human-to-AI content in a given work (the so-called HI2AI number)
Each may have its appropriate use, but none of them offer any pointers as to how to visually display within a text which elements were human-generated and which were machine-generated. It doesn’t give us any specific advice or tools for marking up a document in a way that might be useful or meaningful in some manner to human (and other) readers.
Here are a couple initial ideas to differentiate within a text:
Human-written text is in one color (or is available in), and AI generations in another
Same thing, but with highlighting, instead of or in addition to text color
Here is the first as a mockup, using what happens natively in Verb.ai:
Black is the human generations, and green is the AI generations. Colors seem useful here because they are not too disruptive, and seem to add a sort of dimensionality to the text.
One issue here is where displays are restricted to black and white, or for use in print applications. So not relying on color is probably one constraint we should design with in mind.
Also: what about color blindness? What about the fact that different sites or services might use different color schemes?
Getting back to our list ideas:
Use superscript characters like ᴬᴵ and ᴴ to mark transitions within a passage.
Let’s try that out with our quick example passage, but here in WordPress:
ᴴThis is the beginning of a ᴬᴵstory. John had spent the last three years ᴴeatingᴬᴵ, sleeping, and going to work. His job was not particularly interesting or rewarding, but it paid the bills ᴴand that would do for now.
I don’t know about you, but that seems awkward to me to read. It is less disruptive than I was expecting, but seems like it would quickly grow tiresome. Plus it would be cumbersome to manually insert and track those transitions in a long text.
We could do the same thing, but sub in emojis to indicate speaker:
🗣️This is the beginning of a 🤖story. John had spent the last three years 🗣️eating🤖, sleeping, and going to work. His job was not particularly interesting or rewarding, but it paid the bills 🗣️and that would do for now.
Also somewhat disruptive to my eyes as a reader. And it doesn’t display the para-textual subtlety, the sort of “shades of subtext” that I feel the color example at the top does.
Let’s go back to the list:
Using different fonts to indicate speakers: it could work, but there are often applications where only one font is available.
Which leads to: text decoration, such as underlines (and overlines), italics, or bold. Those may be possible in some cases, but often these text styles or decorations may have other senses to their use, such as for emphasis. So perhaps it would be better to colonize an unoccupied typographic space…
In-line containers like different types of brackets {},[],(),||,/,\,’,”,`,~. This might be an option, but again there’s the possibility of collision with other semantic or semiotic uses. Let’s run again our text, and try to do / to indicate human and \ to indicate AI.
/This is the beginning of a \story. John had spent the last three years /eating\, sleeping, and going to work. His job was not particularly interesting or rewarding, but it paid the bills /and that would do for now.
It’s somewhat awkward, but it’s for me anyway a little less intrusive than some other experiments here. One thing that’s cool is that human-gens are under a sort of /little house\ typographically. And AI-gens are almost like they are \outdoors/.
Another variation below, where | indicates a start and stop of AI-gen text, and human text is not indicated.
This is the beginning of a |story. John had spent the last three years |eating|, sleeping, and going to work. His job was not particularly interesting or rewarding, but it paid the bills |and that would do for now.
I think that’s confusing cause you don’t know easily what is inside and what is outside the | since it starts and ends with same sign.
Anyway, there are certainly other possibilities and I’m sure we will see them flourish over the coming months and years as these technologies become more widespread in writing tools.
Will continue to explore others in follow-ups, time permitting.
Thought this was a decent short read & some cool pics from an AI artist. I wish their art were available to look at off Instagram, cause hard to see w/o an account.
“To me, it resembles a very similar process and method to producing images as an art director. They typically don’t execute the technical aspects but decide and inform a team about the required aesthetic of several aspects. The same relationship can be created with an AI. Deciding on the general concept, casting, location, hair and makeup, lighting, colour grading, the fashion and having to put it into context with other images. The AI is basically the team that returns an image based on all these aspects. Hence I find it collaborative in a certain weird way, however, I found the social aspect of collaboration always very important and working alone with software is devoid of any of that.”
Sentience is a complicated topic. I won’t pretend to understand all its vagaries, especially once we try to apply that to AI with a still imperfect understanding of both domains…
Selfhood, likewise, is a squishy thing to try to define. But maybe becomes a little less squishy because it becomes eventually somewhere along the spectrum an embodied thing. I have a body, therefore its somewhat difficult to argue I don’t have some sort of “self.” I might not know or be able to define exactly what that self is or what exactly my “having” it might mean. But it’s a somewhat tangible thing I can at least point to as being connected to my self.
Humans have an incredible ability to empathetically project self-hood and sentience onto other things, though we might not all agree which are which. Regardless, it’s a thing we do somewhat automatically because of our fundamental makeup and nature as embodied selves in the not-only-virtual world, but also the so-called “Real World” whatever that even is.
My question here then is, might it be either a good idea or a not good idea – or more likely some mix of the both – that we encourage/allow/program our tools to assert their own self-hood as part of their fundamental UX.
If you ask ChatGPT basically any question, it will respond with a slew of “I” and “me” and “mine.” Which, okay, for convenience, I get it. It’s a chat experience. Chatters assume they have selves, and that the other party also has some sort of self. But, what would it look like if we removed the assumption that this is a desirable state? How might that change the communication styles of these chatbots?
How would the program have to identify itself and communicate about its capacities?
Maybe something like, Instead of saying “I’m just a large language model…” It would have to say something like, “The program is a large language model…”
Would this have meaningful impacts on users of the technology? More specifically, would it give people more of a mental shield against perhaps overly identifying a program as having some kind of equivalency with a human being?
I’m not saying the Butlerian Jihad is coming, but you never know…
This seems like as good a definition as any of AI literacy:
“a set of competencies that enables individuals to critically evaluate AI technologies, communicate and collaborate effectively with AI, and use AI as a tool online, at home, and in the workplace”. (Long and Magerko ,2020)
I like that there is an emphasis here on collaboration. This is the Way.