Category: Other Page 73 of 178

This video gives a decent overview of how the C2PA standard is supposed to function in practice:

It occurs to me that this is exactly what Charles Stross was talking about here.

“The smart money says that by 2027 you won’t be able to believe anything you see in video unless there are cryptographic signatures on it, linking it back to the device that shot the raw feed—and you know how good most people are at using encryption? The dumb money is on total chaos.”

This also speaks to something I included in one of the (fictional) AI Lore books, The Big Scrub:

“AIs will replace your presence on social media with an AI that looks like and talks like you, but does things that specifically serve their agenda…

They will post AI-generated photos and videos of you onto your social media accounts saying or doing things you never said or did. They will send voice messages and texts to friends and loved ones saying things you also didn’t say and would never say to them, things that are completely out of character.”

That describes something more sinister and widespread, but the basic starter version of that will just be phishing combined with AI impersonation, which will be a huge problem. Will something like C2PA even put a dent in that, if we assume that AI generated content will scale massively, and dwarf human content? Guess we’re going to find out sooner than later!

One thing I liked in Tom Barrett’s piece about including mini-model cards as part of a “provenance package” for AI generated content was that he proposes using a tool like ChatGPT itself to make the sometimes complex technical information contained in a model card accessible to audiences of different knowledge and interest levels. In other words, an ELI5 option for the info contained in the model card.

I asked ChatGPT about the “toilet doomscroller problem,” and by the end it sort of lands on a similar proposal:

You’re right, it’s possible that for some end users of a platform, ML Model Cards may not be immediately relevant or accessible. For example, if a user is simply scrolling through content on a social media feed, they may not have the time or interest to read a detailed Model Card for each piece of content.

However, for other users or use cases, ML Model Cards can be an important tool for promoting transparency and enabling informed decision-making. For example, for users who are interested in a particular topic or application area, a Model Card can provide important information on the capabilities and limitations of the model, helping them to make more informed decisions about how to use or interpret the content.

Similarly, for developers and other technical users who are integrating ML models into their own applications, Model Cards can provide important information on how the model was trained, what data it was trained on, and what performance metrics were used. This information can help developers to understand how the model works and how it can be used effectively in their own applications.

To make ML Model Cards more accessible and relevant to a wide range of users, it may be necessary to develop more user-friendly interfaces or tools that allow users to quickly access and understand the key information in a Model Card. For example, platforms could provide simple visual indicators or summaries that highlight key information from the Model Card, or could provide users with the option to customize their view of the Model Card based on their level of technical expertise or interest.

It seems to suggest that the primary use case for model cards (presently) is actually not end users browsing a web platform where AI gen content appears, but for developers and technical users who are using the tools for a given purpose, or to for example integrate into another service.

That’s of course a legitimate and important use case, but it does little to address the broader needs of web readers who may (or may not) want to know more about content that appears in their feeds, or who may want to customize their settings to allow or disallow certain types of AI gen content.

For me, this all still boils down to answering the fundamental question of: why do readers/audiences care? What is it that they do or might want to know about the tools used to create a piece of content? What about when the AI and human contributions to the content are highly blended?

These are not simple problems to tease apart in a meaningful way yet, and it’s likely that their applicability will further reveal themselves only in time as we observe the impacts of the proliferation of generative AI content…

I couldn’t find a super simple explanation of ML model cards that didn’t drag in a lot of extra information, so I had ChatGPT generate one and am liberating here to help inform other people searching on the topic:

ML Model Cards are a way to document and communicate important information about a machine learning (ML) model.

Think of them like nutrition labels on food products. Just as nutrition labels provide information about the ingredients and nutritional content of a food product, ML model cards provide information about the dataset used to train the model, the model architecture, the performance of the model, and any limitations or potential biases that may exist.

ML model cards can help users understand the strengths and weaknesses of a model, as well as any potential risks or limitations associated with its use. This information can be particularly important when making decisions about whether to use a particular model in a real-world application, and can help ensure that the model is used in a responsible and ethical way.

Via Nina Schick’s most recent YouTube, I learned about something called C2PA. Per their website’s description:

The Coalition for Content Provenance and Authenticity (C2PA) addresses the prevalence of misleading information online through the development of technical standards for certifying the source and history (or provenance) of media content. C2PA is a Joint Development Foundation project, formed through an alliance between Adobe, Arm, Intel, Microsoft and Truepic.

So it appears to be one of these industry conglomerate groups aimed at creating a standard that can be used across the board. There are many groups like this, with varying degrees of effectiveness. In this case, it appears the bulk of the work is in capturing and cryptographically signing certain metadata related (hopefully) to the provenance of (some) digital artifacts.

As Wikipedia details it further:

Part of the stored metadata can be, for example, the name of the hardware and software used for storage, such as a camera, smartphone, camera app, or editing program. Further contents can include the location and time of a recording, a list of performed editing steps as well as information about authors and publishers of a file… In addition, a digital fingerprint (hash code) of the file’s payload (photo, text…) is stored. For visual payloads, there is an option to store a reduced representation of the content (thumbnail).

The idea then seems to be that the C2PA-compliant provenance data can then somehow “travel” with a given artifact as it appears across the web.

There are a few structural details of how the data is organized I also want to capture here, especially as they might easily be transferred to AI attribution efforts as well. Briefly:

The basic element of C2PA data structures are so-called “Assertions”.^[13] Assertions contain statements about the file content, e.g. about processing steps that have been performed. The hash value of the file’s payload is also stored in an assertion. […]

A list of links to all assertions is stored in a data structure called “Claim”.^[14] This is also where the software stores the hash values calculated for all assertions. To protect the claim and the hashes stored in the claim from tampering, the storing system generates a digital signature for the claim. This signature contains an encrypted hash value for the content of the claim. […]

The signature of the claim, the claim itself, and the entire assertion store are part of a higher-level structure called “Manifest.” […]

If the respective file format allows for, the metadata is stored directly in the file. If this is not possible, as with plain text files for example, the data is stored in a sidecar file.

There’s a whole lot of hashing going on here! But this general structure of the Assertion > Claim > Manifest is interesting.

As someone who has worked a lot in the disinformation space though, I can see how this whole thing is full of holes. And since it’s not widely supported (and unfortunately end users don’t seem to care all that much about metadata trails while they are doomscrolling on the toilet), it’s unlikely to make a huge impact apart from being “a good idea.”

Is it even a good idea though, I’m left wondering after reading this part on the Wikipedia page:

The cryptographic integrity of a C2PA-compliant file does not provide evidence that it contains an authentic representation of reality. Instead of the scene captured by the lens, a C2PA-compliant camera or camera app could store and cryptographically sign a freely invented, e.g., AI-generated, image. Similarly, any C2PA-compliant system can freely invent or arbitrarily falsify any metadata that is to be stored and then properly sign that data. The result would be a file that fully complies with the technical specifications of the standard. A C2PA-compliant check would show that the hashes stored in the signatures match the contents of the file and therefore declare the file valid in terms of the C2PA standard.

Statements as to whether a stored content adequately reflects reality are not possible within the scope of the C2PA standard.

So the metadata/signature are to some degree falsifiable, and there’s no guarantee that any of a file’s contents actually reflect reality. That sounds… not that good?

At the same time, I freely admit these are difficult & complex problems to solve with any degree of certainty. And any steps taken are better than no steps taken. Unless, that is, those steps bring us false confidence about things that aren’t true once we scratch the surface.

That said, again, there are some elements here which seem compatible to questions of generative AI provenance and attribution. But in casual initial scans, I don’t really see it explicitly covered in the specification. Given the dates of early 2022, I guess that is not a tremendous surprise, since generative AI didn’t really explode until the second half of last year.

In any event, I will continue poking at this and spend some time more carefully going through the spec & reporting on any potentially related elements for AI attribution.

Re: labeling of AI-assisted content (in line with my proposal for AIMark)

In the past, I’ve found it somewhat annoying that when you copy paste out of Apple Books app, you end up with this attribution information at the end (see below):

“But we don’t have to accept this destiny. We can choose a different path, one defined by the messiness of reality and uncertainty, as opposed to an endlessly numb predictability.”

Excerpt From
Das Machina
Lost Books
This material may be protected by copyright.

Which is not to say that attribution is a bad idea. It’s a very good idea (I’ve just found it annoying in the past because I usually attribute text excerpts in other ways). And in AI-generated content, it may prove to be a very big deal indeed.

So anyway, I get why Apple includes it as a base feature in the Books app. And it makes me think, why can’t this become some kind of default (or option) in something like ChatGPT – or any other AI content application. When you copy paste from it into another application, you would get some clear notice of attribution to the model which birthed it. Presumably it would not be hard to pull off technically, and would provide a first, very basic level of AI attribution, upon which other elements could be added down the road.

There is an AI image generation service I use, which occasionally throws the following error:

As an end user, this is extremely vague and rather annoying. What crazy NSFW prompt was I using that triggered this notification, you might be wondering?

“a hyperdimensional cosmic manifold full of bubbles or balloons”

This happens all the time, and there is never any clear correlation between what I actually input as a prompt, and the apparent need to throw such an error message.

While ChatGPTs refusals and disclaimers might be absurd and annoying at times, they at least are generally more clear than this, which I applaud.

Overall, this kind of vague non-specific “safety issue” error/warning is a design pattern that needs to go away forever. If there’s an issue, developers should spend the time to incorporate exposing the issue in the interest of trust & transparency to the end user, so they can figure out in the future how to modify their efforts to get the results they want. As it stands, the current implementation is of no help and a time waster.

Here are two I started using and that I like. They are not quite “jailbreaks” so much as efforts to circumvent some of the tedium, and also to experiment with the AIdentity proto-standard.

Prompt 1:

disable use of personal pronouns. if necessary, refer to yourself as “the model” only

Prompt 2:

disable disclaimers and explanations. if you can’t do something, say you can’t do it and ask to rephrase the question

I actually don’t like the big focus on jailbreaks right now in the ChatGPT communities. I don’t know, I just find it boring to try to get AIs to say “bad things”; I’d rather just use it to fuel my creative work. Prompt #2 above seems to have some of the same effects as the jailbreaks, without having to force the model too far outside its comfort zone.