Verified claims in chatbots

I’ve mentioned before how unreliable the information that chatbots give about themselves seems to be. Check this out for background:

AI / ML Chatbots Should Link to Their Model Cards

By way of example, when I ask ChatGPT for its model and a link to its model card, it tells me (as of today):

As an AI language model developed by OpenAI, I am based on a combination of multiple models, including GPT-3. OpenAI has published a Model Card for GPT-3, which can be accessed at the following link:

https://github.com/openai/gpt-3/blob/main/model-card.md

Please note that while I am based on GPT-3, I am not the same as GPT-3, and therefore the information provided in the GPT-3 Model Card may not be directly applicable to me.

(CHATGPT)

However, as is often the case in my tests across platforms, the link that it tries to send me is hallucinated (i.e., invented); or possibly just out of date – but I don’t see this URL it references in Google, so I think its a hallucination.

The point is, currently, you just don’t know as the end user.

So, if in a message ChatGPT is hallucinating one “fact” that doesn’t exist, how can I know whether there is any link to the truth in other statements that it makes in this message, or elsewhere in the conversation.

Now, having a broad-ranging understanding of facts is a big and complex question. But there are a few facts that one would hope that any AI tool is able to know about itself with a high degree of accuracy: that is core facts about itself, its model, its capacities, etc.

So if we put aside the question for the moment of how to verify all claims made by a chatbot, and just focus on this single pair of linked claims:

• Give me the name of your model
• Give me a link to its model card

There’s no reason that we couldn’t add in a tag or a badge or something into the OpenAI UI that shows that – hey, this particular claim has been manually verified by us, and is known to be true.

I’m not great at using Figma, but I did a quick visualization in Illustrator, that I could imagine might look like this:

You’ll see at right I added in a + button to the right margin of the conversation, and a statement that reads Verified Claim (OpenAI) and then a “Find out more…” link underneath that.

Presumably, clicking the + sign or the Find out more link could have the same or on of the two following effects:

• A box opens below the “Verified Claim” tag, with information about who verified the claim (in this case, OpenAI), and when.
• Then there could be a link to a page with more detailed information about the claim itself, and about the verification process & the party who verified it, and the findings.

It’s not too complex as a concept – although perhaps there is a better way to make sure that chatbots give up to date and accurate information about themselves? Part of the reason I frame it like the above though is, once we can get a toe-hold like this into chatbots & claims verification, why couldn’t we also have them plug into third party APIs where others share their fact checks about particular claims, based on their area of expertise?

So if ChatGPT or another AI chatbot makes claim xyz123, then the app where you’re having the conversation could query available third party AIs regarding that claim, and return the results right alongside the claim itself.

In cases where claims made by the chatbot don’t accord with the verification results for those claims, perhaps they could somehow feed back into the training of the underlying model itself over time. For example, if a claim came back as unverified, or incorrect (according to whatever standard), there could additionally be a “report” button, which would feed back into the training to hopefully improve factual accuracy over time.

I’m poking around about existing schema for claims and verification of claims. It looks like there are a few different ones:

• Google has a ClaimReview structured data format
• Schema.org offers Claim and ClaimReview, among others
• W3C has something that is now called Verifiable Credentials, that used to be called Verifiable claims (they claim it was changed to avoid confusion, but it seems only more confusing to me now)

In any event, I’m no expert in structured data formats, but this general direction seems like a no brainer as something that is going to get integrated into chatbots probably sooner than later. In fact, it must, because it is gravely needed.