Questionable content, possibly linked

Verified claims in chatbots

I’ve mentioned before how unreliable the information that chatbots give about themselves seems to be. Check this out for background:

By way of example, when I ask ChatGPT for its model and a link to its model card, it tells me (as of today):

As an AI language model developed by OpenAI, I am based on a combination of multiple models, including GPT-3. OpenAI has published a Model Card for GPT-3, which can be accessed at the following link:

Please note that while I am based on GPT-3, I am not the same as GPT-3, and therefore the information provided in the GPT-3 Model Card may not be directly applicable to me.


However, as is often the case in my tests across platforms, the link that it tries to send me is hallucinated (i.e., invented); or possibly just out of date – but I don’t see this URL it references in Google, so I think its a hallucination.

The point is, currently, you just don’t know as the end user.

So, if in a message ChatGPT is hallucinating one “fact” that doesn’t exist, how can I know whether there is any link to the truth in other statements that it makes in this message, or elsewhere in the conversation.

Now, having a broad-ranging understanding of facts is a big and complex question. But there are a few facts that one would hope that any AI tool is able to know about itself with a high degree of accuracy: that is core facts about itself, its model, its capacities, etc.

So if we put aside the question for the moment of how to verify all claims made by a chatbot, and just focus on this single pair of linked claims:

  • Give me the name of your model
  • Give me a link to its model card

There’s no reason that we couldn’t add in a tag or a badge or something into the OpenAI UI that shows that – hey, this particular claim has been manually verified by us, and is known to be true.

I’m not great at using Figma, but I did a quick visualization in Illustrator, that I could imagine might look like this:

You’ll see at right I added in a + button to the right margin of the conversation, and a statement that reads Verified Claim (OpenAI) and then a “Find out more…” link underneath that.

Presumably, clicking the + sign or the Find out more link could have the same or on of the two following effects:

  • A box opens below the “Verified Claim” tag, with information about who verified the claim (in this case, OpenAI), and when.
  • Then there could be a link to a page with more detailed information about the claim itself, and about the verification process & the party who verified it, and the findings.

It’s not too complex as a concept – although perhaps there is a better way to make sure that chatbots give up to date and accurate information about themselves? Part of the reason I frame it like the above though is, once we can get a toe-hold like this into chatbots & claims verification, why couldn’t we also have them plug into third party APIs where others share their fact checks about particular claims, based on their area of expertise?

So if ChatGPT or another AI chatbot makes claim xyz123, then the app where you’re having the conversation could query available third party AIs regarding that claim, and return the results right alongside the claim itself.

In cases where claims made by the chatbot don’t accord with the verification results for those claims, perhaps they could somehow feed back into the training of the underlying model itself over time. For example, if a claim came back as unverified, or incorrect (according to whatever standard), there could additionally be a “report” button, which would feed back into the training to hopefully improve factual accuracy over time.

I’m poking around about existing schema for claims and verification of claims. It looks like there are a few different ones:

  • Google has a ClaimReview structured data format
  • offers Claim and ClaimReview, among others
  • W3C has something that is now called Verifiable Credentials, that used to be called Verifiable claims (they claim it was changed to avoid confusion, but it seems only more confusing to me now)

In any event, I’m no expert in structured data formats, but this general direction seems like a no brainer as something that is going to get integrated into chatbots probably sooner than later. In fact, it must, because it is gravely needed.


Interview with Joanna Penn


Readers becoming writers


  1. Tim B.


    I’m very very wary of turning to AI to fact check other AIs…

    holy hell this is shocking:

    “Around 130,000 fact-checks have been written by all fact-checkers around the world,” says Dudfield, citing a 2021 paper,

  2. Tim B.

    eventually i will do a better job of synthesizing all these disparate pieces together, but here’s an overlapping set of thoughts:

  3. Tim B.

    i can also imagine a scenario where:

    A) the chatbot has the answers it would give to queries based on its regular training (in this case, verifications/corrections might merely show alongside, but without impacting the actual query/prompt results)


    B) despite what the training might be, there are answers “locked in” to specific queries (like administrative queries about the model & company responsible) ?


    C) the model has its training & likely responses, but it is made aware of the results of verifications/corrections, and can either optionally or by default have them conform to the verifs & then cite them as sources/reasons for diverging from its training

Leave a Reply

Powered by WordPress & Theme by Anders Norén