AIer, AIer, pants on fire

I've already mentioned my own experience with artificial intelligence hallucinations when Google's AI got the facts wrong about the BICEP2 telescope, and when ChatGPT made up an entire industrial accident when asked to provide details of one at the BBC. But if an article by Amanda Guinzburg is true, the ability of AI to mislead and directly lie is even worse that many of us thought.

I use that 'if an article... is true' format because I don't know Guinzberg and have no way of verifying the truth of the experience she describes - but if it is real it surely should put a significant nail in the coffin of using generative AI for many text-based purposes where an accurate response is essential.

In the article, Dialbolus ex Machina, Guinzberg describes asking ChatGPT to select from a number of online articles she has written to include as examples in a query letter. ChatGPT makes suggestions with explanations as to why it has chosen these articles. So far, apparently so good. But confused by one of ChatGPT's responses, Guinzberg asks if it is really reading the articles. ChatGPT says 'I am actually reading them - every word.'*

Guinzberg points out quotes that aren't in her piece, and ChatGPT admits that it 'messed up'. As it couldn't access the full piece it made assumptions about what was in it. After another total miss, ChatGPT confesses 'I didn't read the piece and I pretended I had.' And so it goes on.

You can read the full article, with alleged screenshots of the conversation here. As mentioned, I have no reason to suggest that this item is made up (other than one reading of the subtitle 'This is not an essay'), other than that I don't know the author and have no way of checking the validity of the evidence.

[UPDATE - 14:48, 23 June 2025]

Thanks to Sally Bean for pointing out this equally blatant example of lying. What is particular fascinating is the comments on Sam Coates' X feed, blaming him for being shocked when this is what he should expect an LLM to do. The commenters, I would suggest, entirely miss the point that such models will remain pointless and dangerous unless this can be avoided:

* Emphasis in original. Incidentally, ChatGPT uses that classic generative AI giveaway the m-dash, rather than a hyphen.

Image from Unsplash by Payal Asthana.

These articles will always be free - but if you'd like to support my online work, consider buying a virtual coffee or taking out a membership:

See all Brian's online articles or subscribe to a weekly email free here

Comments

Why I hate opera

If I'm honest, the title of this post is an exaggeration to make a point. I don't really hate opera. There are a couple of operas - notably Monteverdi's Incoranazione di Poppea and Purcell's Dido & Aeneas - that I quite like. But what I do find truly sickening is the reverence with which opera is treated, as if it were some particularly great art form. Nowhere was this more obvious than in ITV's 2010 gut-wrenchingly awful series Pop Star to Opera Star , where the likes of Alan Tichmarsh treated the real opera singers as if they were fragile pieces on Antiques Roadshow, and the music as if it were a gift of the gods. In my opinion - and I know not everyone agrees - opera is: Mediocre music Melodramatic plots Amateurishly hammy acting A forced and unpleasant singing style Ridiculously over-supported by public funds I won't even bother to go into any detail on the plots and the acting - this is just self-evident. But the other aspects need some exp...

Why backgammon is a better game than chess

I freely admit that chess, for those who enjoy it, is a wonderful game, but I honestly believe that as a game , backgammon is better (and this isn't just because I'm a lot better at playing backgammon than chess). Having relatively recently written a book on game theory, I have given quite a lot of thought to the nature of games, and from that I'd say that chess has two significant weaknesses compared with backgammon. One is the lack of randomness. Because backgammon includes the roll of the dice, it introduces a random factor into the play. Of course, a game that is totally random provides very little enjoyment. Tossing a coin isn't at all entertaining. But the clever thing about backgammon is that the randomness is contributory without dominating - there is still plenty of room for skill (apart from very flukey dice throws, I can always be beaten by a really good backgammon player), but the introduction of a random factor makes it more life-like, with more of a sense...

Is 5x3 the same as 3x5?

The Internet has gone mildly bonkers over a child in America who was marked down in a test because when asked to work out 5x3 by repeated addition he/she used 5+5+5 instead of 3+3+3+3+3. Those who support the teacher say that 5x3 means 'five lots of 3' where the complainants say that 'times' is commutative (reversible) so the distinction is meaningless as 5x3 and 3x5 are indistinguishable. It's certainly true that not all mathematical operations are commutative. I think we are all comfortable that 5-3 is not the same as 3-5. However. This not true of multiplication (of numbers). And so if there is to be any distinction, it has to be in the use of English to interpret the 'x' sign. Unfortunately, even here there is no logical way of coming up with a definitive answer. I suspect most primary school teachers would expands 'times' as 'lots of' as mentioned above. So we get 5 x 3 as '5 lots of 3'. Unfortunately that only wor...

Search This Blog