How not to do a Turing test

The idiot news people (sorry BBC) made a splash recently by claiming that a computer had successfully passed the Turing test. This is an idea modified from an article by the great Alan Turing, who suggested that a good test of artificial intelligence is to communicate with the computer down the wire and if you can't tell whether it's human or computer, then it passes - you have AI. (Turing's original concept is actually significantly more confusing, but this is the version usually given.)

There are several problems with this story. One is that the test as described is far too easy to pass. All that is required is that the machine is 'mistaken for a human more than 30% of the time during a series of five minute keyboard conversations.' That's a pretty low pass rate. I don't think you even get a GCSE for a 30% success. (Turing actually asked 'Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?' - in his version the computer was rather oddly compared with two human, one who always tells the truth and one who doesn't.) Apparently the chatbot 'Eugene Goostman' convinced 33% of the judges that it was human, therefore the organisers claim a success. What we aren't told is how many judges there are - if there were only 3, we are talking about one person being convinced - hardly impressive.

Frankly this test at Reading University was a farce, and it's very sad that some academics enthused over the success the way they did (and that the Princeton web page portrayed above seems to think it was a success). The fact is, you only had to spend the requisite five minutes with Eugene and you would know perfectly well that 'he' is a program. Interestingly, the team behind Goostman have taken the chatbot down, probably due to the hilarity of those trying it out and discovering just how bad it is. (It should be here.) But even the simplest of tactics - asking the bot what a word the bot itself had used meant - showed it up as a failure. It couldn't explain the meaning of a single word it used. A 13-year-old Ukrainian speaking a second language, as Eugene is supposed to be, might not give good dictionary definitions, but could and would have a stab at this. And there are lots of other easy conversational ploys that the chatbot failed on.

If you would like a go at a very early chatbot that was also briefly claimed to have passed the Turing test, take a look at the grandmother of them all, Eliza.

I leave the final word with Dean Burnett, who claimed in a Guardian article that an actual 13-year-old boy had passed the Turing test and was declared human. Now that, surely, is impossible.

Comments

Why I hate opera

If I'm honest, the title of this post is an exaggeration to make a point. I don't really hate opera. There are a couple of operas - notably Monteverdi's Incoranazione di Poppea and Purcell's Dido & Aeneas - that I quite like. But what I do find truly sickening is the reverence with which opera is treated, as if it were some particularly great art form. Nowhere was this more obvious than in ITV's 2010 gut-wrenchingly awful series Pop Star to Opera Star , where the likes of Alan Tichmarsh treated the real opera singers as if they were fragile pieces on Antiques Roadshow, and the music as if it were a gift of the gods. In my opinion - and I know not everyone agrees - opera is: Mediocre music Melodramatic plots Amateurishly hammy acting A forced and unpleasant singing style Ridiculously over-supported by public funds I won't even bother to go into any detail on the plots and the acting - this is just self-evident. But the other aspects need some exp...

Murder by Candlelight - Ed. Cecily Gayford ***

Nothing seems to suit Christmas reading better than either ghost stories or Christmas-set novels. For some this means a fluffy romance in the snow, but for those of us with darker preferences, it's hard to beat a good Christmas murder. An annual event for me over the last few years has been getting the excellent series of classic murderous Christmas short stories pulled together by Cecily Gayford, starting with the 2016 Murder under the Christmas Tree . This featured seasonal output from the likes of Margery Allingham, Arthur Conan Doyle, Ellis Peters and Dorothy L. Sayers, laced with a few more modern authors such as Ian Rankin and Val McDermid, in some shiny Christmassy twisty tales. I actually thought while purchasing this year's addition 'Surely she is going to run out of classic stories soon' - and sadly, to a degree, Gayford has. The first half of Murder by Candlelight is up to the usual standard with some good seasonal tales from the likes of Catherine Aird, Car...

Why backgammon is a better game than chess

I freely admit that chess, for those who enjoy it, is a wonderful game, but I honestly believe that as a game , backgammon is better (and this isn't just because I'm a lot better at playing backgammon than chess). Having relatively recently written a book on game theory, I have given quite a lot of thought to the nature of games, and from that I'd say that chess has two significant weaknesses compared with backgammon. One is the lack of randomness. Because backgammon includes the roll of the dice, it introduces a random factor into the play. Of course, a game that is totally random provides very little enjoyment. Tossing a coin isn't at all entertaining. But the clever thing about backgammon is that the randomness is contributory without dominating - there is still plenty of room for skill (apart from very flukey dice throws, I can always be beaten by a really good backgammon player), but the introduction of a random factor makes it more life-like, with more of a sense...

Search This Blog