How not to do a Turing test

The idiot news people (sorry BBC) made a splash recently by claiming that a computer had successfully passed the Turing test. This is an idea modified from an article by the great Alan Turing, who suggested that a good test of artificial intelligence is to communicate with the computer down the wire and if you can't tell whether it's human or computer, then it passes - you have AI. (Turing's original concept is actually significantly more confusing, but this is the version usually given.)

There are several problems with this story. One is that the test as described is far too easy to pass. All that is required is that the machine is 'mistaken for a human more than 30% of the time during a series of five minute keyboard conversations.' That's a pretty low pass rate. I don't think you even get a GCSE for a 30% success. (Turing actually asked 'Will the interrogator decide wrongly as often when the game is played like this as he does when the game is played between a man and a woman?' - in his version the computer was rather oddly compared with two human, one who always tells the truth and one who doesn't.) Apparently the chatbot 'Eugene Goostman' convinced 33% of the judges that it was human, therefore the organisers claim a success. What we aren't told is how many judges there are - if there were only 3, we are talking about one person being convinced - hardly impressive.

Frankly this test at Reading University was a farce, and it's very sad that some academics enthused over the success the way they did (and that the Princeton web page portrayed above seems to think it was a success). The fact is, you only had to spend the requisite five minutes with Eugene and you would know perfectly well that 'he' is a program. Interestingly, the team behind Goostman have taken the chatbot down, probably due to the hilarity of those trying it out and discovering just how bad it is. (It should be here.) But even the simplest of tactics - asking the bot what a word the bot itself had used meant - showed it up as a failure. It couldn't explain the meaning of a single word it used. A 13-year-old Ukrainian speaking a second language, as Eugene is supposed to be, might not give good dictionary definitions, but could and would have a stab at this. And there are lots of other easy conversational ploys that the chatbot failed on.

If you would like a go at a very early chatbot that was also briefly claimed to have passed the Turing test, take a look at the grandmother of them all, Eliza.

I leave the final word with Dean Burnett, who claimed in a Guardian article that an actual 13-year-old boy had passed the Turing test and was declared human. Now that, surely, is impossible.

Comments

Why I hate opera

If I'm honest, the title of this post is an exaggeration to make a point. I don't really hate opera. There are a couple of operas - notably Monteverdi's Incoranazione di Poppea and Purcell's Dido & Aeneas - that I quite like. But what I do find truly sickening is the reverence with which opera is treated, as if it were some particularly great art form. Nowhere was this more obvious than in ITV's 2010 gut-wrenchingly awful series Pop Star to Opera Star , where the likes of Alan Tichmarsh treated the real opera singers as if they were fragile pieces on Antiques Roadshow, and the music as if it were a gift of the gods. In my opinion - and I know not everyone agrees - opera is: Mediocre music Melodramatic plots Amateurishly hammy acting A forced and unpleasant singing style Ridiculously over-supported by public funds I won't even bother to go into any detail on the plots and the acting - this is just self-evident. But the other aspects need some exp...

Murder by Candlelight - Ed. Cecily Gayford ***

Nothing seems to suit Christmas reading better than either ghost stories or Christmas-set novels. For some this means a fluffy romance in the snow, but for those of us with darker preferences, it's hard to beat a good Christmas murder. An annual event for me over the last few years has been getting the excellent series of classic murderous Christmas short stories pulled together by Cecily Gayford, starting with the 2016 Murder under the Christmas Tree . This featured seasonal output from the likes of Margery Allingham, Arthur Conan Doyle, Ellis Peters and Dorothy L. Sayers, laced with a few more modern authors such as Ian Rankin and Val McDermid, in some shiny Christmassy twisty tales. I actually thought while purchasing this year's addition 'Surely she is going to run out of classic stories soon' - and sadly, to a degree, Gayford has. The first half of Murder by Candlelight is up to the usual standard with some good seasonal tales from the likes of Catherine Aird, Car...

Is 5x3 the same as 3x5?

The Internet has gone mildly bonkers over a child in America who was marked down in a test because when asked to work out 5x3 by repeated addition he/she used 5+5+5 instead of 3+3+3+3+3. Those who support the teacher say that 5x3 means 'five lots of 3' where the complainants say that 'times' is commutative (reversible) so the distinction is meaningless as 5x3 and 3x5 are indistinguishable. It's certainly true that not all mathematical operations are commutative. I think we are all comfortable that 5-3 is not the same as 3-5. However. This not true of multiplication (of numbers). And so if there is to be any distinction, it has to be in the use of English to interpret the 'x' sign. Unfortunately, even here there is no logical way of coming up with a definitive answer. I suspect most primary school teachers would expands 'times' as 'lots of' as mentioned above. So we get 5 x 3 as '5 lots of 3'. Unfortunately that only wor...

Search This Blog