Thursday, July 24, 2014

What does it mean 'to be'?



Chatbots are fun little gimmicks.[1] A little while back one chatbot was reported to have passed the Turing test.[2] That has been debunked.[3] Very much so, in fact. Time to debunk the debunkery, from the last source:
"3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses."
Against Turing protocol. Also, Turing test is a test of consciousness, not perception. The chatbot doesn't 'believe' it is a 13-year-old boy from Ukraine, it is told to say that. An AI does not need a backstory, the test does not include memories. It tests a 'normal' conversation, not a flashback episode of a TV show. The point repeats the first point - such a gimmick is not worthy of a Turing test. The results mean nothing, regardless of how much the Turing test has been warped to make it passable by a simple chatbot. True again.

It continues in the same track - Turing test is not a single shot with handpicked judges. The Turing test was designed so that it would be difficult to pass, but easy to understand. Quite simply it meant putting a person (judge) to IM on a computer. After a while he would say whether he was talking to a computer or a person. If people's ability to differentiate between man and machine constantly remains statistically insignificant (you can no longer confidently say a random person can probably make a difference), the test is passed.

To make the test more reliable, it should be a double-blind (the evaluators don't know if the judge is being tested by a human or a computer) and randomized. In fact, it would be best if the people didn't know there was a chance they are talking to a computer (so they'd go for 'normal' conversations instead of trying to trick the computer).
"6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction."
No, it is not. If you can create a chatbot that can convincingly discuss religion and philosophy, art and morality as convincingly as an average human you have indeed created something spectacular. Because to do that you need more than a chatbot, the 'chat' at that point will only be an interface, what goes on in the background is short of creating novels, symphonies, paintings. To fake discussions, you need to simulate emotions. Typically hormonal reactions created by bytes on a board. Manage that and you have revolutionary technology. And the only way to test if you've succeeded in creating the first 'human' computer is that 'needless distraction' commonly known as the Turing test.

Just think of it, if you can create a proper AI with a personality prototype you will be a leap closer to having plastic pals that are fun to be with.[4]

I would not be surprised if in a few months someone declared once again that the Turing test has been passed by a chatbot. And it will be big news once again because news agencies love making huge news out of nothing, regardless of what they report is true or false. Though let's be honest, even some reputable news sources manages to flame this up. But for any readers, take these news with a grain of salt.


No comments:

Post a Comment