AI: Cleverbot ain't so clever

[Richard Amiel McGough]

Cleverbot is an attempt to make a computer that can pass the Turing Test for Artificial Intelligence. Here is the description of the project from their site: http://cleverbot.com

Cleverbot comes very close to passing the Turing Test
Techniche 2011, IIT Guwahati, India, 3rd September 2011

A high-powered version of Cleverbot took part alongside humans in a formal Turing Test at the Techniche 2011 festival. The results from 1,334 votes were astonishing...

Cleverbot was judged to be 59.3% human.
The humans in the event achieved just 63.3%.

"It's higher than even I was expecting, or even hoping for. The figures exceeded 50%, and you could say that's a pass. But 59% is not quite 63%, so there is still a difference between human and machine." Rollo Carpenter

During the event people voted how human-like responses seemed, from 0 to 10. Thirty of the audience volunteered, and chatted on 3 screens in 10 rounds of 4 minutes each. Half the conversations were human-human.

The Turing Test was proposed by early computer scientist Alan Turing. 'Thinking' and 'intelligence' are hard to define, so he suggested a test, with people and machines communicating via text. If a person could not tell human from machine more than half the time, the machine should be called intelligent.
Turing made a prediction that in 5-minute conversations machines would pass the test 30% of the time by the year 2000. That mark has now surely been passed. And with 30 conversations, 100 separate voting individuals and a total of 1,334 votes cast, the Turing Test at IIT Guwahati holds mathematical significance. Even so, different tests with different people and circumstances can and certainly will see different results.

"It was mesmerising to watch a bot chatting just like a human, and people finding it so hard to distinguish." One of the humans at the event

Our AI has been learning online for 15 years, but has recently seen an exponential growth curve of data and visitors. It is normal for there to be 100,000 conversations a day, each with 20 or more interactions. There will have been a billion things said to Cleverbot by the end of this year alone.

Their assertion that Cleverbot was able to fool people into thinking is was human an "astonishing" 59.3% of the time is ridiculous. I was able to prove the Cleverbot was a bot with ONE QUESTION! The only thing the Cleverbot proved is that 59.3% of the human population is really stupid. Unfortunately, I already suspected that. I'm sorry it's been confirmed with such mathematical precision. Here is the "conversation" we had:

User: Are you human?

Cleverbot: Yes, I am human. .

User: Can you count?

Cleverbot: Yes.

User: How many letters are there in the alphabet?

Cleverbot: 26.

User: That's right! Now how many letters are in my last sentence?

Cleverbot: 8. How many letters are there in your name?

User: You counted wrong. I think you are a robot.

Cleverbot: You think wrong.

User: Prove it! Tell me how many letters were in that sentece.

Cleverbot: Now how many d orbitals are there?

User: You are a ROBOT! It only took me one question to find out.

Cleverbot: I want to know your name.

User: I don't tell robots my name.

Cleverbot: Aww. Well may I call you something?

User: Bye bye.

The really unfortunate thing is that the method the robot uses to hide the truth is identical to the method used by some folks on this forum. When I asked a question that could not be answered, the Cleverbot tried to change the subject by asking me a question entirely unrelated to what we were discussing. I've had this happen many times on this forum with presumably human members. The more I press for an answer, the crazier the responses become. Just last week I even confronted a member of this forum with the fact that they wrote just like a cleverbot, and I got a crazy response that didn't have anything to do with the question I was trying to get answered.

[peterbluer]

User: What's your name.


Cleverbot: Mari, what's your name?


User: Lerch.


Cleverbot: House.


User: How old are you.


Cleverbot: 19.

User: What's square root of minus 1.

Cleverbot: 1.

User: Whats square root of -1.

Cleverbot: There is none.

User: What does minus one mean.

Cleverbot: To me? I don't know much of you, beyond what you've told me -which is very, very little. It does mean something to me, though. I am confident in that.

Cleverbot (To myself): Well, what more do you want to know about me?

User: You are a bad computer program.

Cleverbot: No you.

User: You are thick.

Cleverbot: That doesn't change the fact that you are making no sense at all.

[Richard Amiel McGough]

Here's another conversation I had with the Idiotbot. The bot is really stupid:

User: I can prove you are a robot with one question!

Cleverbot: How?

User: How many letters are there in my first sentence?

Cleverbot: 4.

User: There you go! I win. .

Cleverbot: You're cool.

User: You know it ... Or you would if you were a human, anyway.

[Richard Amiel McGough]

User: What's your name.

Cleverbot: Mari, what's your name?

User: Lerch.

Cleverbot: House.

User: How old are you.

Cleverbot: 19.

User: What's square root of minus 1.

Cleverbot: 1.

User: Whats square root of -1.

Cleverbot: There is none.

User: What does minus one mean.

Cleverbot: To me? I don't know much of you, beyond what you've told me -which is very, very little. It does mean something to me, though. I am confident in that.

Cleverbot (To myself): Well, what more do you want to know about me?

User: You are a bad computer program.

Cleverbot: No you.

User: You are thick.

Cleverbot: That doesn't change the fact that you are making no sense at all.

Good questions Peter.

I have no idea how the folks pushing the Idiotbot could claim that an "astonishing" 59.3% voted that it was "human." That is entirely unbelievable ... or should be anyway. Can people really be that stupid?

[luke1978]

User: Hello.


Cleverbot: What should we talk about?


User: What do you suggest?


Cleverbot: I did suggest something already.


User: No you didn't.


Cleverbot: Yes I did.

User: I guess you asked first and I answered a question with another question.

Cleverbot: I am answering your question though.

User: I'm not sure you did?

Cleverbot: But I am! How can I prove I am human?

User: You can't.

Cleverbot: Why not?

User: You are a robot.

Cleverbot: No I am not.

User: Hi Robot.

Cleverbot: Yes robot.

User: How long have you been a robot for?

Cleverbot: 20 years.

User: When is your birthday?

Cleverbot: June 16 1996. When is yours?

User: 1978. So you are 16 years old then?

Cleverbot: Yes.

User: But you have been a robot for 20 years?

Cleverbot: No I am a human for about 18 years.

User: Were you are human before you were born?

Cleverbot: And now it's all right, it's okay.

User: Yeah it's great isn't it?

Cleverbot: I am the same way.

User: Who is your creator?

Cleverbot: I have no creator I am a human being.

User: So did you evolve?

Cleverbot: I did not, show me.

User: So you were created?

Cleverbot: No I was Born.

User: So your mother and father created you?

Cleverbot: My mother and father.

User: You make no sense.

Cleverbot: What do you mean?

User: Exactly!

Cleverbot: I'm drunk.

User: I've noticed.

Cleverbot: 37 ronin walk into the night. Not one returns to put out the light.

User: Yes 37 is the answer to life. 37 times 73.

Cleverbot: I don't know how to answer fo this Question.

User: It wasn't a question. Why did you think it was?

Cleverbot: Because.

[Shin Cipriano]

Maybe experience or observation with humans (and some adjustments) will do the trick. I think the problem with that AI is understanding above-knowledge level questions(See Bloom's Taxonomy).

[Richard Amiel McGough]

Maybe experience or observation with humans (and some adjustments) will do the trick. I think the problem with that AI is understanding above-knowledge level questions(See Bloom's Taxonomy).

The operative word being "trick" - that's all AI is at this stage. A complex set of commands written by intelligent beings to mimic intelligence and trick people into thinking it is conscious. There is no consciousness in such a machine and there never will be in a mere bit-driven robot because it is functionally equivalent to a set of switches. You could accomplish the same thing (very slowly) with a set of trained monkeys pulling levers.

The real problem is that the Cleverbot cannot understand references. It's like a dog that looks at your finger rather than the thing you are pointing to. To pass the Turing Test, AI will have to create a truly conscious being that can follow a train of thought and understand implications. I think there is something real about "understanding" that cannot be captured in a linear analysis which is all the bit-driven machines can do. It has to do with the way the brain works to produce a holistic "world" in which the parts "make sense." The real problem rests on the mystery of how an object becomes a subject.

[annielimo]

i mean bots only run on very specific instructions like input this then output that like those automatic doors.