Chatbots enable machines to emulate human conversation. While research has been done to examine how human-like communication with chatbots can be, heretofore comparisons of the systems with humans have not accounted for abnormal behavior from the users. For example, the people using the chatbot might be lying or trying to, in turn, imitate a computer’s response. Results of a study comparing transcripts from three chatbots and two humans show that student evaluators were able to correctly identify two computer transcripts, but failed on one. Further, they incorrectly guessed that one of the humans was a chatbot. The study also presents a detailed analysis of the 11 responses from the agents.
Vanjani, Mahesh; Posey, Jamison; and Aiken, Milam
"An Evaluation of Three Online Chatbots,"
Southwestern Business Administration Journal: Vol. 18
, Article 3.
Available at: https://digitalscholarship.tsu.edu/sbaj/vol18/iss1/3