In a common setup, a judge exchanges typed messages with two hidden participants, one human and one machine, then identifies which is which. For example, a machine might imitate a person convincingly in a short conversation about hobbies while failing an arithmetic or planning task.
Turing proposed the imitation game as a substitute for debating the meaning of “Can machines think?” His paper begins with a game involving a man and a woman, then considers replacing a participant with a machine. Later Turing-style tests vary in their participants, duration, topics and success thresholds; there is no universal pass score shared by every demonstration.
Those conditions affect what a result establishes. Conversational imitation does not separately test factual accuracy, reliable tool use or safe decisions. A fluent chatbot can invite anthropomorphism, so judge it by its results rather than its manner. For a product, evaluate the work it must perform and the errors that matter to its users.
Sources
- Alan Turing: Computing Machinery and Intelligence — Introduces the imitation game, text-mediated questioning and substitution of a machine for a participant.