Loebner

The Loebner prize for 2009.



The mainstream view on this is that it's not regarded as being a very useful test of AI, although it does stick reasonably closely to the one described by Alan Turing in Computing Machinery and Intelligence. Natural language understanding and commonsense reasoning are amongst the hardest unsolved problems in AI, and as the man himself admits in the video even after a couple of decades of such contests not a great deal of progress seems to have been made.

Probably the biggest criticism is that this type of Turing Test, performed over a five minute period, encourages entrants to produce programs which try to fake the superficial appearance of intelligence rather than actually producing something which has some intelligence. So over a short period of time using some clever statistical tricks based upon analysis of real human diologues it may be possible to fool someone that the computer is a human, but even this seems hard to achieve.

I think Ray Kurzweil's prediction is that by 2019 there will be some question mark over whether this sort of Turing Test has been passed or not, with a more decisive pass by 2029. However, making progress on this problem involves more than simply the assumption of the continuation of Moore's law. I expect that the reason why the test remains difficult isn't just because the conversational databases aren't big enough or fast enough, and that there are conceptual breakthroughs which will need to be made. It has been claimed by some that improvements in algorithms are also following Moore's law, but given the incredibly slow progress in some of the algorithms with which I'm familiar I'm not terribly convinced by that claim.