AGI can be achieved within ten years, according to Itamar Arel.
Well this all sounds like very high level flim-flam, and the guy seems to be making some very confident assertions, such as "will possess much of what we associate with consciousness". I've heard statements like these innumerable times on AI related forums. So he can talk the talk, but what does his walk look like? Reading a few of his papers this looks like fairly conventional reinforcement learning stuff. I'm not seeing anthing here that really seems to justify that a breakthrough has taken place, or is about to take place within a few years worth of engineering refinement combined with some appropriate level of funding. Nor am I especially persuaded that just taking some classical neural net and RL algorithms and hardware accelerating them is necessarily going to produce an AGI.
Overconfidence is a classic trap for AI researchers. If you're going to make big claims that AGI - that is, something with human-rivaling general intelligence skills - is going to happen within a few years you really need to be prepared to give a detailed breakdown of why you think that's likely to happen. If not, then it's just yet more hand-waving and vague references to Moore's law, and rest assured that folks will resurrect old forum postings or videos a few years down the line which make you look silly if the grand prophesies don't pan out.
Reinforcement learning would seem to be only part of what animals and humans do. If you look at the capabilities of humans and also some birds, novel tasks can be accomplished without trial and error in a one-shot operation. It might be argued that what's going on in these cases is an internalised version of reinforcement learning performed within a simulated environment, but at present whether this really occurs seems to be unknown.
Wednesday, September 16, 2009
Subscribe to:
Post Comments (Atom)
6 comments:
Re: "novel tasks can be accomplished without trial and error in a
one-shot operation"
Generalised instances of previously-learned tasks?
Animals have multiple reward channels (rather than just one in a RL system) - but it is hard to avoid the conclusion that reinforcement at a low level is an enormous factor in driving non-instinctive behaviour.
At the moment it seems unclear whether behavior which seems to show intuition is a generalised version of previous behavior learned through trial and error.
And indeed, far from the simple mathematical models which traditionally feature in the RL literature biological systems have multiple value systems (or "critics" to use the RL terminology) mediated by multiple neurotransmitters.
It also appears that value systems may be non-heirachical, or only have a shallow heirachy. Hence high level goals (aka "critics") in humans are sometimes able to override more fundamental survival related value systems, and goals can to some extent be arbitrarily created as a product of culture.
Good stuff, as usual.
and goals can to some extent be arbitrarily created as a product of culture.
Yes. I would go as far as to say that cultural semiotics co-evolved with the human brain, in a manner similar to that which produced lichens (which as you may know are an algae and a bacterium living symbiotically).
It's certainly true for humans that general intelligence wouldn't be possible without socialization. It makes me wonder if AGI is possible without some kind of social component.
Re: "biological systems have multiple value systems"
Yes. I've had this out with several RL guys. Their responses vary, from:
"yes, but multiple values is mathematically equivalent to one value channel with a supplementary sensory data stream"
...to...
"agents can only take one action at once (with a suitable enumeration of action states)" - so, they must be able to prioritise their actions, and therefore they can be treated as having a scalar value system."
(Not actual quotes). This is wrong-headed, IMO. The RL community seems to be in a muddle about the significance of the millions of meta-data-laden reinforcement signals that are present in real animals.
The idea I was presenting is that one-shot novel behaviours are not really very good evidence against models involving ubiquitous reinforcement learning.
From what we can see of brain cells, reinforcement learning is pretty-much the key technology at a low level. Most of the updating in the brain is of the form of making things stronger, or else breaking them down and trying something else.
Axon growth might be a bit of an exception. There, things are complicated, but neurons seem to act as though they want to get in touch with other like-minded neurons.
I think this is because RL has its origins in mathematics - particularly game theory. Introducing multiple critics, amorphous or dynamically generated critics, or (horrors!) networks of interacting critics would make the problems less amenable to mathematically concise solutions.
But I'm not trying to slate RL research here. I think it does have a valuable role to play in understanding the process of behavior modification. The neurological evidence does seem to point to something akin to a RL regime going on, although it may be a more complex multi-dimensional process.
Post a Comment