It looks as if Ben Goertzel is coming around to my way of thinking about SIAI, or perhaps he held this opinion all along but didn't say much about it.
"Personally, I'm a lot more worried about nasty humans taking early-stage AGIs and using them for massive destruction, than about speculative risks associated with little-understood events like hard takeoffs."The problem with The Scary Idea is that it's just not elucidated in enough detail to be credible. Certainly once you get into the realm of AI being embodied, either in large or small format, there are all sorts of hard physical constraints and practical issues which the doomsters consistently hand wave over. They also tend to view intelligence as something which can be extended to indefinitely high levels, and not as a form of adaptation to an environment. So in my view The Scary Idea is actually a bogus idea. It's an idea that will lead you astray and waste your mental resources if you provide it with cognitive free reign.
I think all this scaremongering emerged particularly strongly in the years after 2001, when it was compellingly demonstrated that fear could be used to both mobilise public opinion and generate a revenue stream. In the previous decade AI research seemed to have been languishing in one of its winter periods, but in what was commonly referred to as "the post-9/11 era" highly speculative, poorly characterised or astronomically minuscule risks were suddenly treated with a far greater degree of seriousness. If AI could somehow be conflated with fears about public safety and terror plots then perhaps a new spring time of research funding would be forthcoming. I see The Scary Idea as being one of the outgrowths of that zeitgeist.
In general it's probably not a good idea to have your thinking primarily dominated by fear. Fear limits intellectual scope, encourages paternalism, and can result in overly conservative risk-averse decision making strategies along the lines of the precautionary principle which would appear to be at the heart of The Scary Idea.
Don Quixote complex
From an anthropological perspective there are also status implications for The Scary Idea. Engineers, and especially AI researchers, are low status individuals in most societies. Look how much they get paid/noticed compared to other professions or academic fields. As far as I'm aware nobody has ever been awarded a Nobel prize in science for doing AI research. Contrary to this lowly position The Scary Idea posits the AI researcher as a high status heroic character, striving to save humanity from A Terrible Fate.
For some previous opinions of SIAI and Friendly AI see Singularity Research Challenge: funding the wrong stuff and What is friendly?
2 comments:
Re: They also tend to view intelligence as something which can be extended to indefinitely high levels, and not as a form of adaptation to an environment.
Looking at the size of Google's data centres, it seems as though nature still likes large brains even when they are quite big. Large brains are usually part of the environment of large brains - and recently we have seen what appears to be a sexually-selected brain arms-race, where the brains essentially play complicated games with each other - to decide who gets to reproduce.
Taking these things together, it seems reasonable to think that nature will continue to favour large brains a considerable distance past the human level.
It is starting to look as though there may be some "slacking off" in the "serial processing speed" department, though.
Indeed, other brains or more generally other identities are part of the environment within which adaptations are taking place.
Sexual selection probably has played a role in the size of the human brain, although it may not be the only cause. Other factors would include climate change, changing food sources from one month to the next, hunting in groups and tribal rivalry (politics and keeping track of who knows what).
Post a Comment