You really need to know where your towel is

Often engineering problems are harder to solve than you think they are, but occasionally they turn out to be easier than expected. That was true for autonomous driving in the DARPA grand and urban challenges, and it also seems to be true for towel folding.



Towel folding is difficult because it involves perception and manipulation of non-rigid objects which can have an unlimited variety of textures. Before seeing the above video I would have assumed that this task was impossible to carry out reliably using current technology and unspecialized grippers. But it looks like it's easier than intuition would suggest. "Easy" is a relative term though, and there's still plenty of complicated stuff going on here.

The process is described in this paper, and uses stereo vision and image segmentation to locate the corners of the towel. This isn't a completely general solution, since the green background will make segmentation easy and localization is being done using a motion capture system - which I think is the lights at the back of the robot - but they've managed to solve enough of the problem that it looks as if tasks like this might be amenable to automation in the near term.

A modified corner handling strategy might also be used to handle T-shirts or other kinds of clothing, and if each item were to contain an RFID this would make its identification easy for the robot so that an appropriate grasping and folding strategy could be selected. Also you could have the robot place the final folded items into a box which can then be carried to a storage location.

Variable lighting

This is an interesting talk about the Greek Atikythera mechanism, but it's also interesting because the surface shape recovery method being used is in principle quite similar to what I was trying to do in one of my previous jobs - before the recession brought that particular endeavor to an abrupt halt.



The problem I was facing was trying to detect datamatrix markings made on metal surfaces, and to take various geometric measurements from them to verify their quality. Some surfaces were very difficult to get a good image from, due to scratches, stray indentations, corrosion, distortions or just having been poorly marked in the first place, so the imager was also fitted with banks of LEDs to control the illumination and these were all programatically adjusted.

My initial strategy was just to take a lot of images under different illuminations and pick the best one, or to try running the detection and verification algorithms on every image. This worked in the majority of cases, but there were still some situations under which the codes remained unreadable, so I knew that the ultimate solution would be to create a composite image from multiple illuminations and exposures - which is essentially the same approach shown in the above video. The video demonstrates quite well how it's possible to pick out features more clearly from the synthetic image where surface normals have been calculated, which would have been hard to distinguish from a single conventional photograph or isolated photos under different lighting.

It's also apparent from the video that the illumination around the object should be from an even distance, using a stand which is approximately a hemisphere. The LEDs which I was using were mounted inside a housing shaped like a cube, so that probably wouldn't have helped. If I were tackling the same problem again I'd probably try making the imager illumination housing into a hemisphere so that the whole device resembles an old-fashioned sci-fi ray gun.

Also another possibility, suggested by the black snooker ball, is that perhaps I might be able to obtain light source directions using the omnidirectional stereo vision on GROK2. It wouldn't be as simple as looking at a black object, but might be something worth trying.

Obscured by Klaus

Trouble in toy town

Not that I'm especially excited by the internal machinations of SIAI, but I recently received an email from David Hart, who as far as I'm aware either is, or was, the project manager for the OpenCog AGI system. It sounds as if he got fired, demoted or otherwise excommunicated from the inner sanctum of SIAI after seeking funding for OpenCog from various big spenders - presumably going above his superiors (whoever they may be).

[email comments removed by request]

For a number of years I laboured under what transpired to be a false belief that SIAI was about promoting AI research and trying to foster the sort of "blue sky" AGI projects which probably wouldn't be possible in academia due to their scope and likely duration (hence the "for AI" part of the acronym).

I think I completely misjudged them, and my current understanding is that it's more of a theological institution with a particular agenda which doesn't have much to do with AI or making any serious attempt to advance the sum of knowledge in this area. Instead their focus seems to be upon an inflexible kind of AGI design of a very theoretical kind which, at least in my opinion, is not a helpful research direction. It looks like they've chosen to focus their attention upon issues which are sufficiently open ended that they can be indefinitely pontificated over, whilst ignoring more pertinent AI issues which are likely to genuinely touch people's lives in the coming decades. As I remember Steve Grand saying in one of the Biota podcasts, the whole singularity movement and its titanic marketing machine is pretty much a public relations disaster for anyone trying to do real AI research, simply adding to the already substantially confused way in which the field is popularly portrayed.

My advice to David would be not to waste any more intellectual effort on SIAI and instead just concentrate his skills on trying to push the envelope and keep OpenCog going, if that's his primary concern. I'm certainly in favour of applying open source methodology to AI and robotics problems, and the future holds potential for some exciting developments. On the other hand, theological debates and egotistical power politics are not helpful, and it's always worth remembering that those who claim to be the high priests of rationalist utilitarianism may actually be false prophets trying to lure you away from the critical path.

A good direction for OpenCog to go in would be to try to link up with what Willow Garage have been doing, especially in terms of reasoning about maps and objects in the robot's field of view, or maybe reasoning about possible intentions of people in the environment. If they could acquire one of the PR2 robots and put it online in a test area for project developers to use that might result in some progress being made.

Fuel crisis 2.0

Since being made redundant in the summer of 2009 the amount of driving which I've been doing has been extremely minimal, so I havn't really been paying attention to fuel prices, but it appears from the data that we're now in the midst of a new fuel crisis comparable to the one back in mid 2008.

Acutely high fuel prices have a negative effect on the entire economy, since we're highly dependent upon oil for many things. Having to spend more money on fuel squeezes profit margins in an economic situation in which profits are already reduced or going negative, and has a particularly inflationary effect upon food prices.

The only good thing about extreme fuel price volatility, where prices oscillate wildly from one year to the next, is that it makes alternatives to using petroleum look much more attractive. I hope that at some point in the next few years I'll be able to buy a vehicle which doesn't run on petrol. At the moment it's looking like electric vehicles are going to be the next big thing, at least for commuting purposes (probably not HGVs or agricultural vehicles yet), and the cost of running on electricity is going to be far more predictable, with other fringe benefits such as less pollution and lower maintenance.

RB5X: Successor to the 914?

I recent stumbled upon this video on YouTube.



This demo is completely unimpressive, and really shows why mobile robotics didn't become popular in the 1980s. Amazingly though, this robot is still being manufactured by a company called RB Robotics, and appears to have a small user group. The price range for this robot also seems reasonable - about the same as a high end PC with all the bells and whistles - which is similar to the original Whitebox concept. Unfortunately unlike the 914 it's not controlled by a PC, so this probably isn't something which you could buy and immediately begin writing software for using contemporary software engineering skills and tools in a manner comparable to the home computers of the 1980s. The level of usefulness of its existing EPROM or microcontroller based electronics is almost certainly indicated by its degree of market success (i.e. very minimal. I'm fairly confident that I havn't heard anything much or at all about this robot previously).



The $450 cost of the base unit seems quite reasonable, so using a Beagleboard, phidgets, battery, wireless adapter and a webcam it probably would be possible to build quite a capable robot for under $1000. Not everyone has the time or inclination to build something like this though, despite it only involving the assembly of pre-made parts. What's still needed is a generic off the shelf PC robot platform, which is in a similar price range (maybe $1000 - 2000) so that it's affordable to anyone who might take a passing interest, so that existing software skills can be leveraged and also so that the open source methodology can apply.

Robots as good citizens

A New Scientist article lists six ways in which robots might be made to do no harm.

Keep them in low-risk situations

Make sure that all computers and robots never have to make a decision where the consequences can not be predicted in advance.

Likelihood of success: Extremely low.

I'd agree with this. Even in what you might think is a very constrained environment it's difficult to foresee all the possible outcomes. However, this doesn't rule out the possibility that a robot could make some rough estimate of the risk level of its current activities. Moving up and down stairs, or carrying objects, is probably more risky than moving on flat ground carrying no objects. A risk estimate for the current activity could be used to adjust acceleration and speed limits, which can be implemented at a very low level and so can be subject to a certification process. Also in riskier situations the robot might take longer to process information - like doing a deeper search in a chess game.

When it comes to risk it's really a question of cost benefit. A certain amount of risk is acceptable, provided that failures are infrequent and contained and that the benefits are greater than the risks. Similarly we accept a certain amount of risk when driving road vehicles or using kitchen utensils.

Do not give them weapons

Likelihood of success: Too late. Semi-autonomous robotic weapons systems, including cruise missiles and Predator drones, already exist. A few machine-gun-toting robots were sent to Iraq and photographed on a battlefield, though apparently were not deployed.

It's not too late to legislate on the use of autonomous weaponry, and I expect in time that this will occur. For example, I think that robots which kill indiscriminately (i.e. target any animate object with an infrared signature) within a given geographical zone will be internationally outlawed. There is a long history of treaties banning or limiting the use of certain types of weapons, so adding an autonomous weapons category would not be an unprecedented step.

Give them rules like Asimov's 'Three Laws of Robotics'

Likelihood of success: Moderate. Isaac Asimov's famous rules are arranged hierarchically: most importantly robots should not harm humans or through inaction allow harm to them, of secondary importance is that they obey humans, while robotic self-preservation is the lowest priority.

I'd put the likelihood here as low. Asimov's laws are so abstract that they're unimplementable, even by a human. Self protection is something which would be possible, but being able to recognise and foresee harm to others by action or inaction is extremely difficult, which at minimum involves solving the other minds problem. Being able to recognise harm to the whole of humanity is so abstract that even most people wouldn't be able to do this in most situations. For example, does drinking a cup of tea cause harm to individuals or to the whole of humanity? If you drill down far enough perhaps it does, but this is really a debate for philosophers - not practical machinery which needs to take expedient decisions in the here and now.

Program robots with principles

Building robots motivated to create the "greatest good for the greatest number", or to "treat others as you would wish to be treated" would be safer than laying down simplistic rules.

I'd also rate this as low likelihood. It will be possible to build machines which follow behavior protocols, but is a protocol the same as a principle? Where you can write down the principle as a definite procedure then yes it would be possible to have a robot with principles. But principles like "the greatest good" are not a definite procedure. Just like Asimov's zeroth law this is so abstract that even humans - the non plus ultra as far as contemporary robotics is concerned - can't decide what it really means in practice. If it were possible to unambiguously determine the greatest good, then all dictatorships would be benevolent ones and there would be no need for democracies or much of a political process.

Educate robots like children

Machines that learn as they "grow up" could develop sensitivity to the actions that people consider to be right and wrong.

Likelihood of success: Promising,

I'd agree that this is promising, although this doesn't mean that every robot would need to be a child machine. There is a great deal of stuff that we understand before we learn language, which is arrived at through play and social interaction. In order to have much understanding of humans or human languages a robot may need to undergo some embodied experience in a social context. There are systems today which do natural language processing or translation, but this is not the same as understanding language, and it seems clear that much of the descriptive quality of words comes from non-linguistic forms of experience in the world as an embodied being.

Make machines master emotion

Human-like faculties such as empathy, emotions, and the capacity to read non-verbal social cues should give robots much greater ability to interact with humans. Work has already started on equipping domestic robots with such faculties.

My intuition on this is that after enough research has been done it will be possible to have machines which understand human emotions to some extent, and here I mean something more than just detecting the expressions on people's faces or speech prosody. However, as people like Hans Moravec have pointed out, it may not be advantageous to embed the full range of human emotions into a machine and instead be fairly selective. The likelihood of success here I think is good, because human emotions are a signal which is explicitly intended to be detected by conspecifics, but that this is a fairly long term research project.

Omnidirectional vision on GROK2

The second version of the omnidirectional vision system fitted to the GROK2 robot. This uses two mirrors stacked vertically rather than on the same plane.

Intelligent CCTV or sloppy BBC reporting?

General purpose computer vision remains a difficult and still to a large extent unsolved problem, and the problem tends to be compounded by unrealistic or uncritical reporting of what contemporary computer vision techniques can deliver in terms of practical applications. A similar effect also applies to robotics generally, which can cause unrealistic expectations to be raised.

This BBC report about "intelligent CCTV" is to my way of thinking an absolutely textbook example of bad reporting practice. It would of course be unreasonable to expect news reporters to be experts on computer vision or AI, but in this case it seems that the reporter is relying purely upon marketing material from a single university department promoting particular CCTV research, and has not consulted more widely with industry experts to determine whether the system is really doing what's claimed. In an environment where the research budgets of computer science departments are being squeezed there may be pressure to grab some headlines in order to maintain funding levels.

Just to highlight how bad the reporting here is, firstly you can see the our man drop his bag close to the car.



Then he walks away from the vehicle. Notice the position of the bag, pointed to with the red arrow.



And then behold the intelligent CCTV identifies a threat!



But hold on just a moment. The bag has moved to the right. This highlights a limitation of the system which is important, but which the reporter doesn't bother to mention. If the bag is sufficiently close to an object of a similar reflectance (the car) it may well be classified as belonging to the background and ignored along with other moving shadows. Clearly the reporter needed to go back and reposition the bag so that it has enough contrast against the background not to be classified as a shadow or part of one of the parked vehicles. But he doesn't mention this at all. Probably the university research department fobbed him off with some notion that the product is still in development or that some parameters still need to be tweaked, etc. But this isn't just a trivial detail, it's a fundamental limitation of visual perception given the constraints of camera positioning, resolution, the changing position of the sun and clouds overhead. Images contain a high degree of ambiguity, which is what makes interpreting them using software so challenging.

A potential solution to this problem would be to have multiple cameras with overlapping fields of view, which would allow the position of the bag to be more reliably identified, but this isn't what's being reported on here.

But even if you overlook the amazing teleporting bag incident there is a more general failure of reporting. Can systems like these really identify individuals similar to the 7/7 bombers? Can they even reliably identify a single individual across multiple views? How can you infer someone's intent from their movements? Wouldn't a real terrorist try to make their movements and mannerisms as inconspicuous as possible? Won't most unconventional motion just be due to accidents or children running around, and if so will there be issues with too many false positives? The more you think about the details the more you come to realize how hard these problems are. They're probably not insurmountable though. Given enough cameras, enough resolution and enough computing power a volumetric model could be produced which would allow the system to know exactly where all objects in the scene are. But of course there is a difference between merely being able to locate objects and being able to locate terrorists.

Also systems like this are not new. In 2005 I wrote what is probably a more capable system than is being depicted in this report, which was also able to count individuals, look for loiterers and so on.

Where's my jetpack?

This isn't exactly a jetpack - more like a ducted fan pack.



In this context ducted fans make sense, because they maximise safety for both the pilot and the surroundings and allow conventional engine technology to be used. This form factor isn't very practical though. Probably the car of the future is more like a small weather-proof booth in which the occupant can comfortably sit, with between two and four ducted fans for propulsion. Whether the vehicle is purely ground based like today's, or flying, probably to maximise efficiency people might not own their vehicles but simply subscribe to a personal transport service, rather like a taxi service but which has high availability and reliability. You would enter your desired destination on a mobile device and then a minute or two later your car would show up to take you there, automatically debiting some amount of money from your account and no doubt also displaying advertising and providing other commercial services during the trip.

Lunar astronomy

In an archive BBC radio interview from 1961 an astronomer named James Ring makes the case for putting telescopes on the moon, and also mentions possible space telescopes appearing long before we know that Hubble arrived. He successfully predicts that humans would land on the moon before the end of the decade, but then also predicts lunar astronomical bases a few years subsequent to that achievement.



So why didn't we get lunar telescopes in the 1970s? Even with the rudimentary computer and communications technology available back then, lunar bases and lunar observatories would have been technically possible.

I guess it all comes down to a cost benefit analysis. Given that astronomy isn't a major money making industry, nor is it especially prestigious from a nationalist perspective, the amount of money which it would have taken to build and maintain facilities on the moon could be judged to outweigh the possible returns. Also on the moon there is the contamination issue from dust. Although there is no wind on the moon, any human or robotic activity such as driving vehicles around would kick up dust which could possibly contaminate the mirrors of telescopes.

Probably for the foreseeable future space based telescopes are a better option, since the control issues mentioned in the interview, although they may have been tricky by the standards of 1961, turned out not to be insurmountable. However, if commercial rocketry can lower the cost barrier then unmanned lunar observatories like the one in the concept art above might not be out of the question.

Stacked mirrors

An example of omnidirectional stereo vision using two mirrors stacked vertically. A hole is cut into the lower mirror so that both can be observed by a single camera simultaneously.



A later test where I've reinforced the struts to make the structure more rigid.

Alternative approaches to omnidirectional stereo vision

Apart from the particular approach which I'm using, with a single camera looking at multiple mirrors on the same plane there are also other possible ways in which to obtain stereo images from omnidirectional vision.

The main alternative seems to be to stack mirrors vertically, and use two separate cameras, like this.



There is also another variant of this design implemented by Chuanjiang Luo, which uses only a single camera, where a hole is cut within the lower mirror so that the upper mirror can also be observed through it.

The main practical issue with this sort of arrangement is that the tall stack of mirrors might suffer from vibration issues if the robot is moving quickly or over uneven terrain. Also cutting a hole in the centre of a mirror might be quite tricky to do well without machining equipment, and limits the field of view.

When pseudoscientists conspire...

Once again Brian Dunning does battle with the pseudoscientific, the conspiratorial and the just plain wacky. His shows make a good antidote to the numerous conspiracy theories which regularly circulate on the internet. If I were a science teacher I'd make Dunning's Here Be Dragons video compulsory viewing.

And if you think that most of the bogus medicines, such as homeopathy, are essentially just harmless indulgences, check out this video.



Unfortunately most snake oil salesmen are not as easy to detect as this, and have a far more subtle pitch which sounds somewhat plausible to the non-expert. It's always a good idea to be skeptical of any new ideas, especially if they claim to predict impending doom, miraculous cures or to usher in a new age of techno-utopia.

The cost of conspiracies

To me internet conspiracies are mostly just fun, and I view them in the same spirit as reading a piece of popular fiction. It's amusing to entertain ideas about possible alternative explanations for events or alternative histories. For those who take these things uncritically though there may also be a real cost to belief in some of the more outlandish internet conspiracy theories. For people with borderline psychological problems, perhaps exaggerated by the economic stresses brought on by the recession, or merely vulnerable individuals with below average IQs, conspiracy theories may be the final straw tipping them into forms of behavior which are maladaptive or actually damaging to themselves or others. An example would be conspiracies surrounding the safety of vaccinations, which encourage some people to make very distorted estimations of risk and either not be vaccinated themselves or refuse to have their children vaccinated against common life-threatening illnesses.

Is photography now becoming a crime?

This is quite scary, because over the years I've also taken plenty of photos in public places, often using larger and more conspicuous equipment like tripods, exotic lenses and gigapans. It was taking photos of architecture in York which first got me interested in the problems of stereo vision and 3D reconstruction. Like the man in the report I'm doing it just for fun as a hobby, and am not officially employed as a photographer. So there's probably a very real chance that I might one day be faced with the same situation - I'd be an easy and obvious target.

Preventing people from taking photos and harassing photographers and journalists is one of the classic things which occurs in societies which are becoming more closed and authoritarian.

So the question is, in this sort of situation what's the best thing to do? I imagine that complete compliance would be the best option - as if you're at a border crossing or airport - even though it's not strictly required by law. It might also be an idea to carry some official identifying document such as a passport if you're going to be using conspicuous camera equipment. No doubt this will get you put onto some sort of database, and what happens then is very uncertain because those secret databases can be used to prevent you flying or might harm your job opportunities and so on - essentially all the things which once happened in Eastern European countries during the Soviet era. The opacity of the databases means that you really have no idea how the data might be used.

Robots in the community

The Willow Garage PR2 robot is one avenue by which the issue of repeatability in robotics research is being addressed, helping to counter Minsky's "little robots" criticism. About five years ago I had hoped that the Whitebox 914 would fulfill this sort of common development platform role, but that attempt, though backed by a lot of marketing effort, turned out not to be a success. The PR2 robot is probably fantastically expensive to produce, and its sensors alone must cost at least $5000, but hopefully some good results and data sets will be obtained from it.

Dire communications and the perils of scientific obscurism

I watched most of the parliament inquiry into the leaked climate science emails, and what the scientists had to say for themselves seemed mostly reasonable. One aspect which seemed not so reasonable was the lack of willingness of the Global Warming Policy Foundation to disclose the identity of its donors. Unless it can be ascertained who the donors are it's entirely possible that this group could simply be a front end to companies who have a substantial financial interest in continuing to pollute the environment. That people might endure slight embarrassment amongst family and friends as a result of such disclosure seemed like an extremely feeble argument to me.

Much emphasis was put on the fact that the data and Met Office code is now in the public domain for independent analysis, but what wasn't stressed was that this new spirit of scientific glasnost only emerged as a result of public pressure arising from the CRU email scandal. Had the scandal not occurred I suspect that the Met office data and code would have remained largely secret, or have only been licensed to a few select organizations. It was also reassuring to hear from Phil Jones, the scientist at the epicentre of the climatology blogwars, that a good alternative data source is the Global Historical Climatology Network. I based my own temperature graphing experiments on that data set, so at least I used something which is considered to be relevant by the scientists involved.

These inquisitorial theatrics underline a more general point which seems to have been a problem for a long time, and that's that a lot of published science just isn't independently verifiable. I first came across this phenomena when trying to repeat experiments in computer vision and robotics, and sometimes obtain further information about particular papers. Within my area of interest the situation has improved a great deal over the previous decade, and it's now far more commonplace for code and data to be published. In my opinion robotics is only just beginning to become a proper science, with the introduction of efforts such as Rawseeds, which aims to make at least some types of experiments repeatable, and be able to evaluate and quantify the relative merits of different approaches.