Global warming: an independent investigation

After reading about the Machiavellian machinations of the leaked Fortran source code and its attendant fudge factors I decided to do some independent investigation to satisfy my own curiosity as to whether global warming is actually occurring or whether it's just some inappropriately convolved lookup table.

Taking raw temperature data from the Global Historical Climatology Network (GHCN-Monthly) data base, we can plot average temperatures taken from ground stations over the previous three centuries. I took all the data from all weather stations in all countries and averaged the temperature values per month. Plotting each month separately you get the following graphs.



Then to get a better idea of what's going on overall I calculated the average global temperature per year, and with minimum amd maximum values.



Assuming that the original data is good, and you can find descriptions of how it was collected on the GHCN site, the results seem pretty clear. Over the last three centuries there has been a steady rise in average global temperatures. However, no recent exponential trend is evident. Instead this looks more or less like a linear increase, with the slope beginning before the industrial revolution gets into high gear in the 19th century. The overall average graph (blue line) in the above image shows this best, and I can hold a ruler up to the screen and get a pretty good fit.

The good, the bad and the inconvenient

So the bad news is that after even a cursory examination of the data we can clearly see that global warming is a real phenomena rather than merely a PR stunt or something else which was invented by Al Gore. However, the good news is that this doesn't appear to be significantly related to human activities - despite our inflated sense of self importance as a species. If the warming were primarily due to industrial activity I would expect to see a significant deviation upwards, approximately half way along the graph as factories multiply and begin to belch out smog, steam engines are built, the internal combustion engine is invented and eventually jumbo jets cruise through the atmosphere.

Where you can see alarming deviations though is if you take a small window along the graph, corresponding to a few decades. From the perspective of an individual human life span this seems like a long time, but in the scheme of things it's really not. On smaller scales random deviations begin to look significant, and you could perhaps persuade yourself that an exponential trend was beginning - rather like seeing faces in clouds or trying to find small features within noisy camera images. Also if you're selective about which data you choose then all kinds of theories are potentially supportable. So if I only look at the average minimum temperatures, or if I only look at the average temperature for November I might be able to persuade myself that some sort of anthropogenic foul play was going on.

So which camp would this put me in? Probably I'm a non-anthropogenic or maybe a slightly-anthropogenic warmer. Humans may have had some effect, but it's just not highly noticeable within the data. From this I'd predict that all the proposed carbon dioxide capture, footprint and trading schemes will have no noticeable impact upon the overall temperature trend. This still means that rising sea levels and changing weather patterns could be a significant problem though, and we should take whatever precautions are considered necessary to mitigate against these risks.

Approximate linearity

A line can be fitted fairly well to the average annual global temperatures. There's more variation in the past, but I expect that this can be explained due to less accurate thermometers and calibration methods. It may also be possible to fit a shallow curve.

The more imaginative you get, the more complicated lines or curves could be fitted to the graph, but usually according to the principle of Occam's razor the simplest explanation tends to be the best one. So with a bit of creativity we could maybe fit a nearly horizontal line to the left hand side, and a steeper slope to the right hand side. Or a shallow parabola or circular curve might be fitted. Or using a sliding windowed average the graph could be turned into a complicated looking series of oscillations.



In the interests of full disclosure, so that you can see that I havn't introduced my own fudge factors, the C# source code used to produce these graphs can be downloaded here. The program loads the data from a file called v2.mean and saves a few CSV files, which can then be visualized using the chart tool within OpenOffice spreadsheet.

Fudge factors

Over on ESR's blog it looks as if there's a storm brewing with regard to the leaked source code from the Climate Research Unit. If the source code in question turns out to be genuine, and the fudge factors can't be properly justified (usually they can't, by definition, since a fudge factor is just some made up number inserted arbitrarily), then this could be pretty damaging, and resignations would surely have to follow.

I'm no expert on climate science, and havn't really taken sufficient interest in it to have a very informed opinion either way. One thing which is very clear however is that for a topic as important as global warming, which has significant public policy implications worldwide, all of the data, and the software used to process it, should be open source so that it can be checked and re-checked from first principles by anyone who has doubts or wants to run their own forecasts. It shouldn't require some crook to steal the software then post it online, because the code should already be in the public domain to begin with.

People power

Extending the e-petitions idea to local government I think is a good idea. Such things help to elucidate the relevant issues of the day, allowing voters to measure the performance of their elected representatives against some explicit checklist of concerns when it comes to election time. But I would be cautious about concluding that this will really "give power to the people".

There has been at least one petition success that I'm aware of, namely that of the Alan Turing apology. Whilst this might have been highly enthralling to nerds like myself, and possibly also to scholars of history and mathematical logic, Turing's apology can hardly be characterised as a major political issue of our time. Most petitions which gain a significant number of votes are, as far as can be meaningfully discerned, simply ignored - save for the eventual transmission of a perfunctory email of acknowledgement after some period of time has elapsed.

If there is no link between petitioning and the activities of politicians or local councilors then further extending the system could result in a kind of faux democracy, where people are erroneously lead to believe that they are participating in some civic process of import, when in reality they are doing nothing of the sort.

Some petitions are clearly silly and can be dismissed out of hand, like the one to make Jeremy Clarkson king/president/god, but is there a good way in which the rest could be used? One possibility might be to pass legislation which means that if a petition gets more than some threshold number of votes, and is not obviously silly, then the relevant politician has to ask a question in parliament related to it. This would at least provide some kind of link between petitioning and what goes on inside parliament - connecting voters to their representatives. Something similar could happen at the local government level too, but in each case the resulting debate would need to be broadcast online so that concerned petitioners can verify that the issue is raised in something more than a purely dismissive manner.

Although electronic voting has been pretty much a disaster, and thankfully has not yet been foisted upon the British public, I think that the prospects for useful convergences between politics and technology are good. Internet based systems - provided that they're sufficiently open - will allow individuals and interest groups to monitor their own streets and towns more effectively than before and identify trends and issues in a much more intelligent and quantifiable way. It will also become increasingly difficult for both politicians and campaigners to get away with quoting bogus statistics, as has frequently been the case in the past.

Evolvable neural nets, robots and OpenCog

I was recently watching a video lecture by Hugo de Garis and Ben Goertzel. The topic was of interest because these two researchers are trying to combine high level logic with lower level robotics and perception.

Unfortunately this doesn't look as interesting as I'd imagined. Like many singularitarians De Garis seems to have a penchant for listing large numbers and significant amounts of money, and he also favours a large scale Manhattan project style AI effort which he refers to as the "China Artificial Brain Administration" (CABA). I wasn't tremendously enamored by the other peripheral stuff, such as the baffling machinations of Beijing politics.

After half an hour or so of flim-flam we finally arrive at the beef. De Garis is actually using a very conventional neural net approach which he calls PARCONE, and this was somewhat disappointing because the the richness of structure and behavior which was possible in his earlier 3D cellular automata based neural nets from the late 1990s/early 2000s appears to have been abandoned altogether. He talks about the issues involved with partially connected networks, but under his earlier approach this would have been a non issue, because the growth patterns of the neurons and the consequent architecture was itself evolved.

The earlier 3D CA approach also had the advantage that each cell within the cube was just implementing a few really dumb state transition rules specified by the genome, which meant that the system was readily implementable as a piece of massively parallel hardware. This approach could also potentially evolve types of structure quite unlike classical neural net simulations, such as exotic dendritic connectivity. So I'm more of a fan of the earlier De Garis rather than the later De Garis.

[As a side note, it would be interesting to go to an even greater extreme, where any attempt to simulate neurons is disregarded and you just evolve some cellular automata which transforms a given input pattern into a given output pattern.]

The decomposition of the problem into separate subsymbolic and symbolic majesteria is probably going to turn out to be a mistake also. In a different lecture Ben Goertzel talks about the oddities of the English language, so for example you might get "in a car" but "on a bus". This strange phraseology, where "in" and "on" are used to refer to more of less the same kind of activity, probably originates from the kinds of physical gestures which are typically performed when engaging with these different types of vehicle, with the relevant motion profiles or expected sensations constituting part of the overall concept.

Drawing a strict line between the low level and high level systems is likely to make any genuine understanding of English language (or any human language for that matter) very difficult to achieve. Cognition doesn't occur exclusively along the linguistic dimension, even though language may be a very good semantic indexer, and one of the challenges for OpenCog is going to be finding ways in which to instantiate visual, auditory or tactile experiences derived from simulated or real embodiment within the system as equal hypergraphical citizens alongside their more abstract offspring. Merely adding a symbolic link to an audio file, an image, or even an evolved neural classifier module is not sufficient, in my opinion.

It now seems abundantly clear from multiple sources that information flow in the brain is not unidirectional, and that high level expectations alter the way in which low level features are perceived. Erecting an artificial barrier between perception and cognition is going to prevent synchronisation between multiple levels of abstraction.

Robots and the law

All of these issues will become relevant over the next few decades, as robotics penetrates further into ordinary life. The safety and liability issues aren't insurmountable though, and after some initial period of experimentation during which there could be accidents (as there have been with robots in factories) eventually protocols and best practices will emerge, likely accompanied by some legal guidelines. There are a few obvious things which could feature as part of a future robotics directive, such as.

• Robots should be fitted with some standardised failure detection electronics. Some specification should categorise failure modes, together with an indication of their severity in terms of risks to users. The failure detection system would monitor things like speed, acceleration, motor stalls, attitude and battery power.
  

• If a serious failure occurs, such as battery failure, the robot should default to a safe posture. This would just be an elaboration on existing industrial robot directives, which say that if emergency stop is activated the system should come to a controlled stop, then power down.

• If the robot is connected to the internet, and I expect that most will be, there should be some standardised protocol used for teleoperation. A standardised interface can be rigorously tested for security flaws, to minimise the risk of unauthorised access.

• A specified amount of data about the robots operations should always be logged in some recognised format. This would act like a black box recorder in the event of an accident, allowing a technician to try to recreate the problem in simulation. Courts may require this data to be provided as evidence in legal cases.
  

• There are already industry recognised quality markings indicating that systems have been certified to a given safety standard. Some comparable marking should exist for robotics. There may need to be different certifications for different kinds of application.

If the robot causes an accident, and it's subsequently discovered that it doesn't comply with something resembling the above guidelines, then you can go ahead and sue the company that manufactured it for negligence. If the robot was in compliance, and the accident wasn't just some random "act of god" which couldn't have been foreseen, then there should be some legal responsibility on the company to identify the fault and take reasonable precautions to prevent it happening on future systems, and perhaps pay compensation if necessary.

Why did Second Life tank?

An article does a kind of post mortem examination of Second Life and what happened to it. Google confirms that fewer people have been searching for Second Life related things in the last year or two, and the number of news items has been decreasing. Did it require too much graphics hardware? Did not enough avatars visit virtual shops? I think the main factors why Second Life has not been as big a success as it could have been are as follows:

1. The user interface sucks

If I'm joining any kind of online community, be it Facebook or Twitter or whatever, the first thing I want to do as a user is find other users with similar interests. With the Second Life user interface this seems extraordinarily difficult to accomplish.

Facebook and other social networking sites get around this by actually suggesting new users that you might want to connect to. This sets up a kind of virtuous, community building cycle where the system becomes increasingly good at guessing who you might be interested in, but in Second Life there is no similar feature. This means that in Second Life meeting people with similar interests is largely a matter of chance. If you don't happen to be in the right place at the right time, chances are that you'll miss connecting to interesting people.

2. Too overtly commercial

I've used other similar online 3D chat programs in the past, going back to the very earliest ones in the mid 1990s when the internet was still a kind of wild west landscape. Compared to all comparable systems which I've used the commercialism within Second Life is very "in your face". That wouldn't be so bad, if it wasn't also a barrier to entry. One of the comments on the article says:

"It's the ultimate sandbox to build whatever you can imagine."

As I, and I suspect many other users, quickly discover is that if you actually do try to build something imaginative in Second Life - say a house for example - it's going to cost you money. Not only that, it will keep costing you money, because you never really own land in Second Life - you just rent it. Again this wouldn't be so bad if the amounts of money involved were trivial, but as far as I could tell they're not. This acts as a strong disincentive for new users to contribute and add creative works as citizens of the virtual environment. The new user is socially excluded by default from participation in the main forms of creativity which are possible within Second Life.

3. Spam

Any whiff of real money to be made attracts spammers, like wasps around a honey jar. Taking a look at the lists of events in Second Life typically produces an unsightly barrage of spammy entries. For people like me who have an allergy to spam, this is definitely a turn off. With the user interface there seems to be no obvious way of controlling, limiting or removing the spam, except for adult related content.

Spam can be a service killer. Likewise, if Twitter can't find ways to control the increasing avalanche of spam there will be negative repercussions further down the line and users will start to abandon the service.

4. You need to be there in person

Unlike email, microblogging, or a Facebook status Second Life requires the user to interact directly in person and in (more or less) real time, via their avatar. It means that people need to arrange meeting times, which can be very inconvenient in a world where communities span the globe. This is a general issue with all forms of instant messaging.

5. New users are often treated with disdain

This typically happens in online games, but also seems to occur in Second Life too. The experience for a new user needs to be more welcoming, rather than simply facing a tirade of abuse and/or ridicule from existing users.


There are of course good aspects to Second Life too, and a great deal of effort has obviously been put into building some of the environments and avatars. There are some delightful experiences, such as story or poetry readings, which would be difficult to create in any other way than via a 3D chat environment. With the addition of an improved user interface, more incentives to participate for new users and social networking features similar to other sites perhaps Second Life could make a comeback.

Floods

When floods occur, like this one, I think it's a good idea to make a note of the extent of the flood zone. These days small hand held GPS units are cheap to buy, and it would be fairly easy to survey the area of flooding within a town or city. This is useful because it can provide a historical record, giving an indication of what to expect in future and how often such events occur.

When severe floods occurred in York in 2000 there was for a time a map of the flood zone on one of the web sites of a local newspaper, but after the news story had passed this information quickly disappeared from view. It would be far better instead if this information was public knowledge, and part of Google Earth or similar online maps.

There is an existing flood map created by the Environment Agency, but this is under "crown copyright" (i.e. the monarch owns the data, even though it was gathered using public money - a stupid idea, in my opinion), and gives no indication of the extent of individual flood events or their frequency over time, which is critical for any sensible evaluation of risk.

Is the Microsoft research site blocking Linux users?

Probably not very many Linux users are interested in the Microsoft research site, but whilst searching around for stereo vision related articles I stumbled across something indicating that Microsoft was producing some kind of stereo vision SDK for use with a stereo webcam. Interested to see if there were any demos/videos of this I tried accessing the site, but couldn't get it to load.

At first I thought that perhaps their web site was down. Microsoft have been laying off staff over the last year or so, and I know all too well that when it comes to the crunch R&D is often the first thing to go. I also tried accessing the main Microsoft research site, but again with absolutely no success - every time I just got "The connection to the server was reset". I also tried accessing the site from a different PC, also running Ubuntu 9.10, but with the same result - so it wasn't just a problem with the configuration of one particular computer. The main Microsoft site www.microsoft.com was visible though as I'd expect.

So I assumed that their research site was temporarily down, but as a last ditch attempt I tried booting up the Windows Vista partition on my laptop - something which happens only on rare occasions these days. On Vista, also using the Firefox 3.5 browser, the Microsoft research site loaded perfectly every time, and I was able to check out the stereo vision SDK.

So as far as I can ascertain it looks as if Microsoft is blocking Linux users from certain sections of its web site - not on a browser basis, but on an operating system basis. Perhaps this is old news, but it's the first time that I've encountered it.

The Microsoft stereo vision SDK

Having viewed the demos and downloaded the SDK, this doesn't look like anything to get particularly excited about. They seem to be concentrating primarily on head tracking and background subtraction/substitution, using something which resembles the Birchfield algorithm.

I suppose that the only thing worth noting is that a company like Microsoft are interested in stereo vision at all, since it seems to have been a quite unfashionable topic in the last decade. However, an interest in this area would make sense from the point of view of the 3D sensors which they're using on the next generation of their games consoles. Perhaps they have been evaluating the performance of passive stereo vision compared to monocular time of flight devices.

The argument from ignorance

An amazing rant about the weakness of eyewitness accounts. There seems to be quite an industry in UFO/ghost/bigfoot videos on YouTube these days. Even before the internet arrived there was never any shortage of sensational UFO stories in the tabloid newspapers, always with statements from police men or pilots or retired generals.



There might be extra-terrestrials out there, but so far there seems to be no convincing evidence that our planet has been visited by them. If we are visited at some time in the future, it might be by a robotic probe or rover rather like the ones we send out to inspect the neighbourhood.

It's also possible that we might be the aliens, in that life might not have originated on Earth, but instead arrived in bacterial form via a passing meteor or other space debris (as per the intro of the game Spore).

Quantitative Easing for Dummies

A fairly succinct explanation of the theory behind "quantitative easing".



The latest news on quantitative easing suggests that Mr Banker still isn't lending money to the little guys and that the UK "easing" is going to increase to £200,000,000,000. In reality the "fed", or in this case the Bank of England isn't really buying anything from the banks since the so-called "toxic assets" are really just bad debts which are unlikely ever to be repaid (inappropriately sold mortgages, for example). There are also additional complexities beyond this nice theoretical model, in that some of the banks have been partly or entirely nationalised - which means that in some cases Mr Banker and "uncle sam" are actually the same entity.

It has been much trumpeted in the media over the last week that the US is apparently out of recession, so maybe the worst of it is over and things will begin returning to business as usual in 2010. By delaying the general election for as long as possible I think that Gordon Brown is gambling on some sort of recovery occurring before May next year, so that he can say something like "we faced tough times, but via my enlightened leadership steered a course through the storm".

Should Microsoft make a Linux distro?

It's a good question, and some people might say that they already have a Linux distro in the form of their deal with Novell. It seems fairly certain that Windows 7 will be successful in the sense that it maintains Microsoft's market dominance for at least a few more years, but it doesn't look as if desktop Linux versions are going to go away either. Sooner or later, whether they like it or not, Microsoft are going to be forced to confront the issue of a growing Linux desktop user base.

So far they've been able to hold off the competition via a combination of FUD campaigns (spreading Fear, Uncertainty and Doubt about rival products), special deals with retailers and their legacy inertia in the market. In the past I've heard people in the tech industry use objections to Linux which can are directly attributable to Ballmer and Gates (often without realising it), but to anyone with even the most tenuous grasp on the labyrinthine machinations of the IT industry it's becoming increasingly evident that the familiar objections no longer stand up to much scrutiny. Also anyone old enough will know that in the software industry inertia doesn't last for long and changes can happen quite rapidly - even to companies which seem so large as to be unassailable.

One strategy which they could follow is to apply their "embrace and extend" philosophy to Linux and issue their own distro - perhaps branding it using their existing icons, and arranging the menus and widgets in a Windows-like style. With Google claiming that they also have some sort of Linux based operating system in the pipeline this would actually make a lot of sense if, as seems to be the case, they believe that Google are their main competitor.

"I can't reconcile requiring programmers to donate their work, leaving them to figure out how to earn a living with some kind of subscription or services model."

This is a very old objection going back to Bill Gates open letter to hobbyists, and was addressed a decade ago by Eric Raymond in The Magic Cauldron: The Manufacturing Delusion (I recently re-read the book version of The Cathedral and the Bazaar, and it has withstood the test of time quite well).

As someone who has earned a living from writing software even if the entire world suddenly transitioned to free software under GPL3 (or later) licenses in a manner that Richard Stallman would be completely happy to endorse I don't think I'd be in any great danger of losing the ability to ply my craft in return for moolah. Certainly in the sorts of industry in which I've worked you just can't rely upon volunteers to come up with the applications needed. Volunteers have little spare time after doing their main job, and typically don't have the industry specific expertise (they may be able to acquire the expertise, but not on the time scales which are expected in commercial situations). Also in accordance with Raymond's theory very little of my wages have been derived from the sale value of the software written. Mostly they're obtained by building relations with customers and being able to support whatever their particular requirements might be. One of Eric Raymond's observations, which I think has a lot of truth in it, is that users implicitly recognise that software should to be treated as a service, because once it's announced that a given piece of software will no longer be supported the amount of sale value which you can obtain from it falls dramatically even if it continues to remain available for purchase. This probably doesn't hold true for games, which are expected to have a very short shelf life, but it does seem to apply to operating systems and other kinds of application which have more durability.

"Moaners"

I find myself once again incredulous at the British honours system. How did Alan Sugar manage to become a Lord? Surely a man who believes that the best economic strategy in a recession is denial is not a particularly good champion of UK business. His computer empire was a household name in the late 1980s, but now seems to be pretty much nowhere.

Reading his Wikipedia entry all becomes clear.

"He is a philanthropist...and donated £200,000 to the British Labour Party in 2001. On 5 June 2009 it was reported that Sir Alan Sugar had been offered a peerage by British Prime Minister Gordon Brown"

This seems like a common design pattern amongst people who receive peerages.

The future of farming

The rest of the series can be found here.

Second Life and social networks

This must be one of Joseph Weizenbaum's last appearances, in which the topic is mainly Second Life and its implications for society.



It hardly needs mentioning that I find Weizenbaum's objections to be unconvincing. There are possible criticisms which could be made of Second Life, and similar online worlds, but they're not discussed much in the video.

The biggest criticism which I can think of involves virtual realestate. Philip Rosedale makes the point that in SL you can express your identity by changing your avatar or building a virtual house, but from my admittedly limited experience of SL building a house involves paying "rent" in one form or another, and the values of the rent when converted into real world money turn out to be non trivial. This immediately excludes a large number of number of users from expressing themselves within the virtual environment, particularly if they come from poorer countries, and tends to erect the same kinds of financial barriers within the virtual world as also exist in the real one.

Another criticism, which was only touched upon, is that these new virtual worlds tend not to be run in a democratic way. I know of no virtual world currently in existence which could be said to be truly democratic. Typically they're controlled by a handful of individuals (so-called "masters of the universe") who largely dictate what is or isn't permissible and in which directions the technology evolves.

However, neither of these criticisms are major ones and I expect that in time virtual worlds will exist which are both fairly democratic in the way they're run and also allow wide participation without any significant financial barriers.