Releasing open government data is – of course – a good idea for all kinds of reasons. I strongly support these initiatives even though I also realise that not all that glitters is gold (for further reading I greatly recommend this analysis by Shane O’Neill).

But one thing always makes me laugh: the billions and zillions of “economic benefit” that the release of open data will allegedly create. Yes we all know – intuitively – that there are benefits, but we also know that these benefits are difficult to quantify. As we seem to live in a society where we no longer believe our own judgment unless it is backed up by numbers, we then feel the need to reverse-engineer the pre-determined result with expert bullsh*t known as a market study or economic research.

Let’s be honest, we can make life easier for everyone. In that spirit, here is my contribution to the Open Data world:

Simple OpenData Benefit Calculator™

The fundamental logic is this. A quick Google search shows that, in the EU (population 500 million), the benefit of open data is apparently between €40 billion or €68 billion, depending on who you ask. That is, roughly €100 per citizen. In the UK (population 60 million), the benefit is said to be £6 billion: roughly £100 per citizen. In the US (population 330 million), the benefit might be $750 billion: about $200 per citizen.

Now, do you see a pattern emerging here? Of course, this would be far too simple. So here is a proper formula to help you justify your next Open Data release. I guarantee it works every time:

The Prime minister’s age is expressed in years, and the shoes are in European size (note however that if you use American or UK size then your benefit will be greatly improved). The overall benefit is expressed in local currency but for added accuracy or randomness you could also introduce a foreign exchange factor.

Worked example

Let’s take Romania:  The population is 22 million and the Prime Minister, Emil Boc, is 45 years old. His shoe size is probably 42 but I’m just guessing here – this is where the true research would come in. So here we have:

In other words, the open data economic return for Romania will be € 2.36 billion. As you can see this is a really precise number, so it must be true.

The Earth as seen from Apollo 17. (Source: NASA)

If you’ve moved around a bit you might go for the lazy option and just say where you were born, followed by some apologetic waffle about having lost your accent. “Yes but my accent comes back after a couple of drinks.” Brits are quite good at this, especially since regional accents have become trendy.

Or you might do something more American and claim that you’re actually half Scottish or Irish, because your great-great uncle’s father-in-law had a third-degree cousin who was married to a Celtic woman.

The French, on the other hand, keep things simpler. They have a binary system to determine your place – in every sense of the word. Either you are French, or you are not. Either you are from Paris, or you are not. It’s simply a case of magnifique or merde, 1 or 0.

Now that we’ve got the stereotypes out of the way, a spoken accent is indeed an important clue to where someone is from. But it’s by no means a reliable indicator, and it can be a social minefield. I could hold an entire pub quiz on getting people to guess what my accent is. I remember one happy occasion where a lady in San Francisco thought I was English (based on the way I said “hello”), but generally people tend to think I’m from 1. South Africa 2. Belgium 3. Holland or 4. Switzerland. (Just for the record, the answers are 1. wrong 2. a bloody insult 3. wrong  4. no thanks.) My accent also used to have a tinge of Australian in it, but that’s long gone – except for the swearing.

Where you’re from – or where other people think you’re from – can have more serious ramifications too. Take the case of Atanas Entchev, a well-known and respected GIS professional who, after having lived in the US for 20 years, suddenly found himself detained in jail over a bureaucratic visa technicality. His son, who has not known a country other than the US, was also detained and threatened with deportation to a country he had never experienced – but apparently that is where he is “from”. Apart from the irony that the Entchevs were detained by people who themselves are immigrants (or descendants of immigrants), it is clear that where you are “from” is not just a very personal question but also requires validation by others. I guess it’s a bit like people asking me how tall I am. Everyone can see I’m six foot seven but still, people somehow feel compelled to comment on it as if it needed official confirmation or mutual agreement. But compared to some basketball players I’m actually quite short – so it also depends on who’s asking.

Ever since humans first migrated from Africa we have never stopped moving around the world. From the Bronze Age to the Romans, from the Vikings to the Americans, many inventions like the wheel or the moon rocket would not have happened if people had not migrated and explored new horizons. Today, immigration is certainly an issue in many countries but gets bad press for all the wrong reasons. We seem to have forgotten that migration is what has shaped humans more than anything else, albeit not always peacefully. Migration also remains vital for today’s economies, as it enables people to cluster around skills rather than ethnicity, producing global centres of excellence. Even if you’re not surrounded by international rocket scientists or artists, people who come from different backgrounds will broaden each other’s minds and foster creativity.

You can also take migration too far, though. I’ve known many people on the global expat circuit whose children ended up with no sense of identity because they had been uprooted too many times. There is probably a happy medium, but migrate we must – at least some of us. Having lived in five countries I am now happily settled in the UK with my family, and can proudly call myself the tallest Luxembourger in Devon. Surfing, walking, whisky – it’s my kind of place. When I’m not there, it is where I’m from.

So rather than ask people where they’re from, ask where they have come from. There is a subtle difference. If the answer is “the supermarket” then so be it. Simple, no?

You would probably agree that power pylons are the ugliest things ever invented by man. And let’s be honest, there’s no need to be politically correct here – no woman on earth could possibly have created such hideous things (you’re welcome!). Pylons have an eerie ability to spoil the landscape like nothing else. Alas, our society cannot run without electricity, and power cables are notoriously costly to bury underground. It seems like we are stuck with pylons.

So imagine my surprise today when, visiting an energy conference in Berlin, I saw… beautiful pylons. How can this be? The beautiful pylons are built using the same raw materials as the ugly ones. They even use the same type of construction. But the result is stunning: the new pylons have totally changed my emotional response to them.

The answer lies in their shape. It’s something to do with design. Or more precisely: not designed by engineers.

As you look at these amazing pictures from Choi + Shine Architects, imagine what this type of creativity could do in the so-called world of geo-design which, in a nutshell, concerns itself with supporting holistic decision-making in urban infrastructure planning. Design goes beyond the shape of things but, most importantly, design is about creativity.

Just like engineers not everyone in the professional geo-community is creative. But we do have some creative types amongst ourselves who are able to design beautiful maps and publications. Can they also help us inject a bit of creativity into a world that is still dominated by function over form?

The Land of Giants: Pylon design for Iceland, by Choi + Shine Architects

Pylons arranged to create a "sense of place", by Choi + Shine Architects

Post revised & updated on 6 January 2012, also incorporating a new link to an excellent intro on geodesign by @spatialsustain, see http://www.vector1media.com/spatialsustain/the-third-geodesign-summit-focuses-on-creating-our-future.html.

I just had a realisation. Sometime in the recent past I must have crossed a threshold. I don’t know exactly when it happened, or how. It was a kind of intuition or common sense, taking small steps, making gradual changes, and suddenly I’ve woken up and seen what I have done: I no longer believe in the classical doctrine of old-fashioned GIS and data management.

Looking around my department, staffed entirely by data experts and other tefal heads, it suddenly dawned on me that I was no longer engaged in what I might have called ‘data’ a few years ago. Take our placement student who is doing a project on artificial intelligence. Analysing our real estate data he has shown that if you mention the word ‘radiator’ in a property listing, the house won’t sell. Figure that! Then there is the timeline of the British economy, brilliantly visualised by my in-house guru: 10 years’ worth of housing market data, condensed into dots and bubbles performing a beautiful choreography on a five-dimensional stage. Millions of data points made entirely comprehensible in one clip. Amazing. The list goes on.

I hate to admit it but I’ve joined the latest bandwagon: Big Data. But it’s not just the ‘big’ that is different.

Like many others in the geospatial industry, I had grown up with the notion that the world was there to be abstracted, structured, ordered, and modelled with great accuracy. When I entered the industry in the late 1990s, GIS and relational databases were state of the art. People talked about how spatial data infrastructures would create virtual representations of everything that exists in the world. The digital nirvana was near.

Lego. Picture by Dunechaser (Flickr CC)

When the nirvana finally arrived it didn’t quite look like some people had imagined. Instead of the Legoland which some had expected – a stack of bricks, neatly built from the ground up – it looked more like a pile of Play-Doh balls: amorphous, gooey, messy. Google Earth, for example, was great fun but it didn’t take long for many of us ‘serious’ professionals to dismiss it as eye candy. Not accurate enough. Not the right projection. Bad rubbersheeting. Poor attributes.

Play-Doh. Picture by daniella_caterina (Flickr CC)

Critically, creating spatial data had suddenly become as easy as composing an email, and so KML files began to rain down like a hail storm announcing the arrival of a tornado. People started annotating the maps with random snippets of data without any concern for relevance or quality. Purists who had dedicated their lives to structured order were feeling exasperated: Where are the standards? Where is the metadata? To which most non-initiated people said, meta-what?

The truth is that mapping had simply become a more realistic, a less abstract representation of the world. But hey, we said, we still do the heavy lifting, and we have the degrees to prove it. Who else knows about geodetic datums or spatial intersections? Of course. Not like those lightweights at Google who manage petabytes of data requiring so much energy, they need their own power stations.

The trouble with the old data world is that the only perfect database is an empty one. This is because the world is not perfect, not regular, not linear. It’s a kind of chaos so vast that even its randomness creates patterns  – a bit like those Mandelbrot fractals that were so popular with the first PCs in the 1980s.

If you are a classically-trained spatial data professional like me, don’t let your well-honed perfectionism get in the way of your next ride. Big Data is here now. Take a deep breath and accept that quantity will eventually trump quality. And when the quantities are huge, the insights can be many. In the world of Big Data, your job is not about structuring or managing data. It’s about telling stories.

It’s so hot and humid, my laptop is covered in condensation. I’m dripping with sweat – as are my co-workers, fellow dads cramped on wooden benches, trying to ignore the noises emanating from the intercom – an incomprehensible stream of words echoing round the building at full blast.

Today’s office is a swimming pool. I’m here to watch my son compete. He swims for a minute, waits for an hour, competes, waits, competes… This goes on for the whole day, so in-between races I try to catch up on work. Even though it’s a Sunday, I have a deadline to meet.

Which makes me realise: I cannot remember the last time I could work on a single job uninterrupted until it was finished. Wherever I try to do this – and the obvious place should be the office but it isn’t –  there are noise, movement, people or irrelevant bits of information getting in the way. Is mobile productivity overrated?

Working on a train

Clonk clonk… clonk clonk… tickets please…  ding dong da dah ding dong dah da dong  [pause] hello?… hello? … I’m on the train… what?…. ah yes….sorry the signal is going… clonk clonk clonk..  to those passengers who have just boarded, you’re on the train and it’s going to…  please take all your belongings…. the buffet car sells sandwiches… hello? hello? ding dong da dah…

Working on a plane

Please turn off all electronic devices until the seatbelt sign has been switched off. In what seems like 15 minutes later: As the seatbelt sign has been switched on please turn off all electronic devices.

Working in a waiting lounge

The person who has lost his passport please contact the front desk…  Flight 467 which you are not booked on, or even remotely interested in, is now ready for boarding….  no, I’m talking this loud so everyone in this room can hear how important I am… what? who took this decision?… ding dong da dah..  bing! you have new mail…

Working in the office

Have you got a minute… ring ring…  can you move your car please…  ring ring… can I pick your brains…  bing! new mail… what’s wrong with the air conditioning… dooooooooooooot… fire alarm test completed … ding dong you have new mail… ring ring… yes I’ll be there…  ding! http error…  Outlook reminder:  the next pointless meeting is in 15 minutes…

Working at home

I’m just popping out, can you get the washing out when it’s finished? …  miaouw… food!  hey daddy’s home, what can we play!

Working anywhere with a 3G or Wifi connection

updating…. updating… downloading…  [The will power required not to press any refresh buttons will burn 2000 calories or 80% of your daily energy reserve.]

Is this the new age of productivity?

I’m no religious person but I fully appreciate why many religions advocate *dedicated* times for reflection, work, family and leisure. Whatever happened to resting on the seventh day I don’t know: it’s a Sunday and, while I’m typing on my laptop and my son is competing in the water, my wife and daughter have gone out to do the shopping because there is no time to do it in the week.

Just remind me again, what exactly is it that we were so busy with during the week?

So imagine my surprise this week when Twitter declared to my followers that I was in Italy when in fact I was in Ljubjlana, Slovenia, almost 100km away from the Italian border:

How could this be? Here’s what happened.

My iPhone recorded an accurate lat/long for my location, shown here in Google Maps:

Twitter then reverse-geocoded this lat/long location into a geographic area name. Normally this would be a neighbourhood or a city. In my case it chose to name the country. A bit coarse perhaps, but fair enough if  Twitter hasn’t yet built a full geographic gazetteer for the whole world (has it not?). It would be fair to assume, however, that Twitter would at least get the country right. Unfortunately it didn’t:

That’s right, Twitter took the bounding rectangle of Italy and declared everything in it to be Italian territory. Nice one!

Never mind the proud Corsicans, there will be quite a few people in bordering nations who might have something to say about that. Especially considering that the Balkans are a region where, not so long ago, allegiance to the wrong geographic territory could get you killed.

How difficult would it be to use actual country boundaries as the defining polygon, rather than rectangles? Whether lazy or just plain stupid, this geographic error is unforgivable and simply mind-boggling. Besides the location tags worked correctly in the UK and in Paris a few days ago, so why not in Slovenia? This is such a basic geospatial error it is really unbelievable, but I double-checked it and the error is repeatable.

Does anybody know more about this Twitter error and why it is implemented this way?

I’ve collected my last free coffee, paid for with hard-earned loyalty points.  And now, it’s time to drop the ballast and lighten my wallet. I’m going to abandon loyalty cards from petrol stations, supermarkets, chemists, bookshops, airlines…  you name it. I’m done with the lot of them.

You might quite reasonably assume that I’m abandoning the opportunity to save money because of data privacy concerns, a commonly highlighted issue with retail data. But no, it’s not that. I’m no celebrity in need of a High Court privacy injunction. If someone really wants to know what type of underwear I buy or what kind of music I like, be my guest. Besides I enjoy teasing cold-callers, or just slamming the phone down (where else are you allowed to do that!).

No, the problem with loyalty cards is that they are not designed to save you money, they’re designed to make you loyal to a brand – and that is a major difference. Retailers do this by offering discounts, free goods, or ‘status’ (whatever that is). But it comes at a price. To get a “free” coffee, you first need to buy 10 overpriced ones. And it stops you from sampling other coffee which may be better and cheaper. Brands are terrified by that prospect, so they play on your guilty conscience and blind you into “saving” money.

It gets worse. Take frequent flyer programmes. Even in the days when I jetted around the world in business class for a global multinational, it took me all of 2 years to get to Gold status. Then, after a number of years, I stopped flying and the status went back down to zero in 6 months. They might call this loyalty but I don’t. Besides, what use is it? Most airline lounges are just an empty room with a fridge and a few armchairs. Some of them are nice (like those in Australasia) but most are like the Loser Café from The Apprentice, only with worse coffee.

Of course you can get other rewards as a loyal customer. Like “free” flights that cost £100 in taxes and hidden surcharges.  Or free goods you never knew you needed, like that electric drill that’s still sitting in my garage unused. Or some other free product that prevents you from buying a better one somewhere else, because you can’t justify having two. Or you might get real money off your purchase – hurrah! But don’t do the maths, you could be in for a shock. As the credit crunch so elegantly demonstrated, spending money to save money is a never a good strategy.

Loyalty cards restrict our freedom of choice through manipulative psychology. The best brands don’t need this, so they don’t bother with loyalty cards. Think of companies like Apple or Waitrose. Yes but they are expensive, you say. True, but since I started buying fruit & veg from Waitrose I have saved money. How come? Because it’s more expensive, you choose it more carefully (i.e. only what you need) and you don’t waste any of it. The end result is that you get better quality, cheaper. Everybody wins.

Of course you might say that by abandoning loyalty cards I will subsidise those people who keep theirs. That might be partially true, but it’s it’s not that simple. Besides, if you follow that argument through then I shouldn’t be paying any taxes either.

I’d rather use my precious bandwidth time on the things that really matter, and not have my choices artificially restricted by phoney loyalty. Admit it, how many times have you chosen an indirect or inconvenient flight route with your ‘loyal’ airline, when another one could have gotten you there better or faster? In the short term it might cost more, but in the long term it is cheaper.  And in cases where I can’t afford my preferred choice in the short term, I’ll just wait until I can. Buying the wrong thing now just because it is affordable is the fabulous concept known as false economy.

It’s time to be free again.

Cycling & Skateboarding, the main security threats facing the EC (photo by author)

Here are a few snippets from the day:

• The opendata revolution is clearly underway. In France, it has even entered the run-up to the presidential race in 2012, with Sarkozy and the left-wing candidates likely to come up with storylines such as “my opendata will be better than your opendata”.
• In Europe, the leading opendata governments can be found in Austria, Spain & the Basque country,France, the Netherlands, Denmark, Sweden, the UK and Slovenia. On the day, Austria and Spain won prizes for PSI excellence, and Slovenia was highlighted as having a very efficient Information Officer with a great track record of taking non-compliant authorities to court! (I look forward to visiting Ljubljana next month)
• Data privacy is “the right to be forgotten”.
• The EC directive on the re-use of public sector information (2003) is being revised to amend the charging principles, to reverse the burden of proof onto the authorities rather than the users, to enshrine machine readable formats, and to include the need for independent regulators. The time frame for finalisation and implementation will take a number of years but – many of the rules required already exist under competition law, so you can gain access to the data you need even if the authority doesn’t play ball. There is only a slight snag – the legal fees. For example, it would cost about £1m to take the UK Government to court over an data access rights issue.
• Holland has finally made its base registers a reality. After 15 years of agonising progress every house, every address, every street is now stored, maintained and accessible centrally in one place, for the whole country!
• Many people think that the definition of public task and personal data are slippery slopes that are best avoided, as it is impossible to say where the boundaries lie.  I’m not entirely convinced by this argument but I’m quite happy to leave it to the regulators, whoever they turn out to be, as whoever enters this territory will indeed find themselves in a morass.
• From Google’s presentation, delivered by the ever-present Ed Parsons, I noted down the following data concepts worth looking up in more detail – this is really happening now:
• Ambient information
• The Internet of Things
• Canonical data
• The internet grows by 5 exabytes every 2 days. That is the same amount of data as all of humanity has ever produced in 5000 years, up to the year 2003.
• Ed is also of the opinion that, on the internet, data accessibility trumps data quality every time (citing iTunes vs CDs as an example). Not everybody agreed but I can see both sides of the argument. This is not a cop-out on my part but I think we need both access and quality, depending on the use case. It is impossible to generalise.

I came away feeling that, 10 years from now, governments may well be reduced to being data regulators rather than data creators, except for a few core reference datasets which no-one else could ever hope to produce legitimately or economically. I find it difficult to see how governments will keep up with the Internet of Things, the sensor web, and the speed of new data concepts and technologies. But they are key enablers in creating a whole information ecosystem that will do many of the data jobs for them, and much more.

Someone will probably call this Web 3.0 or whatever. If I was the right person in the right place I would coin a new name for it and make a fortune on the lecture circuit but hey, I’ve got different surfboards to wax. So, goodbye for for now.

Full resolution image available here on Flickr. You are welcome to re-use this cartoon under the Creative Commons licence (Attribution-ShareAlike) by giving credit to the author, Thierry Gregorius.

(photo from authors own Flickr collection)

I’m currently acting as a part-time data usability adviser on an EU-funded geohazard project. The aim of this 3-year project is to create satellite-derived terrain motion datasets for major urban areas across Europe, and to make the data available in the most accessible, understandable and reusable form. The instigators of this EU project are mostly geology and remote sensing experts and so, being more familiar with data creation than data publishing, they are planning to build some kind of INSPIRE-compliant map portal… uurrgh.

Portals date from prehistoric internet times, when people were still chained to their desktops. So I’ve slowly begun to nudge the project team away from the monolithic portal idea. They are now quite excited about opendata concepts such as mash-ups, linked data, the semantic web, and the need to cater for diverse user needs and platforms with simple standards. But, besides all this, and as part of the EU funding requirement, the project still needs to build a site that displays the data in a pre-cooked way which lay people can visually grasp. So a base mapping platform is needed. Should I recommend Google, Bing or OSM?

Rather than pretend to have all the answers (I don’t – see earlier blogpost… ), I’d love to hear your opinions. Does the choice really matter or is it purely ideological? Here’s some of my thoughts to get the ball rolling:

OpenStreetMap: Probably the best choice politically, if only for the term ‘open’. Coverage looks very good and is nearing completion in many parts of Europe. But it will never be as consistent as that of other suppliers, and once coverage is complete there’s still the issue of updates (once the blank spaces are mapped, will people be motivated enough to remap them?).  Also, with OSM being an open community, there is some uncertainty around governance and future direction – it’s all very organic and could go off in multiple directions. Then there’s an unresolved issue around the Creative Commons licence which restricts re-use for commercial purposes (unless you’re in the business of giving away your intellectual property). There has been talk of a new OpenDB licence but to be honest I’m not sure how advanced this is and whether it will resolve this particular issue.

Google Maps: Love or loathe them, their cloud infrastructure is unbreakable and their market share in mass mapping is by far the biggest. This is also why some citizen mappers (albeit a minority) prefer Google Mapmaker to OSM for data collection: where infrastructure is sparse,  Google offers an out-of-the-box solution with minimum hassle – if you can live with Google’s licence terms and commercial aims. Some people are naturally worried about Google claiming usage rights over your data but in some cases – like an opendata project – this can actually be helpful as it massively broadens your audience. And Google certainly does not claim ownership of your data. The deal is basically that you provide non-exclusive access to your data in return for a free professional service. In some circumstances this is quite a good deal, and does not interfere with your ability to re-use your own data in another form on another platform e.g. as part of a paid product.

Bing Maps.  Every year I marvel at Blaise Aguera’s new innovations but in terms of global implementation, Bing Maps seem behind the curve. They were the first to launch bird’s eye imagery but are only now beginning to catch up with Street View (in collaboration with Navteq), and the lag time to implement Blaise’s magic seems quite long, at least in Europe. To my knowledge Bing’s user terms are very similar to Google’s, so there is little difference in terms of IP or re-use.  However, Microsoft recently invested in OSM, which is now included as a layer in Bing Maps (and OSM founder Steve Coast even joined them as an employee), so this could have a very interesting future.

Keeping in mind that the geohazard ‘portal’ (urrgh!) would only use one of these services as visual background mapping – like wallpaper, really – and that it will not launch until 2-3 years from now (a long time in internet terms!), which service would you recommend? Does it even matter, or is the choice purely ideological?