Google: The real questions we should be asking

Google, who famously coined the motto “don’t be evil,” has had a tough time recently. First, it was lambasted for introducing charges to its map service for sites attracting more than 25,000 hits per day. Next, one of Google’s affiliates was caught playing dirty in Kenya, scraping third party IP without permission and trying to poach customers by deception. Around the same time, people pointed the finger at Google for vandalising OpenStreetMap data  in what seemed a related incident. A week later, Google announced a deal with the World Bank which appears to be handing Google a pseudo-monopoly on crowd-sourced mapping in developing countries. Then came the new privacy policy where Google is merging the terms and conditions of multiple products into one, which some said amounts to the end of privacy on the web. And finally, last week a French court fined Google €500,000 over anti-competitive tactics, that is, a “general elimination strategy” of smaller competitors.

All this matters, of course, because Google is a dominant player in a large market and most of us make use of its services. Nobody – including Google – will agree with the use of illegal practices but beyond that it becomes much less black and white. So how did you react to the recent events?

1. If you’re a passionate advocate for open data, privacy or fair competition, you will probably have come to the conculsion that Google is evil. That was indeed the predominant stance taken by the tech media and blogosphere.
2. If you’re a Google fan (or employee) you will see a bigger picture of Google trying to provide a better service around its self-proclaimed mission of “organising the world’s information.” So you may have reacted with a mix of feelings including embarrassment, bemusement, or righteousness.
3. If you’re a bystander you may just be thinking – yeah, Google has simply become a big company and needs to face up to the kind of challenges experienced by any major multinational. That seems to have been the stance of the mainstream media, who largely ignored most of it.

All of these attitudes are valid to a point, but none is entirely satisfactory. What I’ve been missing in recent weeks is reporting that looks at the bigger picture, and much of that should be looking at what we, as a society, actually expect from the internet now.

The Google privacy policy and map charges got some people hot under the collar but what’s the real story here – has Google become such a public commodity that people think it now belongs to them rather than the shareholders? What then, you want to nationalise it? Following through on that argument, be careful what you wish for or you could end up with a new form of communism.

A lof of issues of the online market are directly related to common attitudes and practices of national government bodies and regulators. For example, why did the World Bank’s lawyers think it was okay to sign this deal with Google? Were they so naïve that competition concerns didn’t even occur to them, or was it a conscious decision based on hardnosed pragmatism? Are we comfortable with national governments interfering in commercial markets, handing single players (including sometimes their own agencies) a unique competitive advantage? Also see e.g. these pieces on ESRI or Ordnance Survey.

Should regulators not be more concerned about the ever deeper entrenchment of established players? This is not just about Google, Facebook, Amazon, Microsoft or Apple, but also about smaller niche companies who are busy creating pseudo-monopolies in their own sectors. There are solid garden walls emerging everywhere on the web, partly because the established players capitalised on their first mover advantage, and partly because their product is such that it feeds off its own users in a perpetual virtuous circle at the exclusion of others (think e.g. of social networks like Linkedin or proprietary formatted music or data sites that are not interoperable).

The result is that users are suffering from vendor lock-in and that new players are prevented from entering the market. Where’s the interoperability? Where’s the consumer choice and freedom to move your data between suppliers? Try migrating your friends between Twitter, Facebook or Google+, or your cloud data from Amazon to Azure, and you see what I mean.

It is this that we should be worried about, not (just) the behaviour of individual companies. What we need is a considered approach and not cheap headlines. Whatever Google does or doesn’t do is mostly a matter for them, their shareholders and their customers, and good luck to them all. What we need is a level playing field, open to all, and that is not Google’s job alone. This is where political leaders and regulators need to step in.

And they will only do so if we ask the right questions.

Simplicity

Politicians around the world argue about making the tax and welfare system fairer, but the system has grown so complex that even experts no longer understand it, so it cannot be reformed.

A state-of-the-art Air France airliner crashed into the Atlantic Ocean, killing all on board, because the avionics have grown so complex that only a computer can control them, and the junior pilot in charge had no idea how to fly the plane manually.

The iPhone app from the UK Meteorological Office used to be nice and simple, but the latest version has so many data points and buttons and map overlays that it is impossible to tell what the weather is going to be.

Are you going to create something simple today?

 

 

Let’s cut the bull: A simple OpenData benefit calculator

You will no doubt have noticed that, throughout the past couple of years, local and national governments all over the world have been announcing open data initiatives. At the moment this is particularly the case in Europe where, from Portugal to Lithuania, hardly a day goes by without a new press release. The most prominent was the recent publishing of the EU Open Data Strategy.

Releasing open government data is – of course – a good idea for all kinds of reasons. I strongly support these initiatives even though I also realise that not all that glitters is gold (for further reading I greatly recommend this analysis by Shane O’Neill).

But one thing always makes me laugh: the billions and zillions of “economic benefit” that the release of open data will allegedly create. Yes we all know – intuitively – that there are benefits, but we also know that these benefits are difficult to quantify. As we seem to live in a society where we no longer believe our own judgment unless it is backed up by numbers, we then feel the need to reverse-engineer the pre-determined result with expert bullsh*t known as a market study or economic research.

Let’s be honest, we can make life easier for everyone. In that spirit, here is my contribution to the Open Data world:

Simple OpenData Benefit Calculator™

The fundamental logic is this. A quick Google search shows that, in the EU (population 500 million), the benefit of open data is apparently between €40 billion or €68 billion, depending on who you ask. That is, roughly €100 per citizen. In the UK (population 60 million), the benefit is said to be £6 billion: roughly £100 per citizen. In the US (population 330 million), the benefit might be $750 billion: about $200 per citizen.

Now, do you see a pattern emerging here? Of course, this would be far too simple. So here is a proper formula to help you justify your next Open Data release. I guarantee it works every time:

The Prime minister’s age is expressed in years, and the shoes are in European size (note however that if you use American or UK size then your benefit will be greatly improved). The overall benefit is expressed in local currency but for added accuracy or randomness you could also introduce a foreign exchange factor.

Worked example

Let’s take Romania:  The population is 22 million and the Prime Minister, Emil Boc, is 45 years old. His shoe size is probably 42 but I’m just guessing here – this is where the true research would come in. So here we have:

In other words, the open data economic return for Romania will be € 2.36 billion. As you can see this is a really precise number, so it must be true.

Where are you from?

Spending a lot of time in taxis I get asked this question quite often. Unless you still live in the town you were born the answer is not straightforward, especially once you start thinking about it. Is it where you grew up? Where you live now? Where you feel most at home? Where your ancestors lived? Or is it your personal geo-midpoint of the places you’ve lived at, statistically weighted by the time spent at each place?

The Earth as seen from Apollo 17. (Source: NASA)

If you’ve moved around a bit you might go for the lazy option and just say where you were born, followed by some apologetic waffle about having lost your accent. “Yes but my accent comes back after a couple of drinks.” Brits are quite good at this, especially since regional accents have become trendy.

Or you might do something more American and claim that you’re actually half Scottish or Irish, because your great-great uncle’s father-in-law had a third-degree cousin who was married to a Celtic woman.

The French, on the other hand, keep things simpler. They have a binary system to determine your place – in every sense of the word. Either you are French, or you are not. Either you are from Paris, or you are not. It’s simply a case of magnifique or merde, 1 or 0.

Now that we’ve got the stereotypes out of the way, a spoken accent is indeed an important clue to where someone is from. But it’s by no means a reliable indicator, and it can be a social minefield. I could hold an entire pub quiz on getting people to guess what my accent is. I remember one happy occasion where a lady in San Francisco thought I was English (based on the way I said “hello”), but generally people tend to think I’m from 1. South Africa 2. Belgium 3. Holland or 4. Switzerland. (Just for the record, the answers are 1. wrong 2. a bloody insult 3. wrong  4. no thanks.) My accent also used to have a tinge of Australian in it, but that’s long gone – except for the swearing.

Where you’re from – or where other people think you’re from – can have more serious ramifications too. Take the case of Atanas Entchev, a well-known and respected GIS professional who, after having lived in the US for 20 years, suddenly found himself detained in jail over a bureaucratic visa technicality. His son, who has not known a country other than the US, was also detained and threatened with deportation to a country he had never experienced – but apparently that is where he is “from”. Apart from the irony that the Entchevs were detained by people who themselves are immigrants (or descendants of immigrants), it is clear that where you are “from” is not just a very personal question but also requires validation by others. I guess it’s a bit like people asking me how tall I am. Everyone can see I’m six foot seven but still, people somehow feel compelled to comment on it as if it needed official confirmation or mutual agreement. But compared to some basketball players I’m actually quite short – so it also depends on who’s asking.

Ever since humans first migrated from Africa we have never stopped moving around the world. From the Bronze Age to the Romans, from the Vikings to the Americans, many inventions like the wheel or the moon rocket would not have happened if people had not migrated and explored new horizons. Today, immigration is certainly an issue in many countries but gets bad press for all the wrong reasons. We seem to have forgotten that migration is what has shaped humans more than anything else, albeit not always peacefully. Migration also remains vital for today’s economies, as it enables people to cluster around skills rather than ethnicity, producing global centres of excellence. Even if you’re not surrounded by international rocket scientists or artists, people who come from different backgrounds will broaden each other’s minds and foster creativity.

You can also take migration too far, though. I’ve known many people on the global expat circuit whose children ended up with no sense of identity because they had been uprooted too many times. There is probably a happy medium, but migrate we must – at least some of us. Having lived in five countries I am now happily settled in the UK with my family, and can proudly call myself the tallest Luxembourger in Devon. Surfing, walking, whisky – it’s my kind of place. When I’m not there, it is where I’m from.

So rather than ask people where they’re from, ask where they have come from. There is a subtle difference. If the answer is “the supermarket” then so be it. Simple, no?

Why we shouldn’t let engineers design our world

Or, where is the creativity in geo-design?

You would probably agree that power pylons are the ugliest things ever invented by man. And let’s be honest, there’s no need to be politically correct here – no woman on earth could possibly have created such hideous things (you’re welcome!). Pylons have an eerie ability to spoil the landscape like nothing else. Alas, our society cannot run without electricity, and power cables are notoriously costly to bury underground. It seems like we are stuck with pylons.

So imagine my surprise today when, visiting an energy conference in Berlin, I saw… beautiful pylons. How can this be? The beautiful pylons are built using the same raw materials as the ugly ones. They even use the same type of construction. But the result is stunning: the new pylons have totally changed my emotional response to them.

The answer lies in their shape. It’s something to do with design. Or more precisely: not designed by engineers.

As you look at these amazing pictures from Choi + Shine Architects, imagine what this type of creativity could do in the so-called world of geo-design which, in a nutshell, concerns itself with supporting holistic decision-making in urban infrastructure planning. Design goes beyond the shape of things but, most importantly, design is about creativity.

Just like engineers not everyone in the professional geo-community is creative. But we do have some creative types amongst ourselves who are able to design beautiful maps and publications. Can they also help us inject a bit of creativity into a world that is still dominated by function over form?

The Land of Giants: Pylon design for Iceland, by Choi + Shine Architects

Pylons arranged to create a "sense of place", by Choi + Shine Architects

Post revised & updated on 6 January 2012, also incorporating a new link to an excellent intro on geodesign by @spatialsustain, see http://www.vector1media.com/spatialsustain/the-third-geodesign-summit-focuses-on-creating-our-future.html.

From Lego to Play-Doh: I plead guilty at the altar of Big Data

Or: Why the geospatial community needs to shed old habits and learn new skills to take advantage of Big Data.

I just had a realisation. Sometime in the recent past I must have crossed a threshold. I don’t know exactly when it happened, or how. It was a kind of intuition or common sense, taking small steps, making gradual changes, and suddenly I’ve woken up and seen what I have done: I no longer believe in the classical doctrine of old-fashioned GIS and data management.

Looking around my department, staffed entirely by data experts and other tefal heads, it suddenly dawned on me that I was no longer engaged in what I might have called ‘data’ a few years ago. Take our placement student who is doing a project on artificial intelligence. Analysing our real estate data he has shown that if you mention the word ‘radiator’ in a property listing, the house won’t sell. Figure that! Then there is the timeline of the British economy, brilliantly visualised by my in-house guru: 10 years’ worth of housing market data, condensed into dots and bubbles performing a beautiful choreography on a five-dimensional stage. Millions of data points made entirely comprehensible in one clip. Amazing. The list goes on.

I hate to admit it but I’ve joined the latest bandwagon: Big Data. But it’s not just the ‘big’ that is different.

Like many others in the geospatial industry, I had grown up with the notion that the world was there to be abstracted, structured, ordered, and modelled with great accuracy. When I entered the industry in the late 1990s, GIS and relational databases were state of the art. People talked about how spatial data infrastructures would create virtual representations of everything that exists in the world. The digital nirvana was near.

Lego. Picture by Dunechaser (Flickr CC)

When the nirvana finally arrived it didn’t quite look like some people had imagined. Instead of the Legoland which some had expected – a stack of bricks, neatly built from the ground up – it looked more like a pile of Play-Doh balls: amorphous, gooey, messy. Google Earth, for example, was great fun but it didn’t take long for many of us ‘serious’ professionals to dismiss it as eye candy. Not accurate enough. Not the right projection. Bad rubbersheeting. Poor attributes.

Play-Doh. Picture by daniella_caterina (Flickr CC)

Critically, creating spatial data had suddenly become as easy as composing an email, and so KML files began to rain down like a hail storm announcing the arrival of a tornado. People started annotating the maps with random snippets of data without any concern for relevance or quality. Purists who had dedicated their lives to structured order were feeling exasperated: Where are the standards? Where is the metadata? To which most non-initiated people said, meta-what?

The truth is that mapping had simply become a more realistic, a less abstract representation of the world. But hey, we said, we still do the heavy lifting, and we have the degrees to prove it. Who else knows about geodetic datums or spatial intersections? Of course. Not like those lightweights at Google who manage petabytes of data requiring so much energy, they need their own power stations.

The trouble with the old data world is that the only perfect database is an empty one. This is because the world is not perfect, not regular, not linear. It’s a kind of chaos so vast that even its randomness creates patterns  – a bit like those Mandelbrot fractals that were so popular with the first PCs in the 1980s.

If you are a classically-trained spatial data professional like me, don’t let your well-honed perfectionism get in the way of your next ride. Big Data is here now. Take a deep breath and accept that quantity will eventually trump quality. And when the quantities are huge, the insights can be many. In the world of Big Data, your job is not about structuring or managing data. It’s about telling stories.

The futility of mobile productivity

Right now I should be writing some marketing collateral for the international arm of our business. But my ‘office’ is not conducive to being creative.

It’s so hot and humid, my laptop is covered in condensation. I’m dripping with sweat – as are my co-workers, fellow dads cramped on wooden benches, trying to ignore the noises emanating from the intercom – an incomprehensible stream of words echoing round the building at full blast.

Today’s office is a swimming pool. I’m here to watch my son compete. He swims for a minute, waits for an hour, competes, waits, competes… This goes on for the whole day, so in-between races I try to catch up on work. Even though it’s a Sunday, I have a deadline to meet.

Which makes me realise: I cannot remember the last time I could work on a single job uninterrupted until it was finished. Wherever I try to do this – and the obvious place should be the office but it isn’t –  there are noise, movement, people or irrelevant bits of information getting in the way. Is mobile productivity overrated?

Working on a train

Clonk clonk… clonk clonk… tickets please…  ding dong da dah ding dong dah da dong  [pause] hello?… hello? … I’m on the train… what?…. ah yes….sorry the signal is going… clonk clonk clonk..  to those passengers who have just boarded, you’re on the train and it’s going to…  please take all your belongings…. the buffet car sells sandwiches… hello? hello? ding dong da dah…

Working on a plane

Please turn off all electronic devices until the seatbelt sign has been switched off. In what seems like 15 minutes later: As the seatbelt sign has been switched on please turn off all electronic devices.

Working in a waiting lounge

The person who has lost his passport please contact the front desk…  Flight 467 which you are not booked on, or even remotely interested in, is now ready for boarding….  no, I’m talking this loud so everyone in this room can hear how important I am… what? who took this decision?… ding dong da dah..  bing! you have new mail…

Working in the office

Have you got a minute… ring ring…  can you move your car please…  ring ring… can I pick your brains…  bing! new mail… what’s wrong with the air conditioning… dooooooooooooot… fire alarm test completed … ding dong you have new mail… ring ring… yes I’ll be there…  ding! http error…  Outlook reminder:  the next pointless meeting is in 15 minutes…

Working at home

I’m just popping out, can you get the washing out when it’s finished? …  miaouw… food!  hey daddy’s home, what can we play!

Working anywhere with a 3G or Wifi connection

updating…. updating… downloading…  [The will power required not to press any refresh buttons will burn 2000 calories or 80% of your daily energy reserve.]

Is this the new age of productivity?

I’m no religious person but I fully appreciate why many religions advocate *dedicated* times for reflection, work, family and leisure. Whatever happened to resting on the seventh day I don’t know: it’s a Sunday and, while I’m typing on my laptop and my son is competing in the water, my wife and daughter have gone out to do the shopping because there is no time to do it in the week.

Just remind me again, what exactly is it that we were so busy with during the week?

Twitter location tags: wrong, lazy or just plain stupid?

If you use Twitter then you will have noticed that the service encourages people to geocode their tweets, that is, to record their physical location at the moment of tweeting. What particular purpose this may serve is another point altogether, but let’s not get into that. Suffice to say, I sometimes make use of Twitter’s location feature.

So imagine my surprise this week when Twitter declared to my followers that I was in Italy when in fact I was in Ljubjlana, Slovenia, almost 100km away from the Italian border:

How could this be? Here’s what happened.

My iPhone recorded an accurate lat/long for my location, shown here in Google Maps:

Twitter then reverse-geocoded this lat/long location into a geographic area name. Normally this would be a neighbourhood or a city. In my case it chose to name the country. A bit coarse perhaps, but fair enough if  Twitter hasn’t yet built a full geographic gazetteer for the whole world (has it not?). It would be fair to assume, however, that Twitter would at least get the country right. Unfortunately it didn’t:

That’s right, Twitter took the bounding rectangle of Italy and declared everything in it to be Italian territory. Nice one!

Never mind the proud Corsicans, there will be quite a few people in bordering nations who might have something to say about that. Especially considering that the Balkans are a region where, not so long ago, allegiance to the wrong geographic territory could get you killed.

How difficult would it be to use actual country boundaries as the defining polygon, rather than rectangles? Whether lazy or just plain stupid, this geographic error is unforgivable and simply mind-boggling. Besides the location tags worked correctly in the UK and in Paris a few days ago, so why not in Slovenia? This is such a basic geospatial error it is really unbelievable, but I double-checked it and the error is repeatable.

Does anybody know more about this Twitter error and why it is implemented this way?

Free at last: Why I’ve abandoned my loyalty cards (And it’s nothing to do with data privacy)

That’s it. I’ve had enough.

I’ve collected my last free coffee, paid for with hard-earned loyalty points.  And now, it’s time to drop the ballast and lighten my wallet. I’m going to abandon loyalty cards from petrol stations, supermarkets, chemists, bookshops, airlines…  you name it. I’m done with the lot of them.

You might quite reasonably assume that I’m abandoning the opportunity to save money because of data privacy concerns, a commonly highlighted issue with retail data. But no, it’s not that. I’m no celebrity in need of a High Court privacy injunction. If someone really wants to know what type of underwear I buy or what kind of music I like, be my guest. Besides I enjoy teasing cold-callers, or just slamming the phone down (where else are you allowed to do that!).

No, the problem with loyalty cards is that they are not designed to save you money, they’re designed to make you loyal to a brand – and that is a major difference. Retailers do this by offering discounts, free goods, or ‘status’ (whatever that is). But it comes at a price. To get a “free” coffee, you first need to buy 10 overpriced ones. And it stops you from sampling other coffee which may be better and cheaper. Brands are terrified by that prospect, so they play on your guilty conscience and blind you into “saving” money.

It gets worse. Take frequent flyer programmes. Even in the days when I jetted around the world in business class for a global multinational, it took me all of 2 years to get to Gold status. Then, after a number of years, I stopped flying and the status went back down to zero in 6 months. They might call this loyalty but I don’t. Besides, what use is it? Most airline lounges are just an empty room with a fridge and a few armchairs. Some of them are nice (like those in Australasia) but most are like the Loser Café from The Apprentice, only with worse coffee.

Of course you can get other rewards as a loyal customer. Like “free” flights that cost £100 in taxes and hidden surcharges.  Or free goods you never knew you needed, like that electric drill that’s still sitting in my garage unused. Or some other free product that prevents you from buying a better one somewhere else, because you can’t justify having two. Or you might get real money off your purchase – hurrah! But don’t do the maths, you could be in for a shock. As the credit crunch so elegantly demonstrated, spending money to save money is a never a good strategy.

Loyalty cards restrict our freedom of choice through manipulative psychology. The best brands don’t need this, so they don’t bother with loyalty cards. Think of companies like Apple or Waitrose. Yes but they are expensive, you say. True, but since I started buying fruit & veg from Waitrose I have saved money. How come? Because it’s more expensive, you choose it more carefully (i.e. only what you need) and you don’t waste any of it. The end result is that you get better quality, cheaper. Everybody wins.

Of course you might say that by abandoning loyalty cards I will subsidise those people who keep theirs. That might be partially true, but it’s it’s not that simple. Besides, if you follow that argument through then I shouldn’t be paying any taxes either.

I’d rather use my precious bandwidth time on the things that really matter, and not have my choices artificially restricted by phoney loyalty. Admit it, how many times have you chosen an indirect or inconvenient flight route with your ‘loyal’ airline, when another one could have gotten you there better or faster? In the short term it might cost more, but in the long term it is cheaper.  And in cases where I can’t afford my preferred choice in the short term, I’ll just wait until I can. Buying the wrong thing now just because it is affordable is the fabulous concept known as false economy.

It’s time to be free again.

Europe and the unstoppable opendata revolution

Today I spend a very pleasant day in Brussels attending the annual conference of the European PSI Alliance, of which I am a member. In an intimate environment, amongst the towering buildings of the EC, we had a good representation of speakers and delegates from Austria to Spain, from Google to city councils, from GIS officers to competition lawyers. It was a packed day.

Cycling & Skateboarding, the main security threats facing the EC (photo by author)

Here are a few snippets from the day:

• The opendata revolution is clearly underway. In France, it has even entered the run-up to the presidential race in 2012, with Sarkozy and the left-wing candidates likely to come up with storylines such as “my opendata will be better than your opendata”.
• In Europe, the leading opendata governments can be found in Austria, Spain & the Basque country,France, the Netherlands, Denmark, Sweden, the UK and Slovenia. On the day, Austria and Spain won prizes for PSI excellence, and Slovenia was highlighted as having a very efficient Information Officer with a great track record of taking non-compliant authorities to court! (I look forward to visiting Ljubljana next month)
• Data privacy is “the right to be forgotten”.
• The EC directive on the re-use of public sector information (2003) is being revised to amend the charging principles, to reverse the burden of proof onto the authorities rather than the users, to enshrine machine readable formats, and to include the need for independent regulators. The time frame for finalisation and implementation will take a number of years but – many of the rules required already exist under competition law, so you can gain access to the data you need even if the authority doesn’t play ball. There is only a slight snag – the legal fees. For example, it would cost about £1m to take the UK Government to court over an data access rights issue.
• Holland has finally made its base registers a reality. After 15 years of agonising progress every house, every address, every street is now stored, maintained and accessible centrally in one place, for the whole country!
• Many people think that the definition of public task and personal data are slippery slopes that are best avoided, as it is impossible to say where the boundaries lie.  I’m not entirely convinced by this argument but I’m quite happy to leave it to the regulators, whoever they turn out to be, as whoever enters this territory will indeed find themselves in a morass.
• From Google’s presentation, delivered by the ever-present Ed Parsons, I noted down the following data concepts worth looking up in more detail – this is really happening now:
• Ambient information
• The Internet of Things
• Canonical data
• The internet grows by 5 exabytes every 2 days. That is the same amount of data as all of humanity has ever produced in 5000 years, up to the year 2003.
• Ed is also of the opinion that, on the internet, data accessibility trumps data quality every time (citing iTunes vs CDs as an example). Not everybody agreed but I can see both sides of the argument. This is not a cop-out on my part but I think we need both access and quality, depending on the use case. It is impossible to generalise.

I came away feeling that, 10 years from now, governments may well be reduced to being data regulators rather than data creators, except for a few core reference datasets which no-one else could ever hope to produce legitimately or economically. I find it difficult to see how governments will keep up with the Internet of Things, the sensor web, and the speed of new data concepts and technologies. But they are key enablers in creating a whole information ecosystem that will do many of the data jobs for them, and much more.

Someone will probably call this Web 3.0 or whatever. If I was the right person in the right place I would coin a new name for it and make a fortune on the lecture circuit but hey, I’ve got different surfboards to wax. So, goodbye for for now.