The Exascale Projects: technologies and infrastructures for tomorrow
Billion of billions of data are being created every year. That is Exascale. In a talk given at PARC on the Physics of Data, Marissa Mayer, Google VP of Search products and User experience, estimated that 281 EB of data were available on line in 2009 (compare this with the 5 EB that were on line in 2002). http://www.parc.com/event/936/innovation-at-google.html . This number is going to increase faster than the Moore’s law predicted increase of processing power. According to HP CEO, Mark Hurd, more data will be created between now and 2013 than the amount of data created by the humanity since the invention of writing.
The increased ratio is due to several factors: more people connected to the Internet, more people creating content and uploading it to the Web (the average person who uploaded data to the web in 2006, uploaded 15 times as much in 2009). But there are also other important factors: a flood of real time data (web cam, to name but one), the higher resolution of videos thanks to greater processing power and, coming soon, the explosion of sensors and the ever more complex structure of the data they provide.
Regarding sensors Mayer placed cell phones in this breed, noting that they are not just pairing up with people, they have, like people, sight (camera), ears (mike and loudspeaker) and senses (touchscreen and more). In the future they will be equipped with vibration sensors, tilt, rotation, location (many already have), navigation, sound, airflow, light, temperature, bio-detectors, chemical detectors, humidity, pressure.
Once you have this kind of ambient awareness, and you can harvest information from million of cell phones, you can leverage that for retail, defence, traffic, weather, climate, social moods…
Of course one of the question is “Who is You?”, and another one is: “Can we make sense out of this avalanche of data?”.
HP is trying to answer both questions by working of a computing platform that , they say, will have the power to deal in real time with massive, distributed data analyses. Google, on the other hand, sees itself as a provide of Exascale data services.
You may want to take a look at the Exascale organization roadmap, http://www.exascale.org/mediawiki/images/4/42/IESP-roadmap-1.0.pdf . They met in Oxford, UK in the middle of April to discuss those two questions.
The game for the management of these Exascale Informatino world is just beginning and clearly Google is well positioned to play a main role, and so is HP (look at the Census project on sensors), but so are, in my view, some Telecom Operators. In order to be significant players, however, they have to help in shaping the complex regulatory scenario, provide an open field for innovation to third parties and understand that their assets are not twisted pairs but information.