I heard this sentence from a friend of mine, Andrea Granelli, in a nice talk he gave last Saturday urging for a broader view when looking at Smart Cities and the application of technology in general.

A technology marvel brought some problems along with it… (Credits: www.cagle.com)

Indeed, it is easy to provide examples of new problems created by technology, by evolution,… by anything, religion included!
If you search on Google for the sentence “problems created by technology” you get 256 million pointers (I didn’t check them all…), a clear sign that it is a topic that has attracted attention (and now after this post there will be 256 million plus one).

However, this is not just an issue for modern technology and innovation, and this is why I liked so much the comment of Andrea. It is a general issue related to whatever is shifting the present environment (and mindset)  to a new one, possibly solving present problems but inevitably creating new ones.

Hence, it is naive to present technology, and solutions, as silver bullet solving once and for all any issue. You might solve present ones but because of that you open the door to new ones. The point, therefore, is not to fight resistance to evolution by declaring that whatever comes in is better but to be ready to face what the new will bring along. This kind of attitude rather than hampering innovation may actually ease its adoption by taking on board those that are pointing out its downsides.

Several times in these posts I end up mentioning how scientists are learning more and more by observing how Nature solved problems through millions of years of evolution and how this knowledge is then leveraged for creating new technical solutions.

And this one is another instance of that. When developing a lens optical engineers have to fight distortion and depth of field,

This chip has been called “bugs view”! Now that means something!

that is a limited amount of a scene can be on focus. However, it has been noted that insects do not face this kind of problem. A fly needs a “macro lens” to focus at just 3 mm from its head and it also needs to focus a few meters away to see any incoming danger. This would be impossible for a lens, and for our eye as well!

How could the fly overcome optical limitations that are rooted in physics? Well, by using other parts of physics!

Flies, as most insects, have composite eyes, that is they have hundreds of eyes each one with a very small aperture providing huge depth of field, everything is in focus. They make up for the lower aperture (less light) by having many eyes. Moreover, the eyes are geometrically disposed on a curved surface that avoid distortion.

Now scientists have created a sensor mimicking the insect composite eye to produce a camera with incredible depth of field and no distortion. As you can see in the photo the sensors are placed on a curved surface made possible by flexible connections among the various elements.

First application of this sensor is expected in endoscopic instruments where the depth of field is crucial.

The miniaturisation of digital cameras (now they can fit a 4mm cube) and the decrease in power consumption along with huge storage capabilities are transforming the world. The transformation is taking place under our eyes but it is so subtle that we probably don’t realise what is happening.

Glasses embedding a digital camera by PivotHead

Digital cameras are ending up in glasses frame, in cars bumper, in bicycles and so on. One of these camera can film for seven days on a single charge compressing the video to fit the storage. When required, or on an autonomous decision, it can move into HD filming. Some of them need to be connected to download the clips and images, other can connect in real time via BlueTooth.

So far they have been adopted by people doing sports, wishing to record the activities, by professionals for their job. As an example policemen in Austin, Texas, are now using glasses with an embedded camera to record their daily work. Realtors are using these glasses as they observe a real estate. Those clips will be used by a program to create a 360° view of the estate for showing it to prospective clients.

Researchers at the MIT Media Lab are working on the exploitation of this life feeds to create an augmented memory prosthetics that can help people remember.

Now, just take a step forward and imagine a world where these cameras can be connected in real time to the web. Imagine thousands of people moving around with these cameras picking up the daily life of a community, of a city…

Clearly, one might immediately think of the Big Brother, of being constantly monitored. But one can also think of the opportunity in service creation. Think about the huge amount of data becoming available to the single person and to the community.

Today when you walk around there are thousands of people “seeing” you, and of course we take this for granted, actually we do not even notice that. Obviously it is quite a different thing to be “recorded” and potentially been seen by people that were not at the same place you were. We clearly need some new framework. However, already today this is happening. You are picked up by hundreds of security cameras every single day, and probably you are on thousands of photos taken by someone just as you were wandering around. Now applications can go on Flickr, FB and other places amassing billions of pictures and find you!

It is just going to get worse, in a way. But it is also going to get better, since the awareness of what is going on will be more widespread, and services will become available to leverage those … Big Data. Among those services we might probably want to have one that would preserve our privacy, a sort of invisibility cloak. Problem is .. no-one today has an idea how to ensure such invisibility…

Our life is controlled by how our brain, and nervous system in general, react to stimuli. The more complex the nervous system the more variability in responses but the relation stimuli – action is clear.

Scientists are trying to understand more these relations to learn more about the way the nervous system processes information. In some studies the scientists have manipulated the genome to see how this can lead to different wiring in the nervous system and to different reaction to certain stimuli like a flash of light.

Now, in an article published on Nature Chemical Biology, a team of researchers working in several US hospitals is describing a molecule, optovin, that binds to pain receptors and is activated by a flash of light. When activated it generates a pain signal that in turns creates a reaction in the animal, in this case a fish.

As you can see in the photos on the left a zebrafish reacts to a flash of light by starting to swim faster. The zebrafish has been given the molecule optovin and when a flash occurs the fish feels some pain that it associates to the place it is in so the response is to quickly swim away from what it perceives a painful situation.

Now, I do not like that we induce pain into a poor fish, but even more it frightens me the possibility that this kind of understanding of our “processing and reaction” system opens up in controlling people. We are even further beyond than the Big Brother!

On the other hand, if I think about our world, I have to admit that a good portion of our life is subtly conditioned by many things, form advertisements to the news and the way news are delivered and many more artificial stimuli we are subjected every single day. And I see that disciplines such as neuro-marketing are leveraging on the growing understanding we are gaining on how our brain processes information and reacts to it to grab our attention and steer our behaviour.

What is also frightening is that most of this is happening below our perception thresholds. In this digital connected world we are losing the grasp we use to have on stimuli and the risk of becoming (happy) puppets controlled by the system is more and more real.

Scientist have learnt to use light in many different ways to create, transport and read an information. Your television is an example of a device able to convert an electrical signal into light to reach your eyes as an image. And your digital camera is an example of a device able to capture light and convert it into an information (a flow of electrons that are used to code the information in bits).

To achieve high speed in coding information into light and capturing light to decode the information requires lasers and special components that are used at the end point of optical fibre in telecommunications, thus allowing the transport of thousands of billions of information per second (on the big pipes). On the other hand, the conversion made by a television is made by LCDs and LEDs technology that can work at a snail pace, if compared to the ones achieved in optical fibre: few hundreds cycles per second (that is billions of times slower than what is used in telecommunications fibre). The same applies to the digital camera sensors where we can reach some thousands information capture per second (a professional digital camera can have shutter time as low as 1/8000 of a second, still some billion times slower than fibre).

Schematics of the Antenna on a Chip

Now a group of researchers at the Rice University have managed to create a chip that can be manufactured using the CMOS technology, hence with present state of the art fabs, and that can work at a 10GHz rate, the same used in telecommunications on fibre.

This enables the construction of new devices with exceptional performances at low cost.

The technical name for the chip is “micron scale Spatial Light Modulator”, or SLM for short, and it can be seen both as a receiving or emitting antenna (as all antennas are).

One application they are considering is developing a digital camera having a 1 pixel sensor! Today’s digital camera have millions of pixels but since those are receiving information at a maximum rate of a few thousands per second, having one pixel sensor that can receive information at a rate of tens of billions per second would provide and equivalent “quality”. Of course you would need a system able to move continuously that pixel to intercept rays coming from different points, so that the image can be constructed. It will require completely different cameras, that would be so much smaller and so much cheaper! Indeed it would revolutionize our idea of digital photography, with a camera that can capture absolutely everything in perfect focus, without focussing!

And the photographer in you that would immediately start to objects since defocusing (bokek) is an important part of the composition of a photo will have to acknowledge that a bit of post processing can easily create all the defocussing one may wish to have (it is already a standard feature of CS6).

Let’s get ready for yet another shift in our (falsely) consolidated world.

Most cell phones today have a digital camera and a growing number of digital cameras start to have a connection of same sort (usually WiFi). What about connecting a cell phone with a digital camera?

Well, there have been a number of apps letting you to use your cell phone as a trigger for remote click. We can expect to have the possibility of seeing the image on our cell phone in a sort of “Live View” mode (if you happen to be a Nikonian you know what I mean…).

Now I see that Trigger is proposing an App, on Android and IOs, that capitalise on both the cell phone and the camera to enhance your photographic capabilities.

The figure on the left gives you an idea of what can be done.

You connect your cell phone to your camera (using a dongle) and you can control it leveraging on some of your cell phone capabilities.

In the case shown you take advantage of the GPS in your phone to direct the camera to take a picture every 10 m. The App on the cell phone will measure the distance with the GPS and will trigger the camera to shot.

It is just a tiny example (there are more with this specific application) but it shows that the cell phone and a digital camera can operate as a new system cooperating with one another.

I feel this is going to be a path of evolution in the coming years. Objects will talk to one another more and more and the overall ambient will grow in capability.

Another interesting twist resulting from this evolution is that some objects may “de-facto” take up the role of “orchestrator” and possibly “virtualizator”.
My cell phone may take up the role of my personal interface. As I start to control my camera through it, I might get used to have that interface and as I will change the camera once a new sensor becomes available I will continue to use my cell phone. More photographic features will be controlled through the phone and will be “provided” by the phone.  In the end this might change the whole value chain.

Take a look at the video showing Trigger in action:

[vimeo 40453214]

AT&T has announced the sale of the Samsung Galaxy Camera with a connection data plan. Users will be able to send photos using AT&T HSDPA fast data network to share photos at the same time they “click”!

The camera is also equipped with LTE, 4G, wireless and it is very likely that also this network will become part of the AT&T offer.

With this camera you can also browse the internet but you cannot make phone calls (at least for the time being…).

Now, isn’t this interesting? Today we have most cell phones equipped with a digital camera but the quality of the photos is not as good as that you get out of a proper digital camera.  And we have Memory card with embedded WiFi to let you send the picture you just took with your digital camera to Facebook or any where you like. Clearly this latter solves the problem of having a good quality camera and being able to immediately send your photo but we do not have WiFi connectivity everywhere, whilst we have a much more ubiquitous 3G connectivity.

So this explains this new product! Still, I am curious to see what kind of success it will have, What is most interesting to me is that it is something different from a phone, and yet it is being marketed by a phone company. Whilst I am sure that in the future we will have plenty of objects having an embedded connectivity and using such a connectivity as a way to deliver part of their functionality I see this move of AT&T as a way to affirm that connectivity is not embedded but explicit. You want to send photos, you need connectivity and hence you buy a camera from them.

Would you also buy a doll from AT&T in the future since that doll interact with your kid using connectivity to a server? Or is it something you will buy at a Toy Store with embedded connectivity?

Connected things not necessarily need a Telco network…

Looks like there is quite a difference between the two numbers, to get a better idea would you like 20 euros or 1,000? Hence, it is difficult to understand how a company like Ericsson is forecasting 50 billion “things” connected to the Internet by 20120 whilst a company like HP puts the number in the trillions.

I was yesterday at the ETIS annual gathering where I gave a talk on IoT and before me there was a guy from Ericsson who made the 50 billion forecast. Nothing new, actually. It seems to be the magic number Ericsson is pushing on all tables.

Both Ericsson and HP are well known and respected companies, so why such a big discrepancy?

I think I spotted the problem, and it is an old one, going back at least to the 80s. It is rooted in the different perspectives of telecom vs computer industry.

On the one end the world seen by the telecom companies is the one at the termination points of their network, a network that is fully controlled by them and that can be shaped to their liking.

On the other hand the computer industry sees as its world whatever embeds a microchip, be it connected or not. And of course, these days those chips are more and ore connected, even though they may be connected through networks that are not telecommunications network. Well, these latter simply are invisible to telco’s guy, they do not exist. The problem is, of course, that by taking this approach you, as a Telco, are limiting the scope of your market and in a world where volume makes a difference you are undercutting yourself.

So let’s go back to numbers: is it 50 billion or 1 trillion?

Let’s do some simple math on the back of a handkerchief. Today there are 7 billion people on the Earth. By 2020 there will be a bit more. Already today each of us have several sensors in his context.The computer I am typing on has an accelerometer, a light detector, a multitouch, a temperature sensor, my cellphone has accelerometer, a compass, a touch screen, a temperature sensor, same goes for my iPod and iPad. All of them have virtual sensors, that is it is possible to know the location they are, how much memory, how much power is used, what type of interactions are taking place and so on.  Just looking at me, then, I count over 20 sensors. Multiply it by 7 billion and you get 140 billion “things”. Clearly other people may have 0 sensors (although there is close to 1 cell phone per person in the world and that alone would qualify for 35 billion sensors!).  Other people, on the contrary, may have a few more.

And of course I have sensors at home (several for the anti-intrusion system, temperature sensors, weight sensors, light sensors, I counted over 50 at home and it is likely I missed several more…). Then I have sensors in the car, over a hundred of them… And there are sensors in the building I live, on the roads, in the city….

50 billion means 7 sensors for each person on the Earth. Just with the quick calculation above it is easy to see that today we are already well over the 50 billion mark.

One trillion would mean less than 140 sensors pro capita. Not that much if you think about it! And I have not mentioned all the other things that can be connected (or just logically connected) to the Internet, like the ones having a tag. Any suits, t-shirt, towel in a shop is likely to have a tag that is intercepted and identified by the shop security system.

What will happen when all bar codes will become electronics? How many trillions of objects will be related to the Internet?

The crucial point is that you can look at IoT has things that have a SIM card (the Telco approach) or as things that have a mirroring element in the Internet (a number of bits attached to them). The latter is the computer world vision and it is in this latter vision that most of the biz opportunities lie.

When a video camera is connected to the Internet via a WiFi link Telcos add 1 to their IoT, but the computer world adds 1,000 since that camera brings in images of thousands of objects captured by its lenses and identified by image recognition applications.

Eventually both Ericsson and HP are wrong in their forecast, but HP has an head start placing it 20 times ahead of Telcos… And so do SAP, Oracle….

Our cameras have reached the 30+ Mpixel resolution and although pixels are not the silver bullet for better photos they easily capture the imagination of everyone of us, since it is so easy to understand a concept (although an incorrect one) of more pixels better resolution!

Actually, the resolution depends on two main parameters: the optical resolution of the lens and the way the software manipulates the data harvested by individual pixels. Our intuitive idea that a photo is like a mosaic where the individual tiles are the pixels is just wrong! Nevertheless, it is so convincing that we keep believing it. So more pixel is good!

Given a certain surface (the sensor surface) once you exceed a certain number of pixels you no longer increase the potential resolution since you start getting more noise. On the other hand, increasing the sensor size requires an increase in the lens size and makes it not just bulkier and more expensive but also more prone to defects that lead to a decreased resolution.

Insects have found a way to increase resolution by using many more eyes (composite eye). This is the approach being taken today for pushing the resolution beyond present limits, and it takes place on both directions of increasing the number of cameras and increasing the software capabilities.

The gigapixel camera, like an insect eye it clusters many cameras (98)

The increase in pixels count by increasing the number of cameras is presented in an article on Nature, published on June 20th 2012.

It has been written by a team of researchers from the Duke university, from the University of Tucson and from Distant Focus Corporation.

They propose using 98 commercial 10 Mpixel cameras (their sensors) and to combine the generated signals through a software that can create an image in excess of 1 Gpixel. One should note that multiplying 98*10Mpixel does not equal 980Mpixels because the resolution pixels are being generated by the software program and may be lower or greater than the number resulting from a direct multiplication. Actually, the researchers claim in their article that this system can be pushed up to generate a 50 Gpixel image.

This is because the sensor pixels are not translated one to one. A micro area of a sensor contains a pixel with a filter that lets only light wavelength corresponding to our vision of green to reach the sensor. The nearby left pixel on the right will capture wavelengths corresponding to red and the one to the right wavelengths corresponding to blue. But that same pixel is also juxtaposed to four other “green” pixels in the  corners and to two more red and two more blu pixels in the up and down direction. The software analyzing the data can therefore create “resolution” pixels by combining all these 9 pixels and this create a set of sets resulting in 31 different possible patterns for every single sensor’s pixel. Hence you can increase the resolution by 31 times with respect to the number of physical pixels in the sensors and with some more tweaking you can increase it further (up to the claimed 50GPixel). Since all of this happens in software there is no problem to run into problems with the lens resolution limitation.

This is the kind of image resolution resulting from such an approach:

A panorama containing such a resolution to spot birds in the distance

Just yesterday I posted the news of a fabric screen that can be deformed by your hands and fingers resulting in increased interactions with images. Today goes along the same lines with a device created by Anatomage letting medical students to practice anatomy on what looks like a surgery table with all the bell and whistles of augmented reality and digital images.

real life size human body can be seen through

The table has the same size of a surgery table and the screen can reproduce a patient body. With the assistance of a computer the students can interact with the image and move inside the body in the same way the would do using scalpel and retractors.

You can watch the video made by BBC.

The image can be part of a data base so that students can see many cases, or it can be generated from exams taken on a patient. In this case it becomes useful for a surgeon in looking inside the patient and studying the approach to the surgery. You can explore (if you are not faint of heart!) the various applications and images that can be visualized.

This kind of interface, here applied to medicine and it makes sense given the current high cost of the system, will slowly become more and more applicable to a variety of application fields, as its cost will go down. My expectation is that by the end of this decade we will start to see desk top screens in offices and in some school. By the end of the next decade most surfaces will double up as screen.

What I think is really interesting is that through these interfaces we move complexity from atoms to bits and we can perform all sort of manipulations at the bit level, thus dramatically reducing the cost and expanding the possibilities.