Several times in these posts I end up mentioning how scientists are learning more and more by observing how Nature solved problems through millions of years of evolution and how this knowledge is then leveraged for creating new technical solutions.

And this one is another instance of that. When developing a lens optical engineers have to fight distortion and depth of field,

This chip has been called “bugs view”! Now that means something!

that is a limited amount of a scene can be on focus. However, it has been noted that insects do not face this kind of problem. A fly needs a “macro lens” to focus at just 3 mm from its head and it also needs to focus a few meters away to see any incoming danger. This would be impossible for a lens, and for our eye as well!

How could the fly overcome optical limitations that are rooted in physics? Well, by using other parts of physics!

Flies, as most insects, have composite eyes, that is they have hundreds of eyes each one with a very small aperture providing huge depth of field, everything is in focus. They make up for the lower aperture (less light) by having many eyes. Moreover, the eyes are geometrically disposed on a curved surface that avoid distortion.

Now scientists have created a sensor mimicking the insect composite eye to produce a camera with incredible depth of field and no distortion. As you can see in the photo the sensors are placed on a curved surface made possible by flexible connections among the various elements.

First application of this sensor is expected in endoscopic instruments where the depth of field is crucial.

4k televisions have now hit the market, at least in some parts of the world and rumours have it that Apple is considering a new line of products – an iTV- based on 4k standard.

What we are missing is content. There is some content in the movie industry filmed in 4k (actually the cinema standard is 4k) but the transmission capacity required to bring that content to your home is scarcely available (it basically requires a fibre to the home connectivity to bring the 100 Mbps required – slightly less, actually, but in that range) and DVD and Blue Ray are out of the question.

Actually we do have sensors able to pick up video with 4k resolution: the ones in our digital cameras. 4k means 8Mpixels resolution and that is well within the range of most consumer cameras today, at a price that is often as low as 100$!

True, most cameras will not be able to support a 25 frame per second filming at an 8Mpixels resolution but we know that the microprocessor industry is just there to produce faster and faster chips.

This Compact Flash support 4k resolution recording

What we are still missing is a flash memory that can store at the hight bit rate required for 4k filming. Actually, that is what we “were” missing, because Toshiba announced at the end of 2012 a new series of Compact Flash able to support 4k resolution recording and yesterday they have started selling them.

According to their press release:

“The EXCERIA PRO™ CF cards integrate Toshiba’s high performance NAND flash memory and specially developed dedicated firmware. They achieve a read speed of 160MB/s and write speed of 150MB/s⁴, the highest level yet reported.

The new cards are compliant with the CompactFlash Association (CFA) standard CompactFlash^® Specification Revision 6.0 and compatible with the UDMA7 high speed interface, ensuring they can support high performance DSLRs to the full. The cards are also compatible with the latest Video Performance Guarantee standard, VPG-20. VPG-20 secures Full HD video capture streams at a minimum write speed of 20MB/s for compatible host devices and recording media. VPG-20 enables high quality Full HD video capture at high frame rates with no dropped frames.“

Hence I feel that a real push to 4k will not come from the Majors but from each of us. By 2015 I expect many televisions will be 4k and many of us will be using them to watch our own clips.

The miniaturisation of digital cameras (now they can fit a 4mm cube) and the decrease in power consumption along with huge storage capabilities are transforming the world. The transformation is taking place under our eyes but it is so subtle that we probably don’t realise what is happening.

Glasses embedding a digital camera by PivotHead

Digital cameras are ending up in glasses frame, in cars bumper, in bicycles and so on. One of these camera can film for seven days on a single charge compressing the video to fit the storage. When required, or on an autonomous decision, it can move into HD filming. Some of them need to be connected to download the clips and images, other can connect in real time via BlueTooth.

So far they have been adopted by people doing sports, wishing to record the activities, by professionals for their job. As an example policemen in Austin, Texas, are now using glasses with an embedded camera to record their daily work. Realtors are using these glasses as they observe a real estate. Those clips will be used by a program to create a 360° view of the estate for showing it to prospective clients.

Researchers at the MIT Media Lab are working on the exploitation of this life feeds to create an augmented memory prosthetics that can help people remember.

Now, just take a step forward and imagine a world where these cameras can be connected in real time to the web. Imagine thousands of people moving around with these cameras picking up the daily life of a community, of a city…

Clearly, one might immediately think of the Big Brother, of being constantly monitored. But one can also think of the opportunity in service creation. Think about the huge amount of data becoming available to the single person and to the community.

Today when you walk around there are thousands of people “seeing” you, and of course we take this for granted, actually we do not even notice that. Obviously it is quite a different thing to be “recorded” and potentially been seen by people that were not at the same place you were. We clearly need some new framework. However, already today this is happening. You are picked up by hundreds of security cameras every single day, and probably you are on thousands of photos taken by someone just as you were wandering around. Now applications can go on Flickr, FB and other places amassing billions of pictures and find you!

It is just going to get worse, in a way. But it is also going to get better, since the awareness of what is going on will be more widespread, and services will become available to leverage those … Big Data. Among those services we might probably want to have one that would preserve our privacy, a sort of invisibility cloak. Problem is .. no-one today has an idea how to ensure such invisibility…

In the last decade scientists forecasted that within ten years they would be able to create a bionic eye, that is to implant in an eye that has lost the capability to “see” a chip that can recreate that function. There are many people who have lost their sight, many after having been hit by retinitis pigmentosa, a degeneration of the retina leading to blindness.

Schematics of a retinal implant

Scientists have followed two approaches to restore sight: implanting electrodes on the visual cortex to simulate signals received by the eyes and implanting a chip on the faulty retina to connect directly to the optic nerve. The latter is by far the better approach, but it is also the most difficult one.

It is therefore a great news to see that the FDA has granted the first approval to a chip for implant in the eye, the Argus II.

The chip is part of a system to provide visual stimuli to the optic nerve: a pair of glasses embed a video camera that sends the images being captured to a tiny computer worn by the person. In the future we can expect to have this computer embedded in the glasses (the critical issue here is the size of the battery).

The computer transforms the images into signals that are brought back to the glasses to be transmitted via an antenna to the chip implanted on the retina. The transmission is also providing the power for the chip to capture the signal and relay it to the optic nerve.

The signals are not exactly the same as the ones that a normal functioning retina would provide to the optic nerve but they are close enough to let the plasticity of the brain to interpret them. After a while the brain rewires itself and accepts these signals as the “usual” visual signals recreating the image on the visual cortex.

The images that are perceived are quite a long shot from the images that a normal retina is able to create. Here the person is able to see light in different shades and so become aware if it is dark or not and can recognise forms well enough to find her way in a room without bumping on a chair or a table. There is no way, today, to recognise a face, nor to read a book.

But this is TODAY! The biggest issues have been resolved and now it is a matter of evolving to increase the resolution and the variety of signals. And we know very well that evolution is an intrinsic property of electronics. I am therefore quite confident that by the end of this decade we will be able to implant a chip that would allow a blind person to read a book and to recognise her friends. And this looks more like magic than technology!

The Sharp 8k television

At CES we have seen presented the first television prototype designed for the mass market in the 8k standard. This is delivering a resolution of 7680*4320 (that is slightly over 33Mpixels). There were also at least one commercial product in the 4k standard, that is 3840*2160, i.e. 8+ Mpixels, presented by the various brand.

The problem, of course, is that we do not have any content for an 8k television (and very very few for 4k television). Sharp had to borrow and experimental clip produced by NHK that is working on 8k since 1995!

However, for a photo enthusiast, we already have very high resolution content! Our digital cameras today have sensors in the range of 10+Mpixels and this is already 5+ times resolution of present HD television (2Mpixels) and in the range of 4k television.

The new digital cameras have passed the 20 Mpixels and some are exceeding the 33 Mpixels of the 8k television (like the Nikon 800).

Hence, I for one, would like to have this kind of screen in my living room (though finding space for an 85″ screen is challenging) providing the price becomes affordable. No price was announced for the prototype but we can expect it to be very high. At the same time we can also expect it to come down over time and by next decade I have little doubt that it price should go below the 1,000 euros, as it is the case today for most HD television.

Interestingly, the consumer electronics is now exceeding the capabilities of the network and is pulling innovation in the network. Delivery of an 8k content push the envelop of bandwidth in the 100Mbps, not unfeasible of course (and already available to a few) but that requires pervasive fibre distribution networks (with possibly radio drops).

Such screens will transform our perception of content. Since their resolution exceed the resolution of the human eyes (and brain) they create an immersive reality where make-believe becomes part of everyday experience.

Human beings (as well as several animal species) have a knack to get a feeling that something is going on by observing how other people are behaving. In particular our brains are very well in tracking the gaze of other people and whenever they detect a convergence of “gazing” they get arouse and focus their attention on that converging point.

This is how in a social gathering, at a party or in a square, at the office or at a metro station, we are continuously scanning other people’s gaze (and if someone is staring at us, making us the convergent point of gazing we rapidly get uneasy…).

Robots have a difficult time in understanding what is going on. For them any single input has basically the same importance and they have to work out what could really be important. This has been the issue that some researchers at the Carnegie Mellon Robotic Institute have tried to solve.

To do that they tried to understand how people are tracking attention in a crowd by placing a camera on each person to track their gaze and computing any convergence of gazing.

This, as shown in the figure below, produces some maps of the social space highlighting those areas where there is a convergence of gazing.

Interestingly, it turns out that people gazes intersect not just when there is something they are looking at but also whenever there is someone they are listening to. This provides the information about who if attracting the ears of people, not just their eyes.

The era of social robots is not too far away. Expect to see them mingle in our social circles starting in the next decade and become commonplace in the following ones. At that point we will start to wonder which is which, and that might be embarrassing…

Scientist have learnt to use light in many different ways to create, transport and read an information. Your television is an example of a device able to convert an electrical signal into light to reach your eyes as an image. And your digital camera is an example of a device able to capture light and convert it into an information (a flow of electrons that are used to code the information in bits).

To achieve high speed in coding information into light and capturing light to decode the information requires lasers and special components that are used at the end point of optical fibre in telecommunications, thus allowing the transport of thousands of billions of information per second (on the big pipes). On the other hand, the conversion made by a television is made by LCDs and LEDs technology that can work at a snail pace, if compared to the ones achieved in optical fibre: few hundreds cycles per second (that is billions of times slower than what is used in telecommunications fibre). The same applies to the digital camera sensors where we can reach some thousands information capture per second (a professional digital camera can have shutter time as low as 1/8000 of a second, still some billion times slower than fibre).

Schematics of the Antenna on a Chip

Now a group of researchers at the Rice University have managed to create a chip that can be manufactured using the CMOS technology, hence with present state of the art fabs, and that can work at a 10GHz rate, the same used in telecommunications on fibre.

This enables the construction of new devices with exceptional performances at low cost.

The technical name for the chip is “micron scale Spatial Light Modulator”, or SLM for short, and it can be seen both as a receiving or emitting antenna (as all antennas are).

One application they are considering is developing a digital camera having a 1 pixel sensor! Today’s digital camera have millions of pixels but since those are receiving information at a maximum rate of a few thousands per second, having one pixel sensor that can receive information at a rate of tens of billions per second would provide and equivalent “quality”. Of course you would need a system able to move continuously that pixel to intercept rays coming from different points, so that the image can be constructed. It will require completely different cameras, that would be so much smaller and so much cheaper! Indeed it would revolutionize our idea of digital photography, with a camera that can capture absolutely everything in perfect focus, without focussing!

And the photographer in you that would immediately start to objects since defocusing (bokek) is an important part of the composition of a photo will have to acknowledge that a bit of post processing can easily create all the defocussing one may wish to have (it is already a standard feature of CS6).

Let’s get ready for yet another shift in our (falsely) consolidated world.

Most cell phones today have a digital camera and a growing number of digital cameras start to have a connection of same sort (usually WiFi). What about connecting a cell phone with a digital camera?

Well, there have been a number of apps letting you to use your cell phone as a trigger for remote click. We can expect to have the possibility of seeing the image on our cell phone in a sort of “Live View” mode (if you happen to be a Nikonian you know what I mean…).

Now I see that Trigger is proposing an App, on Android and IOs, that capitalise on both the cell phone and the camera to enhance your photographic capabilities.

The figure on the left gives you an idea of what can be done.

You connect your cell phone to your camera (using a dongle) and you can control it leveraging on some of your cell phone capabilities.

In the case shown you take advantage of the GPS in your phone to direct the camera to take a picture every 10 m. The App on the cell phone will measure the distance with the GPS and will trigger the camera to shot.

It is just a tiny example (there are more with this specific application) but it shows that the cell phone and a digital camera can operate as a new system cooperating with one another.

I feel this is going to be a path of evolution in the coming years. Objects will talk to one another more and more and the overall ambient will grow in capability.

Another interesting twist resulting from this evolution is that some objects may “de-facto” take up the role of “orchestrator” and possibly “virtualizator”.
My cell phone may take up the role of my personal interface. As I start to control my camera through it, I might get used to have that interface and as I will change the camera once a new sensor becomes available I will continue to use my cell phone. More photographic features will be controlled through the phone and will be “provided” by the phone.  In the end this might change the whole value chain.

Take a look at the video showing Trigger in action:

[vimeo 40453214]

AT&T has announced the sale of the Samsung Galaxy Camera with a connection data plan. Users will be able to send photos using AT&T HSDPA fast data network to share photos at the same time they “click”!

The camera is also equipped with LTE, 4G, wireless and it is very likely that also this network will become part of the AT&T offer.

With this camera you can also browse the internet but you cannot make phone calls (at least for the time being…).

Now, isn’t this interesting? Today we have most cell phones equipped with a digital camera but the quality of the photos is not as good as that you get out of a proper digital camera.  And we have Memory card with embedded WiFi to let you send the picture you just took with your digital camera to Facebook or any where you like. Clearly this latter solves the problem of having a good quality camera and being able to immediately send your photo but we do not have WiFi connectivity everywhere, whilst we have a much more ubiquitous 3G connectivity.

So this explains this new product! Still, I am curious to see what kind of success it will have, What is most interesting to me is that it is something different from a phone, and yet it is being marketed by a phone company. Whilst I am sure that in the future we will have plenty of objects having an embedded connectivity and using such a connectivity as a way to deliver part of their functionality I see this move of AT&T as a way to affirm that connectivity is not embedded but explicit. You want to send photos, you need connectivity and hence you buy a camera from them.

Would you also buy a doll from AT&T in the future since that doll interact with your kid using connectivity to a server? Or is it something you will buy at a Toy Store with embedded connectivity?

Our cameras have reached the 30+ Mpixel resolution and although pixels are not the silver bullet for better photos they easily capture the imagination of everyone of us, since it is so easy to understand a concept (although an incorrect one) of more pixels better resolution!

Actually, the resolution depends on two main parameters: the optical resolution of the lens and the way the software manipulates the data harvested by individual pixels. Our intuitive idea that a photo is like a mosaic where the individual tiles are the pixels is just wrong! Nevertheless, it is so convincing that we keep believing it. So more pixel is good!

Given a certain surface (the sensor surface) once you exceed a certain number of pixels you no longer increase the potential resolution since you start getting more noise. On the other hand, increasing the sensor size requires an increase in the lens size and makes it not just bulkier and more expensive but also more prone to defects that lead to a decreased resolution.

Insects have found a way to increase resolution by using many more eyes (composite eye). This is the approach being taken today for pushing the resolution beyond present limits, and it takes place on both directions of increasing the number of cameras and increasing the software capabilities.

The gigapixel camera, like an insect eye it clusters many cameras (98)

The increase in pixels count by increasing the number of cameras is presented in an article on Nature, published on June 20th 2012.

It has been written by a team of researchers from the Duke university, from the University of Tucson and from Distant Focus Corporation.

They propose using 98 commercial 10 Mpixel cameras (their sensors) and to combine the generated signals through a software that can create an image in excess of 1 Gpixel. One should note that multiplying 98*10Mpixel does not equal 980Mpixels because the resolution pixels are being generated by the software program and may be lower or greater than the number resulting from a direct multiplication. Actually, the researchers claim in their article that this system can be pushed up to generate a 50 Gpixel image.

This is because the sensor pixels are not translated one to one. A micro area of a sensor contains a pixel with a filter that lets only light wavelength corresponding to our vision of green to reach the sensor. The nearby left pixel on the right will capture wavelengths corresponding to red and the one to the right wavelengths corresponding to blue. But that same pixel is also juxtaposed to four other “green” pixels in the  corners and to two more red and two more blu pixels in the up and down direction. The software analyzing the data can therefore create “resolution” pixels by combining all these 9 pixels and this create a set of sets resulting in 31 different possible patterns for every single sensor’s pixel. Hence you can increase the resolution by 31 times with respect to the number of physical pixels in the sensors and with some more tweaking you can increase it further (up to the claimed 50GPixel). Since all of this happens in software there is no problem to run into problems with the lens resolution limitation.

This is the kind of image resolution resulting from such an approach:

A panorama containing such a resolution to spot birds in the distance