Remember the movie Matrix where the characters were interacting with the world using gestures? It is now getting closer to every day life, thanks to Oblong, the company founded by John Underkoffler the very same guy who provided technology consultancy to the authors of the Minority Report movie.

If you have some time to spare, take a look a the two clips (lengthy ones but worth watching) where a journalist interviews John and he is shown what technology let’s you do with just “your hands”!

What I felt interesting goes beyond the possibility of interacting with an image on a screen with my hands. What really intrigued me was a sentence spoken by John, saying ” in the future everything will be a pixel”. All you see around you will be a pixel in the sense that it will be visualised by a computer and that will allow manipulation of images. If reality is rendered in pixels then it becomes possible to transform reality and adapt it to the person in that ambient. And of course, that person will be able to warp that reality into something that fits his whims and needs. The boundaries between bits and atoms are indeed getting fuzzy…

How many times did you ask that and may be expected to be invited to taste it yourself? Quite straightforward, dig your fork in that spaghetti and meat balls and … taste them!

Of course you would not expect your question to be fully satisfied if you were calling a friend over your smart phone. Smart goes only as far…

An haptic jacket on a chicken let Adrian pat it from his office…

But now a professor at the Keio University, Adrian Cheok, is working to let you taste from a remote location directly on your cell phone!

This is part of his general interest to transform the Web from a place of information to a place for experiences. He has developed a variety of haptic interfaces like the one shown in the photo where a chicken donning a special haptic jacket can feel the “patting” sensation when he is patting a chicken doll in his office at the University.

Now he has invented a lollipop that through electric and thermal stimulation is able to recreate taste on your tongue.

Actually taste is a very complex sensory experience. If you don’t believe me, try to taste an apple with your eyes closed and your nose pinched so that you miss both the visual and olfactory stimulation. The taste will be completely different, and nothing like an apple!

Thus, the lollipop can provide you with sensations of the six basic taste components, salt, sugar, bitter, acid, metallic and umami, but it cannot really provide you with the “real” taste that is created in your brain as result of a multi-sensorial experience.

Still, it may be fun to try it … or no?

Have a look!

In Italy we are used to gesticulate as we speak. And we understand very well each other. However, we do not understand, in general, the hand language used by people who cannot speak. And this applies, of course, to most people in the world.

Why can’t technology come to our help? Indeed, this is what three Ukrainian students have been pondering and, presto, they come up with a solution. A pair of gloves with embedded sensors that can relay the movements of the hands and fingers to a computer for a translation of gestures into voice.

They called their invention “enabletalk”  and enter it into the Imagine Cup 2012 competition that was held few days ago in Sydney …. an they won!, The competition is promoted by Microsoft and targeted to young inventors who are asked to use technology to solve real problems.

In the picture on the side you can see the sensors they have embedded in each glove to be able to report each type of movement to the computer for analyses and translation in spoken words.

The sensors, 15 in each glove, plus an accelerometer, a compass and a gyroscope, are connected to an embedded controller that communicate with the computer via Bluetooth using a cell phone as an intermediary (you do not want to lug a computer around as you walk but you will have your phone with you).

The sensors are powered by a lithium battery that is recharged using a solar panel sewn on each glove.

Take a look at the video clip…

Besides being a very useful invention, it provides us with a clear example of the kind of evolution we can expect in the future years in the area of interfaces. They are likely to provide us with more and more seamless ways of interaction.

Once upon a time most customers had no choice in choosing their telecom provider. Given the area they were in they could only choose the Operator providing service to that area. That was true also for wireless at the very beginning.

Then, competition changed the rules. Each customer could choose the Operator (among a limited few…). In the last five years we have seen a further change: customers are choosing the cell phone they like and in turns this may lead to the choice of the Operator when that cell phone is only available through a specific Operator. Even when that particular cell phone is available through several Operators the real tipping point in the customer choice is now the cell phone, not the Operator portfolio of services.

Clearly it is no longer the dog wagging the tail, it is the tail wagging the dog.

The iPhone has changed the paradigm of customer decision

The services attracting the customer are by far those provided on that phone by a variety of third parties and the look and feel is what drive the choice. Design and customer experience have taken the upper end in a buying decision and these are related to the phone, not to the network.

The iPhone is certainly a point in case, but tat goes as well for most smart phones on the market and we know that their market share is increasing.

In this new scenario can the Operators change once again the paradigm?

The power of the consumer in the selection of the Operator is not going to be diminished in the coming year. And we also know that the lion share in her decision is based on the perceived experience that in turns is strongly tied to the cell phone itself. How can an Operator become part, a significant part, in the user experience?

Some new technologies may offer a chance, in particular biometrics and the cloud.

Biometrics gives the opportunity to get rid of the SIM authentication via a password replacing it with biometric characteristics of the person. The palm vein scanning from Fujitsu is an example. We can imagine that cell phones will be equipped with this (or similar) systems in the coming years and that will give the possibility for a seamless authentication with some interesting twists. An Operator can manage the customer biometric data and also set up services to grant access to the cell phone to other persons that have been authorized by their customer by cooperating with other Operators in the management of the biometric data (it may be very similar to the handover of a call from one Operator’s network to another one, in this case it will be the handover of one customer management to his service Provider).

Clearly this requires the management of customer data (the biometric data) and its protection. Those same data can be the key to enable new service access and customization and this in turns is the key for a better customer experience.

The cloud can be the technological underpinning for data management, including data sharing.

This is an open field. Operators are not the only ones that can play this game, although they have the advantage of being controlled by some sort of public Authority that can provide the seal of trust that is needed whenever we are dealing with personal data.

Possibly, the biggest hurdle to this evolution is the Operator itself that in many cases is not “culturally” prepared to transform itself from a network provider into a data manager.

Our understanding of the brain inner working are progressing at top speed, thanks to sophisticated brain imaging that let researchers correlate brain activity with input and output stimuli. Clearly, this goes far beyond the pure accumulation of knowledge, it opens up new possibilities to cure disabilities.

Decoding your thoughts and speck them up...

Just imagine you suffered a stroke that disable you speech capacity. You could still be able to think and vocalize in your brain but you would be unable to speak it up. What if scientist can read your mind and use a computer to actually speak up for you? That would be great!

And this is exactly what scientist are doing at Berkely University in California.

For the time being they have placed 256 electrodes on the brain of patients who underwent brain surgery for intractable seizures. The electrodes where intended to monitor the electrical activity of the brain on the temporal lobe to determine the origin of seizures for treatment. In the research these patients (with their consent of course) where also studied to detect specific patterns related to specific stimuli. The temporal lobe is the place where the voice is processed and the researchers where able to reconstruct the voices heard by those patients looking at the electrical activity in the area.

Interestingly, those same activities are performed when those patients wanted to utter a phrase so that it becomes possible to translate “thoughts” into spoken phrases (when we think we actually “speak” within our brain.

This news has triggered the imagination of many and it has already been published in several daily newspapers.

Now, this is very good, and clearly provides another way to interface our brain to the world but at the same time doesn’t it make you feel uneasy knowing that at some point in the future your thoughts can be “read”?

No longer needed to “read my lips”: just “read my brain”!

Where is “I”?

It is somewhere in the working of my brain. Notice that I didn’t say: “it is my brain” nor “it is in my brain”, rather “in the working of my brain.

If I can connect my brain to the web, to the cloud, the “I” might be able to live forever.  Read the Hofstadter book: “The mind’s I” and see what I mean. If there is a continuous link from my brain to its virtual self in the cloud once I pass away I could keep living in the cloud, I can still receive stimuli that would activate (virtual) sensation in my (virtual) self. And, of course, I could react to these and interact with other people (physical and virtual ones). As I interact my virtual brain will drift, it will keep learning and adapting (should I say “it” or “he”?). As a matter of fact life in the cloud can go on forever and as in the Plato’s cave will be indistinguishable from the real one from the point of view of conscience.

But, besides the philosophical questions that such a scenario opens up, if direct communication from my brain to the web becomes viable a whole spectrum of possibilities opens up.

Communications so far has been mediated by “visible” devices. We can expect that, if technology would support it, the desire to communicate without tethering of any sort will lead to a direct brain to brain communication.Technology to make this possible is there, somewhere, but you don’t see nor perceive it.

Someone may call this telepathy. From a technology standpoint we can predict that the capability to intercept volitive actions in the brain, like speaking or sharing a sound or an image… will be available. Something like SMS telepathy should be available in twenty years (the big issue is the addressing…think about your friend and that creates the pointer to an address where to send the SMS…).

The major hurdle that I see is control. I would not like to see my thoughts being intercepted nor to see that the thoughts leading to the final craft of the message are being packed into the message.

There are many technologies involved in “telepathy” that need to be finely tuned before you can use it….

By the way, a recent post circulated few days ago on the Internet was forecasting that by 2170 more people will die because of computer virus than physical virus…

This year I had not the opportunity to go and check CHI 2011 conference (Computer Human Interaction), which I consider as one of the most instructive conferences on interfaces, with a strong participation of both academy and industry research centres, like Microsoft, IBM, Xerox etc.

Anyway I had a look to the best awarded papers published online and one came to my attention: Microsoft Research and the formidable Desney Tan, one of the gurus of innovative interfaces in that company, have come out with such an easy to understand idea that I wonder why no one else have had it before (at least to my knowledge).

Our houses are cabled with electrical wiring and full of appliances: those generate some electromagnetic noise out in the surrounding environment (that noise that, at least in the past, was for instance annoying our AM radios…).

Now the MS researchers had the idea of using our own body as an antenna and measuring how we interfere with this noise, using machine learning and artificial intelligence to be able to recognize our position inside the house plan and the gestures we do. In this way we could move one hand on the wall to control an appliance in the same room. Check all details of this concept in the awarded paper.

Analyzing the electrical wiring noise, the whole house can become an interaction surface.

I’m fascinated by gestural interfaces. About an year ago I bumped into one that was developed and demonstrated by the Hasso Plattner Institut in Berlin, a research institution created in 1998 by professor Hasso Plattner, co-founder of the well-known software company SAP.

Here the idea was to develop an interface for a device which does not have a screen nor a keyboard. This interface has been called “imaginary interface”. Using video analysis the system detects the user’s hand silhouettes from a small camera hanging from the neck. One hand, that has to show an L using thumb and forefinger, is used as a reference both for the person who is sketching something in the air, and for the system that sets this hand as the origin of a “virtual whiteboard”.

The other hand is used to actually draw on this “whiteboard” and the resulting drawing is transferred via cellphone to a remote PC. When you want to use the virtual ink on the whiteboard, you have to connect your thumb and forefinger in a circle.

Have a look to the video that shows how simple this can be.

This imaginary interface will let us sending some indications to a friend, taking a short note, sketching a memo just using our free hands…

TMS or tDCS are acronyms for technologies that in the future will be associated to “computer-to-brain” interfaces, the new frontier in connecting humans directly to machines.

Today BCI or “brain-to-computer” interfaces are already existing and used in several areas, from assisted living for disabled people to entertainment application as an innovative controller for games or toys [see this post on BCI].

TMS stands for Transcranial Magnetic Stimulation. It consists in creating a controlled magnetic field that penetrates inside the human brain. This non-invasive method (meaning that no surgery is needed…) is currently used to treat some neurological conditions, like dystonia (a disorder in which muscle contractions cause twisting and repetitive movements or abnormal postures) or major depression. The magnetic field induces an electric field inside the brain, which can activate neurons. Some companies, like BrainsWay,  are manufacturing such systems, that have already being used in clinical trials.

tDCS stands for Transcranial Direct Current Stimulation. This method requires electrods to be put on the head, which drive a small amount of current through the skull and influence the area of brain just beneath the electrods. tDCS only affects neurons that are already active. In a recent article published on Nature, tDCS is mentioned for its potential to enhance the minds of healthy people. Some students treated with tDCS showed improvements in working memory, word association and complex problem-solving.

A few months ago on the neuroscience journal Neuron, a paper was published that refers to the investigation of transcranial pulsed ultrasound on neuronal activity. The author claims that this technique has a better spatial resolution compared to the previously mentioned ones, and can be used to elicit motor responses when the ultrasound stimulation is applied to an intact motor cortex. Darpa, the military US research project agency, decided to fund this research aiming to create a helmet with this non-invase direct to-brain interface.

Maybe the future to transfer bit into the brain is yet unclear, but scientists are clearly looking for that.

The images from Minority Report of manipulation of information on floating screens are still vivid in our memory. They were the product of special effects, implemented using computers. But they were not just a figment of imagination.

At that time a researcher at the MIT, John Underkoffler, was working on application of gesture recognition to the manipulation of information. Similar work were taking place in other universities.

Few years later John founded Oblong, with the idea of revolutionizing the world of human environment interface.

[vimeo 2229299]

As you can see from the video this idea is now a product, an expensive one, though. But we know that when technology is the cost, that cost is going to come down in a short time.

My bet is that by the end of this decade we will have this technology in several ambient and some geeks, or well to do, will have it in their living room. Give it a few more year and the remote controls, the light switches and the washing machine knobs will be history.