Gaming has reached levels of fidelity with reality that the same approaches are starting to be used for simulating real life job. This is true for aircraft piloting, as well as driving cars and more and more in surgery training.

This is the case reported by a paper published by Wolters Kluver few days ago on the use of a simulator for training future neurosurgeons.

NeuroTOuch – National Research Council of Canada

Researchers have developed a brain surgery simulator with 3D graphics able to match what a surgeon would actually see through the microscope used for surgery. The simulator makes use of haptic interfaces shaped as the surgical instruments used during the operation to recreate the exact feeling to the trainee. A consumer PC is now sufficiently powerful to support this kind of simulation and precise force feedback in the interaction.

The graphics make use of the same “engines” being used in games, and also the engagement of the students follows the same paradigm. Students gets points determined on the accuracy and speed of the operation, when they remove cancerous tissue they get penalties if they touch healthy parts of the brain…

Using the gaming paradigm in education (gamification) is now becoming common in several areas, from learning physics and math, to chemist. And it is likely to be adopted also in other subjects.

Take a look at a video clip of the simulator at work (if you are not faint of heart):

Researchers have found various ways to simulate the sensation of touch (haptic interfaces). Now they have gone a step further by finding a way to provide a different touch sensation.

This has been demonstrated at SIGGRAPH 2012 by Disney researchers who have found a way to trick our touch by circulating a weak electric current through our body as our fingers touch an object.

The physical principles is to send an oscillating current (at high frequencies currents mov on our skin, not inside our body) that interacts with the object surface (the object has to be connected to the same computer generating the current in your body) and creates the artificial sensation.

In the future, I can imagine, we can touch a teapot and feel it wet if it contains tea inside, dry if it is empty… (you can also imagine touching your car near the gasoline tank and get this kind of “touch signal”).

It is yet another way to provide a sort of augmented reality!

Researchers are perfecting technologies to be able to replicate our sense of touch, for a robot and for ourselves.

For the first time a team of researchers working in the VII European Framework Program have managed to create an artificial finger that mimics quite well the set of data that one of our finger will pick up by touching a surface.

There are several technologies involved. First of all, of course, you need to have very tiny sensors that independently can capture irregularities in the surface being touched and transform them into data. Then you need to “integrate” the single sensitivity in a set, like it happens with our finger. For that the researchers use a synthetic skin that is layered over the sensors. Then you have to translate the row data into signals that can be transmitted and then interpreted by … a computer or a brain.

Indeed, since they aim at creating an artificial finger that can be used by a robot (a computer) and a human who has lost the hands (a brain), the data have to be adapted to the receiver. In case of a robot you can program its computer accordingly, but in case of a human you need to stimulate the peripheral nerves that remains attached to a prosthetic limb in just the right way for them to carry the signals to the brain.

The artificial finger is equipped with an array of pressure sensitive sensors mimicking the spacial resolution of a real finger. The artificial skin covering the array allows them to perceive vibrations. So far experiments indicates that a person using it would be able to determine if the surface is smooth or scratchy. It is a start. Our sense of touch is very refined and to mimic it in all its nuances is a very difficult task.

Well of course, I am not entitled to judge future habits but still looking at  Lovotics last product … well it feels creepy to me!

Kissing via Internet … as real as it gets

It is a robot, dressed up in various shape to your liking, with a pair of lips that you can kiss. Why would you want to kiss those lips? Well they are connected to sensors able to feel the pressure of your lips. A computer transforms those pressures into messages that are relayed to the other robot connected to yours via Internet. Actuators will reproduce your kiss on those lips so if your partner has her lips on the robot lips it can feel your kiss, or so it is claimed.

Not sure if it is going to be “the thing” for the 2013 S. Valentine day… We’ll see.

What is good to see, is the progress being made in the interface domain, with sensors able to pick up subtle nuances and actuators replicating them.

If you are curious to see Kissenger at work take a look!

In Italy we are used to gesticulate as we speak. And we understand very well each other. However, we do not understand, in general, the hand language used by people who cannot speak. And this applies, of course, to most people in the world.

Why can’t technology come to our help? Indeed, this is what three Ukrainian students have been pondering and, presto, they come up with a solution. A pair of gloves with embedded sensors that can relay the movements of the hands and fingers to a computer for a translation of gestures into voice.

They called their invention “enabletalk”  and enter it into the Imagine Cup 2012 competition that was held few days ago in Sydney …. an they won!, The competition is promoted by Microsoft and targeted to young inventors who are asked to use technology to solve real problems.

In the picture on the side you can see the sensors they have embedded in each glove to be able to report each type of movement to the computer for analyses and translation in spoken words.

The sensors, 15 in each glove, plus an accelerometer, a compass and a gyroscope, are connected to an embedded controller that communicate with the computer via Bluetooth using a cell phone as an intermediary (you do not want to lug a computer around as you walk but you will have your phone with you).

The sensors are powered by a lithium battery that is recharged using a solar panel sewn on each glove.

Take a look at the video clip…

Besides being a very useful invention, it provides us with a clear example of the kind of evolution we can expect in the future years in the area of interfaces. They are likely to provide us with more and more seamless ways of interaction.

Haptic is progressing towards providing more real life sensations. Such is the case with the technology developed by EPFL researchers. Take a look:

Researchers at the Ecole Polytechnique Federale de Lausanne have invented a way to create tactile sensation on special surfaces that can change their shape. Imagine touching a touch screen and feeling the texture of the object you see under your fingertips or moving your fingers on your touchpad and feeling resistance to movement…

Where is the trick?

The researchers have inserted between two transparent layers nano particles that vibrate once activated by an electrical current. Their vibrations create a movement of the surface of a few microns, something that is too tiny to be felt by your fingertips. However, the vibrations of the surface create a movement in the air and your fingertips are tricked by feeling these movements.

By changing the intensity and frequency of the electrical stimulation one can create different vibration patterns and these in turns create different displace of the air. Basically, all sensation our fingertips are able to pick up can be recreated.

So let’s get ready for a whole new feeling when touching a screen. It will no longer feel like touching a glass surface but like touching that what is being displayed at that particular time, an orange, a lake or the cheek of your kids.

Disney and CMU have demonstrated a new tactile technology  that is a significant leap forward with respect to present haptic interfaces where you can feel something with your hand. Now, it’s you whole body that can sense what’s going on!

The name given to this technology is, aptly so, Surround Haptics and it is based on a set of vibrators that are surrounding your body. The vibration is controlled by a computer, on the bases of perception models to recreate the feeling of motion, of being hit by objects and much more.

You can read the details on line.

What matters to me is that once this kind of devices becomes mainstream people will discover a new need for optical fibres. Only with this kind of connectivity you’ll be able to really feel what’s going on because our sense of touch requires refresh times in the order of 1/1,000 of a second (millisecond) and that means basically no delay in the telecommunications infrastructure, something that can only be achieved once you have a full optical connectivity, end to end.

Imagine your friend shopping and telling you to feel the softness of a cashemere sweather she is considering buying and yo having to respond that you can’t actually feel it because of lack of bandwidth and processing power. Sound strange?  Just because you are not used to haptic virtual reality, yet!

Haptic Virtual Reality lets you feel virtual objects

Researchers at the Swiss Federal Institute of Technology (ETH) in Zurich have demonstrated a system for creating a virtual model of an object through 3D laser scanning (already commercially existing and used to create 3D images of statues in museums aroud the world) and supplementing it with information on texture and touch feelings so that a person can look at the virtual object and manipulate it with an haptic interface to feel it as if it were in her hand.

http://www.ethlife.ethz.ch/archive_articles/100816_virtuelle_realitaet_cho/index_EN

When I read this news I started to consider what would it take to bring this idea into a mass market that I assume would be interested in augmenting the possibility of communication from voice and images to real presence. Touch is a fundamental sense to provide us with a perception of “being there”.

The work being done at ETH is important because it provides the conceptual tools and practical devices to convert the sense of touch into a model that can be transmitted and re-enacted to let people feel the object. But how much would the burden on the infrastructure?

For a single object it is not that much. You need to create a model of it that contains the visual information, hence you have to take a 360 degree image of it, let’s say you have to take some 32 images (each one covering 45 degrees horizontally and vertically to provide a high quality modelling). That’s equivalent to little more than one second of filming the object. Then you have to take sample of the texture of the object with an haptic device so that this texture feeling can be recreated at the viewing end. This is not much information, may be equivalent to doubling the previous amount of data used for the visual part.

Let’s say then in order to create this model (and transfer it) you need an equivalent of less than three seconds of high definition movie, that is something like 4-10 MB of data (depending on the quality you desire).  This is ok to create a virtual representation in the “web”. Now, this is not a tremendous amount of data “per-se” but if we imagine that MMSs will morph into HMMSs (Haptic MMS) then it means to multiply by 30 times (on the average) the amount of data created today by MMSs.

On the receiving side there are two options: transferring the model to the terminal device being used and have it perform the transformation of the model into a perception or have some application in the network (or connected to the network) to take care of this. The former requires a high processing power in the terminal the latter a high bandwidth channel to the terminal. The problem is that our sense of touch needs to be stimulated 1,000 times per second (compare this with the 16-20 times per second to create the perception of movement through still images, as it happens in movies and television) and this requires high bandwidth and low latency.

It is likely that a mixture of network and terminal based processing will take place. Processing is not a really limiting factors since we know that it keeps increasing at the Moore’s law rate. More tricky is the demand on power that is the really limiting factor on mobile, battery powered terminals. This is the reason why network based computation may be required. But, alas, this brings along the requirement of bandwidth. It will be interesting to see the evolution in the second part of this decade as a variety of applications, the one being discussed here is just one of them, will steadily increase the bandwidth demand.