Imagine a library containing billions of books without any centralized organization and librarians. Anyone may add a document at any time. How would you access a piece of information in a few seconds ? It looks like the search on the WWW.

Search engines, like Google, have computer programs retrieving pages from the web, indexing the words in each document, and storing this information in an efficient format. This means that, for most searches, the result will be a huge number of pages. What is needed is a means of ranking the importance of the pages so that the pages can be sorted. One way to determine the importance of pages is to use a human-generated ranking. This is what “PageRank” does.

“PageRank” is a link analysis algorithm used by the Google that assigns a numerical weighting to each element of a hyperlinked set of documents with the purpose of “measuring” its relative importance within the set (source, wikipedia.org).

Amazingly PageRank can be used also in Computational Chemistry.

Researchers at Washington State University have realized that the interactions between molecules are similar to links between Web pages. They have adapted Google’s PageRank to understand how molecules interact. PageRank algorithm is particularly efficient and capable of looking at a massive amount of Web pages at once; similarly, it has been used to characterize quickly the interactions of millions of molecules and help Researchers predict how various chemicals will react with one another.

PageRank adapted for Computational Chemistry

This is a nice example of Industrial Mathematics cross-fertilization: an algorithm, invented for search on the Web, is adapted and used by Computational Chemistry.

Any further cross-applications ? Well, Chemistry (loosely speaking) is studying the dynamics of atomic units, self-aggregating by attractions and bonds, in a constant flurry of motion and change. Replace the atomic units with nodes (devices, smart objects, sensors, machines,…) of future networks and think about their self-aggregation for fleeting into networks in a highly dynamical environment…

They have estimated that, in less than ten years, there will be a few hundreds of billions of electronic devices connected with each other and to the Internet. Services and data will be virally delivered through multiple devices, machines, objects interconnected by dynamically emerging networks. This raises many important techno-economic issues for Stakeholders to consider: capturing the “simplicity” behind this scenario will determine great advantages. Welcome again to world of complexity. How handling it ? Let me use once more the metaphor of a termites nest.

Have you ever thought of managing the behavior of a single termite ? We know it is not possible. On the other hand, even without centralized control, we realize that a termites’ nest is a wonderful example of self-organization. Adaptation emerge from a myriad of interconnected simple behaviors.

When we imagine future networks at the edge, we see a myriad of nodes, devices, machines and smart objects interconnected through embedded communication capabilities. In principle these network entities will be simple (metaphorically like termites) and we cannot expect to manage them. Networks and their properties will emerge dynamically as results of a myriad of interactions (like in the termites’ nest).

Obviously it will be still important for Stakeholders to keep a certain level of communication and control with these self-organized networks (e.g. for meeting overall business and operational objectives). If will not be impossible controlling the behavior of each single entity of the network, on the other hand it will be possible to guide the network evolution by altering the context and interacting with all the factors which contribute to shape it (which is like altering the physical context of the termites’ nest).

This is a sort of reflexive communications. One means for handling complexity is context steering: a reflexive, decentralized steering of the context conditions of nodes enabling self-referential internal control of each individual node (which have to be sensitive to the context).

Drosophila embryo interpreting morphogens (communicating positional information to individual nuclei) with very simple physical principles. T. Gregor, D. W. Tank, E. F. Wieschaus, and W. Bialek. Probing the Limits to Positional Information, Cell (2007).

 As another example (even more complex), consider the morphogenetic field, proposed by experimental embryologists to account for the self-regulative behavior of embryos: it is based on the concept of diffusion of chemical signals or “morphogens” which are altering the cells context and as such steering cells evolution (one of the most investigated issues by A.Turing).

Have you ever read how ants build trail networks in their nests ? It’s amazing. There is no planner, no centralized control but the network emerges from self-organized feedback mechanisms: ants leave small amounts of a chemical compound -a pheromone- as they move across space. What’s more, these networks are highly efficient for searching and transporting food!

One question has always fascinated scientists: what’s the algorithm that governs the way ants respond to pheromones. In the past, it has been assumed that a trail can only be reinforced if ants have a disproportionately higher probability to follow a trail with higher pheromone concentration: i.e. the way an ant tend to turn towards a pheromone deposit is related in a non-linear fashion to the concentration (even if this is conflicting with the Weber’s Law, which relates the perceived intensity of a stimulus to its physical magnitude).

In this paper, surprisingly they have developed an entirely new perspective: Non-linearity does not reside in the perceptual response of the ants, but in the noise associated with their movement.

Evolution of the pattern formed by one colony over time.

This is how simply in Nature a random system (as noise) is transformed into a coherent one.

Now, let’s imagine an application for distributing contents across a network whose capacity is highly variable, according to the dynamics of traffic flows. Exploiting what we’ve learnt from ants’ behaviors would mean that an initially ”planned” network could adapt autonomically through the noise associated to its traffic dynamics. Simple like that. It’s Nature.

I wish resuming from the nice piece of comment of Roberto to the post Network Science.

 Actually, in most cases, there is no network! The spontaneous, autonomous interactions taking place are forming a fleeting network without it having to exist in a physical sense. It is just in our perception. This is crucial when we come to think about future telecommunications networks. The CAPEX for such networks may be 0,  since it is hidden in the nodes. It is the collection of autonomously interacting nodes that creates the network. This latter is an emergent property of the set.

 That’s true, I see it : in most cases the network is just a creation of our mind in order to understand and explain certain complex phenomena (I mean  not only when studing a cell but also when looking at a telecommunications network)

The intricate network of microtubule (yellow) and actin filament (purple) fibers that builds a cell's structure. Credit: Torsten Wittmann, UCSF.

 In 1948 Claude Shannon published his paper “ A Mathematical Theory of Communication”. He argued that “the fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point.”

 In this paper about Order and Chaos of networks, they borrow from Shannon arguing that every process is a communication channel. Again this is an abstraction to model a process or a system. Any node of a network is like a web of channels communicating its past to its future through its present.

 Actually, the state of a system or a node in a given moment of time can be characterized by values of state variables (at that moment). The minimum number of independent state variables which are necessary to characterize a state is called the number of degrees of freedom and it can be represented in an n-dimensional space (phase space). In a node’s phase space, a process is a series of gradual changes (a trajectory).

So, we may conclude that:

Any node is like a channel and the nodes of a network interact with each other again through other channels.

 In this sense a “network” is an abstraction of our mind: it is a web of communication channels, but it might not exist physically, being potentially embedded, hidden, into the inter-, intra- nodes processes.

 Time to change our perception of the network ?

In this paper “The network takeover”, Albert-László Barabási elaborates how data-based mathematical models applied to complex systems are creating a new rapidly developing discipline: Network Science.

We will never understand the workings of a cell if we ignore the networks through which its proteins and metabolites interact.

Understanding a cell through the networks of its proteins and metabolites

I would add, we will never understand the workings of an ecosystem if we ignore the networks through which its components interact. And despite the many differences in the nature of the nodes and the interactions the networks behind most complex systems are governed by a set of fundamental laws. Universality is one of them.

Welcome to Network Science: the aims is to understand the characteristics of networks that hold together the components in various complex systems.

Today, the huge amounts of data collected through sensors and smart devices are creating a new way to understand the inner behavior of many complex systems, and the networks behind them. I’m not just talking about communications: consider for example the proteomic tools allowing to collect data on human proteins networking.

No understanding of a cell, of social media or of the Internet can ignore the network fundamental laws. Data-based mathematical analysis will pave the way to this understanding.

Imagine what we may exploit from understanding these networks.

“Suppose we immerse a large porous stone in a bucket of water. What is the probability that the centre of the stone is wetted?” This is how Geoffrey Grimmett begins his book Percolation.

Percolation is a mathematical theory dealing with fluid flow (or any other similar process) in random media. It is a theory of great theoretical importance in diverse fields as biology, physics, and geophysics. Amazingly I’ve just learnt that it may have an immense practical importance in the context of future networks.

Percolation Theory

Imagine a network as a…porous stone immersed in a in a bucket of water! Let’s make an example, very much related to Roberto’s post of 20th December. Consider a wireless networks composed by an large number of communication entities interacting with each other by overlapping their communication halos. Imagine also we wish to find control algorithms based on local decisions, adaptive to local changes, aiming at guaranteeing full connectivity and efficient routing in this meshed network.

Well, this paper (by Caltech) solves this complex problem, in an effective and elegant way, by using (a variant of) the Percolation Theory.

This is to me a nice example how breakthroughs involve crossing diverse scientific disciplines and finding new way of solving problems with methods developed in other contexts.

We’re seeing already some prototyping today, when considering the introduction of Self-Organizing Network (SON) features in LTE, or embedding autonomic-cognitive features in IP routers. These avenues of research have been motivated by the need of simplifying human efforts in management and optimization of a network environment which is becoming even today more dynamic, heterogeneous and complex. Imagine in 10 years.

Well, independently from the network technology (e.g. wireless or wired), this progressive embedding of self-* features into network nodes (or ensembles of nodes, sub-networks) will transform them into players of dynamical games. Node’s resources/functions (e.g. levels of power or load, different transmission rate, flow control, etc) can be seen as the action sets of the players while the algorithms, methods, self-decisions used by the nodes to modify their behavior recall utility functions and learning processes within the game.

Actually, Game Theory is  the study of mathematical models of competition and cooperation between interacting decision making players: the developed models have the strength of helping to define optimal strategies. Game theory together with its extensions to an evolutionary context has become an invaluable tool to address the evolution of cooperation in population of interacting entities…as exactly it will happen in future complex networks. This is like saying that the dynamics of a network can be seen as the result a multitude of micro-games (of autonomic nodes and/or sub-networks); and also that the related attractors (of said dynamics) are direct consequences of strategies of cooperation and competition between the Players of such games.

Network nodes interacting as players of a game

 

Interestingly parallels between phase-state transitions and evolutionary game theory are also possible. Looking at the prior-art, most of the research done so far on complex networks has addressed either phase-state transition on a network with a fixed topology or topological transformation of a network with no dynamic state changes. In reality, these two dynamics interact with each other and coevolve over the same time scales. Actually, modeling and predicting state-topology coevolution is now recognized as one of the most significant challenges in adaptive networks. For example, the behavior of a group of Users in making use of a set of virtual resources impacts the virtual topology changes in the network interconnecting said resources; at the same time, said virtual topology can affect how the behavior of a group of Users changes.

My take is that these interdisciplinary links between physics, biology and social sciences can generate new way of running future networks as an arena of dynamical games.

Universality is one of the most studied and fascinating challenges in physics.

Universality originated in the study of phase transitions in statistical mechanics. Loosly speaking, a phase transition is a change of state of a nonlinear system occuring when a control parameter  is varied across a critical point. For systems exhibiting universality, the closer the parameter is to its critical value, the less sensitively the state depends on “the details” of the system.

This concept is applicable to networks as well. Considering a network (whatever, from a neural network to protein-protein interaction network, from the Internet to a social network) the question is whether there exist certain characteristics making the networks to exhibit universal dynamics, regardless of their difference in topology, structural changes or perturbations.

Human Protein Interaction Network

Let’s suppose that we wish to design a network that is stable and robust to external perturbations. Looking for the universality characteristics of the network means finding the way to make network dynamics stable against any structural changes that may be caused by attacks or failures.

Amazingly, this seems to be possible. Have a look at this paper.

In particular, in  the paper they claim they have found “weighting schemes for which the details of various real-world networks, whether biological, technological, or social, have little influence on typical dynamical processes such as synchronization, epidemic spreading, and percolation”.

Imagine the impact of discovering and applying universality principles to future networks, and not only… !

Yesterday I’ve made a talk at ITU Telecom World 2011 on challenges and opportunities of future networks: in particular, I’ve focused the discussion on the “edge”. Technology advances and the related cost reductions are paving the way to a wider and wider embedding of communications, storage and processing powers inside any network nodes, machines, smart things and any Consumers’ electronics devices.

Then we can argue that future networks, more specifically at the edge, will interconnect a huge number of real and virtual entities providing the Users with any services by using local processing and storage resources.

The thesis I’ve put forward during the talk has been that this future growing complexity of edge networks will require looking at them with the mathematical instruments of nonlinear dynamics. Traditional approaches will be no longer applicable.

Nonlinear dynamics is the theory of nonlinear systems and processes, those where the “result” is not proportional to the “cause”. It includes theory of deterministic chaos (which doesn’t mean random disorder). Chaotic systems behave like there were stochastic but in fact they are deterministic: they show predictability in a short-time-scale but non-predictability in a long-time scale due to extremely high sensitivity to initial conditions and to system’s parameters.

So, I’ve argued that the myriad of real-virtual entities interacting with each other at the edge will behave like a chaotic system, a sort of dynamical game where rules will change dynamically. The challenge is modelling the related dynamics (beyond Nash equilibria) and mastering said complexity to extract value.

Overall the vision has been well received and it has created an interesting debate; one of the most interesting questions I’ve got has been: how can we really apply nonlinear dynamics (which has per se a rather complicated mathematics) to such a complex environment as future edge networks, where there will be millions of nodes? I’ve replied that, obviously, I don’t have an answer today, but I a relatively strong feeling, which follows.

In physics state of a system in a given moment of time is characterized by values of state variables (i.e. data). The minimum number of independent state variables that are necessary to characterize the system’s state is called the number of degrees of freedom: if a system has n degrees of freedom then any state of the system may be characterized by a point in an n-dimensional space (with appropriately defined coordinates) called the system’s phase space. Attractor is a subset of the system’s phase space that attracts trajectories (i.e. the system tends towards the states that belong to some attractor).

Why don't model states attractors rather than single neurons ?

The behaviour of a local network (or recursively a node), like a nonlinear systems that change with time, is dominated by a relatively small number of “attractors”, which correspond to activity patterns (i.e. eventually sets of data). So let’s abstract said behaviour of a local network with the attractors. Then, let’s suppose that these local networks communicate with each others by means of multiple connections, that is, by activity patterns (i.e. sets of data). Let’s abstract also this: the degree of influence that the state of one local network would have on the state of other ones would be given by a “multi-dimensional matrix” coupling attractor states.

Turn nonlinear complexity into interactions of attractors and you’ll get a simpler picture. What if we push this approach even up to getting User’s behaviours in terms of attractors ? I think that for a Telecom Operator it’s better knowing the anonymous Users’ attractors rather than the termination attached to them.

We are staying in contact with the guys that I met at the conference to make a toy model and some simulations.

I’ll keep you posted.

In a recent post we’ve elaborated about the challenges of understanding analogies between neurons networks and future networks in order to exploit some of the principles of nervous system functioning. An ideal scenario is viral networks at the edge, where, in the future, a huge number of small self-adaptive nodes (enactive nodes) will be able to create dynamic networks (of processing and storage resources) expanding and contracting over time. Each node will be able to perceive its environment and  to self-adapt dynamically cooperating and competing (in sharing functions and resources) with all the neighbor nodes as in “games”.

How can we embed a sort of “nervous system” in such simple nodes?

We wish to go beyond the traditional concept of neural networks to increase the level of flexibility and learning features.

Imagine such node’s nervous system as a dynamical system with a certain number (theoretically the higher the better) of dimensions so that mathematically it can be treated as a field, a continuous space where the nervous activity takes place. This is the basic idea of Dynamic Neural Fields (DNFs). This is an example of equation modeling it.

Example of equation of a DNF

DNFs have amazing applications in the fields of A.I. and Robotics. As an example, have a look at this recent paper:

 http://www.uni-ulm.de/fileadmin/website_uni_ulm/iui.inst.130/Mitarbeiter/oubbati/Publications/OubbatiICANN11.pdf

 Researchers have modeled the flocking behavior of a number of entities (e.g. agents) through DNFs. Simulations have shown the emergence of a synchronized motion of the group even without a leader; entities’ behaviors have been first transformed to separate stimuli entries of DNFs, and then combined in a global stimulus by assigning a situation-based priority to each behavior (remember the loops of F. Varela). In this case, the simulations have demonstrated the feasibility of the DNFs approach to simulate an emergent flocking behavior. Now they are planning a real-world implementation for a swarm of robots.

 In one sentence, DNFs represent an example of a nervous system-like “missing link” between sensors and actuators. I see potential applications of this avenue of research not only in A.I., robotics but also in future networks at the edge. Certain levels of cognition can be embodied (with DNFs-like approaches) into enactive nodes (as introduced in a former post) to create, dynamically, viral self-adapting networks of networks.