Systems modelling: how useful is it ?

Modelling complex systems, be they social, economic, terrorist-ic, environmental, ecological, whatever seems to be all the rage nowadays. Everyone (including myself!) seems to be so intrigued with the future that they cannot wait until it arrives.

But what is a model ? and how can/should it be used ? these are questions which are normally unanswered and can lead to disasters like the current financial circus. A systems model is, at its most abstract, a simplification of reality which can help us to answer questions about the future in a computationally tractable way (since everything at the end of the day gets reduced to number crunching, except maybe philosophy). It focusses on internal variables (state), inputs and outputs of a system that are considered important to understand its behavior. The contribution of a model is two-fold: to understand the underlying (simplified) structure, and to use this to answer questions that interest us.

We have to face that we cannot really understand even moderately complex systems properly, and we make certain idealizing assumptions (like there won’t erupt a World War), to make things simple. We then use an intuitive understanding (sometimes called pre-analytic vision) of the system structure to decide how to build the model. For example, economics models are built using the vision that man wants to maximize something (which makes it easy to use calculus of variations or mathematical programming), atmospheric models have to obey the laws of physics, and so on.

Once we identify a structure which can be represented using available mathematical tools, we put them in a computer and start crunching. If they can be represented using nice equations (called deterministic model), you would use differential equations or some cousin or mathematical programming. If it cannot, then you don’t give up, you simply say that it is random but with a structure and use stochastic models, using Monte Carlo methods or time series analysis or some such thing (Read this for a critique of the latter).

Before one gets immersed in fascinating mathematical literature, one must understand that each model comes with an ‘if’ clause: If such-and-such conditions are satisfied, then things might look like what we get here. Which is why I get irritated with both MBAs who talk about ‘market fundamentals being good’ and environmentalists who predict that I’m going to need a boat soon – neither qualifies results which come out a black box. Even worse, there are those who compare results which come out of different black boxes, which need not be comparable at all, just because they, like the modellers, have no idea what is going on. Atleast the modellers admit to this, but those who use these models for political purposes cannot dare to admit shades of gray.

Different models can take the same data and give you radically different answers – this is not due to programming errors, but the pre-analytic vision that the model is based upon. The reason why climate change remained a debate for so long is because of such issues, and vested interests, of course. Therefore, we see that the ‘goodness’ of a model depends critically on the assumed ontology and epistemology, even more than the sobriety of its programmer (well, maybe not).

Thus, as intelligent consumers of data that models spew out everyday, we should make sure that such ’studies’ do not over-ride our common sense. But in the era of Kyunki Saas bhi, one cannot hope for too much.

Information and Energy, and Entropy!

What follows is quite incomprehensible if you do not have some idea of maths/physics, but read on anyways :)The question that I have been asking for quite some time is : “What is information?”, from an abstract point of view. Would have hardly expected a physicist to answer this, but they did. Looks like there is a connection between the energy of a system and the information it contains. More precisely, the number of states that a system can take is directly proportional to the information it contains. This was put forward by Boltzmann, who is rightly well known for his huge contributions to physics. Take for example two people, A who sits on a chair the whole day and B who keeps running around the whole day. If someone tells you A is sitting on the chair, you would already know it, so it is nothing new to you. However, news about B’s whereabouts will always be new information to you. Therefore, an energetic system tends to contain more information than a static one.

One could also understand this by looking at a storage cell, which can contain n bits. As n increases, amount of information stored increases, so does amount of energy it contains (Not to be confused with present day memories like RAM, where most power is consumed by resistance and (silicon) crystal imperfections.) Boltzmann stated that if a system can have mutually distinguishable states, then the amount of entropy is given by . Entropy can be called a measure of the randomness in a system, that part of the system’s energy that is unavailable for useful work.

Claude Shannon, founder of practically everything we know (if we exaggerate a bit), generalized Boltzmann’s hypothesis to a case where we have some idea of the probability of a state of the system occuring, i.e, we have a probability distribution of the states of the system. In Boltzmann’s view, the distribution was uniform, and hence there was the result. In a probabilistic system, one can only talk of the expected information, and that is what we get: Information entropy is given by . Note that this reduces to the previous form if . Here, since the system behaves in ways we understand, the total entropy has been reduced by a factor . If the base of the logarithm being taken is 2, the unit of the value is called bit. If natural logarithms are used, it is called a nat. Nat, interestingly,  corresponds to the Boltzmann constant when used to in an entropy context. Thus, we see the beginning of a link between entropy and information. We reduce the entropy of a system by gaining more information about the states in which it can be present. If we know exactly what state it is in, the amount of entropy is effectively zero.

How do we find the distribution of states ? we have to measure. Therefore, what is being implied is that measurement reduces the entropy of a system, or that measurement reduces the uncertainty we have about a system, which seems intuitively correct. If system A measures system B, B’s entropy reduces, whereas the entropy of the system consisting of both A and B does not (from the viewpoint of a system C which has not measured either A or B). The information got via measurement must be stored (and/or erased) somewhere, and this requires energy. This could be seen as the solution for the famous Maxwell’s Demon paradox, which claimed to violate the second law of thermodynamics.

These views have importance in the theory of computation, especially in the lower limits of energy required for computation. Say, I have a system that takes in 2 bits and gives out 1 bit (like an OR gate), then the amount of energy expended must be atleast equal to the difference in information entropy, which is 1 bit. Similarly, if information entropy increases, the system must take in energy. You can read all this and more (especially qualifications) in this paper.