Math: The geometry of Matrices

Was attending a workshop on Linear Algebra, and one of the lectures was by Dr. C. R. Pradeep. It was supposed to be on positive-definite matrices (whatever that might be!), but finding that no one really understood what a matrix was or stood for, it became a geometry class, to everyone’s benefit.

The standard way to look at a matrix is that is a set of numbers arranged in some order in between some brackets, and that they can be added with some effort and are multiplied in a completely obscure manner. This much one learns by the time one leaves Pre-University, and this does not help one bit in appreciating the whats and the whys behind the whole thing.

The best way to start off is with an example. Consider a general vector multiplying a matrix:

If we take the general vector to be some point on the plane, then note that every point on the right hand side of the above equation has its x-coordinate equal to its y-coordinate. Therefore, if we think of this matrix as a machine that takes in all vectors in the plane and spits out some other vectors also in the plane, then things begin to look very nice.

This is because, if we start feeding this machine all the points on the plane, what we will get as a result is all the points whose coordinates are equal. From elementary geometry, this is a line passing through the origin at an angle of 45 degrees to both the coordinate axes:

How is it managing to do this ? Consider two special cases: and . These are the well known unit vectors on the plane, representing the y axis and the x axis respectively. Substituting these values in the equation, we see that they both are sent to the same point! Therefore, this matrix is collapsing the plane onto a single line, very much like closing a Chinese hand held paper fan. We can write any point on the plane in terms of the unit vectors, and similarly, we can write any point on the line using both the columns of the matrix. In this case they are the same, so it is a trivial relationship. But in general, the columns of the matrix are such that any point at the output can be written uniquely in terms of them.

Therefore, this matrix seems to be taking as its input a 2 dimensional ‘space’, i.e, the entire plane and giving back a 1 dimensional ‘space’ – a line through the origin. Another interesting thing to note is that both the points and both end up in the same point . This means given a point , we would not know which point on the plane it corresponds to, i.e, the inverse is not well-defined.  (In fact, given any point at the output there are an (uncountably) infinite number of points which could have produced that) When we say that a matrix is singular or non-invertible, this is what we mean. We normally check this by finding the determinant of the matrix (which is zero in the singular case), but this normally does not appeal much to intuition.

To generalise this, any matrix that takes a higher dimensional space to a lower dimensional space is not invertible. The natural next question to ask is that if a matrix maintains the dimension of the output space equal to the input, is it invertible ? This can in fact be proved to be true, and this can be taken as a simple interpretation of the Rank-Nullity Theorem.

Going back to our example, what if the point/vector at the input was already on the line ? For example, the point would end up at , which is the same as . This is consistent with out Chinese fan approach, the line in the middle of the fan does not really move when it is closed (It does not get elongated either, but that is a special case of this). In Linear Algebra terminology, such points/vectors are called eigenvectors, and the value by which they are multiplied is called the eigenvalues of the matrix. In the general case, the axes do not collapse into each other, but maintain some angle between them. Even then, there will be some set of points/vectors which do not move, and these are called the eigenvectors.

Almost all the basic concepts of Linear algebra can be interpreted in this geometric manner. The heart of this whole discussion is the concept of linear transformations, which are represented by matrices for convenience and analysis.

Linear Algebra is an interesting subject, and on this is built almost all of engineering!!

Spherical Cows and Geometry

The first part of the name comes from the famous book on Environmental problem solving called Consider a spherical cow by John Harte. According to the book, the title comes from a joke within the environmental science community about a group of scientists who are approached by a farmer to ask why his cow was producing less milk. After large amounts of deliberation, the scientists call the farmer and start explaining : “Consider a spherical cow….”

The title is to reinforce the fact that certain problems cannot be solved exactly, and idealizations need to be made so that the solution is computable. In fact, the book (just started reading it!) has quite a few problems which do not expect answers exact to the 100th decimal point but an answer in the orders of magnitude is sufficient, i.e, can be written as . If one begins to wonder what kind of quackery this fellow preaches, consider the first problem statement that he puts forward: Calculate the number of cobblers in the US! no other info, one has to make amazing assumptions and only then even start to solve the problem. This, for me, is to drive in home the fact that solving problems in as large and complex a system as the environment is simply too hard to do precisely as a watchmaker might like to. One has to rely on intuition and great deal of ‘common sense’ to go ahead without being bogged down by the finer details. It is uncertainties like this which are really making a debate of global warming possible. People who say that humans are putting too much crap into the atmosphere, are ‘reasonably certain’ that humans will destroy the planet. Sceptics say that there is evidence to show that global warming and cooling cycles have always been happening, and this is not too great a cause for concern. The debate, however, seems to be converging in the favor of the ‘we are suicidal ‘ camp, due to more and more data pointing to the adverse effects of anthropogenic intervention in the global climate system.

Thus, until one gets more data, one guesses, and more data only refines the guess to a state where we can be ‘reasonably confident’ (or ‘quite sure’, especially when giving lectures or talking to mediapersons ;). This is where the role of geometry comes in, hidden sometimes in algebra, but ever-present in any situation which calls for a guess. One of the fundamental beliefs of the natural sciences is that nature follows a pattern, and that can be discovered and propounded as laws. Therefore, given a set of data or readings regarding any phenomenon, they would try and see if they can fit some known functions (or shapes, to the mathematically challenged), which can help them find intermediate values, and also help predict future outcomes. If the model (that is what fitting a function would be called) is bad, chuck and find another more suitable function, and hopefully get more readings to confirm their hypothesis. Thus, if anyone hears a scientist talking about ‘exponentially increasing population’ or ‘linearly changing system’, you can be sure that the above is what they have probably done.

Polynomials, exponentials, sines, cosines, are among the darlings of the geometric modelling community, and you will be hearing more about them here, hopefully with as little math and as many pretty curves as possible :)