Systems and "Systems theory"

     "Think often of the bond that unites all things in the universe,
     and their dependence on one another. All are, as it were,
     interwoven, and in consequence linked in mutual affection . . ."
     Marcus Aurelius (Meditations VI, 38)

At Outback Software, "Systems" - which is to say, the way "components" are assembled (or assemble themselves) into larger behavioral entities - is a way of thinking, perhaps a way of life.

"Systems," with a capital S, implies the existence of general rules or patterns that apply to many, or all, types of systems, from mechanical to biological. In particular, these rules ought to apply to software systems.

Software systems, as models of other systems, are in one sense simpler and less interesting than what they model. (Though networks of software systems, like the Internet, may exhibit complex "emergent behavior"; but that's a different kind of system.) However, software system development projects, being human endeavors, exhibit all the psychological, social, economic and political complexities of human affairs, and as such they are very interesting to us. Understanding the systemic complexities of the development process itself often makes the difference between success and failure in projects.

For the management of practical projects, it is usually sufficient to have some intuitive sense of "systems" and their "general laws." But do these notions stand up to rigorous analysis? Is there a science of systems?


"General System Theory"

What does one call a science of systems?

"Systematics" comes to mind, but it's already taken for another, different meaning: the classification and naming of biological organisms (in the tradition of Carolus Linnaeus). From the Systematics Association of New Zealand: "Systematics is the science of naming species and of recovering the relationships between species. In short, systematics describes and analyses Earth's biodiversity. Systematics is a combination of taxonomy and phylogenetic analysis."

"Systemantics" is a humorous coinage by John Gall, author of Systemantics: How Systems Work and Especially How They Fail and Systemantics: The Underground Text of Systems Lore. These books are excellent compendiums of folk wisdom about systems, but don't fall into the "rigorous" category.

The term that has entered the lexicon is "General System Theory," (or "General Systems Theory") due to Ludwig von Bertalanffy. Bertalanffy was a biologist; the desire to understand biological organisms and collectives has been a continual stimulus to the search for "general system" laws. This is not surprising, since the individual and collective behavior of organisms is the paradigm case of extremely complex, heterogeneous systems that nevertheless exhibit regularities of behavior. (Physicists early on adopted the approach of studying radically simplified models of complex systems, which worked so well within its domain that for centuries anything not fitting this paradigm was excluded from consideration. Biologists had no such easy out.)


L. von Bertalanffy

A couple of striking examples of the "general systems" theme in biology are D'arcy Wentworth Thompson, author of the celebrated book On Growth and Form (1917), which attempted to explain many biological phenomena in terms of general mathematical laws; and Alfred Lotka, whose Elements of physical biology was published in 1925. As far back as the Renaissance, Leonardo da Vinci had studied the general principle of spiral phyllotaxis in plants, and in the 17th century, Galileo wrote, in the Discorsi e dimostrazioni matematiche intorno a due nuove scienze (Dialogues Concerning Two New Sciences), a lucid exposition of the power laws which biological and mechanical structures must follow as they scale up in size.


D'Arcy Thompson

Bertalanffy was motivated, in part, by the trend towards reductionism and specialization in science. Against reductionism , he said "It is necessary to study not only parts and processes in isolation, but also to solve the decisive problems found in organization and order unifying them, resulting from dynamic interaction of parts, and making the behavoir of the parts different when studied in isolation or within the whole..." And against specialization, "Modern science is characterized by its ever-increasing specialization, necessitated by the enormous amount of data, the complexity of techniques and of theoretical structures within every field. Thus science is split into innumerable disciplines continually generating new subdisciplines. In consequence, the physicist, the biologist, the psychologist and the social scientist are, so to speak, encapusulated in their private universes, and it is difficult to get word from one cocoon to the other..."


Alfred Lotka

General system theory, then, is about "wholeness...The meaning of the somewhat mystical expression, 'The whole is more that the sum of its parts' is simply that constitutive characteristics are not explainable from the characteristics of the isolated parts. The characteristics of the complex, therefore, appear as 'new' or 'emergent'. . . We can also say: While we can conceive of a sum being composed gradually, a system as total of parts with its interrelations has to be conceived of as being composed instantly..." (This last statement presumably can't be taken literally, since organisms, let alone complex constructed systems such as a Swiss chronometer or a software application, are not "composed instantly." As a reference to how we think about systems, the point seems valid. But does it deter us from thinking about systems that evolve in space and time?)

This leads, finally, to this formulation: "There appear to exist general system laws which apply to any system of a particular type, irrespective of the particular properties of the systems and the elements involved. . . These considerations lead to the postulate of a new scientific discipline which we call general system theory. Its subject matter is formulation of principles that are valid for 'systems' in general, whatever the nature of the component elements and the relations or 'forces' between them."

The exact nature of these "general system laws" has been a subject of some controversy, and Bertalanffy was accused in some quarters of peddling pseudoscience. The modern study of complex systems seems to have vindicated at least the spirit of "General System Theory." Perhaps because of the negative connotations, the term "Systems Science" now seems to be superseding "General System Theory."

(All the Bertalanffy quotes are excerpts from General System Theory.)


Cybernetics and information theory

Around the time that Bertalanffy was writing about ". . . a general systems theory which deals with formal characteristics of systems," mathematicians such as Norbert Wiener and Claude Shannon were looking at the issue of information, the glue that connects parts of a system. Partly as a result of studying military problems such as antiaircraft target tracking, Wiener formulated the general notion of cybernetics (from the Greek work for "steersman") to describe the process of control guided by feedback through communication of information, which is critical to both living and non-living systems. His book on the subject, Cybernetics: Or Control and Communication in Animal and the Machine, has become a classic.


Norbert Wiener

Wiener, like Bertalanffy, was aware of the danger of excessive specialization in science. He credited the origin of the ideas in Cybernetics to his collaboration with a medical researcher, Arturo Rosenbleuth: "For many years Dr. Rosenbleuth and I had shared the conviction that the most fruitful areas for the growth of the sciences were those which had been neglected as a no-man's land between the various established fields."

Some of Wiener's ideas were anticipated in non-mathematical form by the physician and physiologist Walter Cannon. In his book, Wisdom of the Body, published in the early thirties, Cannon introduced the important term "homeostasis" to describe the process by which living organisms maintain a constant internal state in the face of environmental challenges. The general concept had been articulated earlier by the French scientist Claude Bernard in his studies of the maintenance of stability in the milieu interior : "Far from the higher animals being indifferent to their surroundings, they are on the contrary in close and intimate relation to it, so that their equilibrium is the result of compensation established as continually and as exactly as if by a very sensitive balance."

In addition to Shannon's well-known Mathematical Theory of Communication, his master's thesis, A Symbolic Analysis of Relay and Switching Circuits, introduced the use of Boolean algebra to analyse switching circuits - a critical insight for subsequent generations of computer engineers.

These developments added notions from the theories of stochastic processes, groups, linear operators, Boolean algebra, and combinatorics to the science of systems, expanding the toolkit far beyond the differential equations used by Thompson, Lotka and Bertalanffy.

Subsequent developments along these lines included neural networks, cellular automata, and agent models. These were inspired by biological and social systems, but introduced formal techniques that could be used for modeling a wide variety of system types.


Claude Bernard


Walter Cannon


Claude Shannon


Scale, abstraction level, and causality

Coming back to the question of what characterizes "general system laws," and what some examples might be . . . Through the 1960s and 70s, the IBM Systems Research Institute taught a curriculum to systems engineers that, pragmatically, provided a number of "general systems" techniques that were valid in the universe of heterogeneous computer systems: queueing theory, information theory, mathematical programming, simulation modeling, etc. These tools, and some of the "laws" derived from them, are also useful and valid in a much larger universe of systems. Since then, we have become interested in reliability analysis, which has similarly broad applicability. (Not only are reliability analysis models applicable to a broad range of mechanical and electronic systems - the things we commonly associate with "reliability" - but they apply to biological systems as well.)

On reflection, certain not-quite-formal characteristics seem to be associated with all these "general systems" disciplines:

Medium level of generality: It goes without saying that anything claiming to be a "general systems law" must be formulated at a high enough level of abstraction to be applicable to many systems as systems. (We specify "as systems" because, e.g., in some sense the laws of physics govern all systems, but not at the system level.) It is not so obvious that it must not be at a "too high" level of abstraction. The mark of success in science is to provide a formal law expressed in mathematics, so in some sense we could say that the ultimate corpus of general systems laws is identical with mathematics. But no one would accept this - even if the mathematical content of a law predicting failure in mechanical systems is isomorphic to the mathematical content of a law predicting survival of cancer patients, their generalization must somehow specify the characteristics of the physically embodied systems to which it applies.

The law of not-so-large numbers: Classical physics does quite well where there is a very small number of entities (e.g., the two-body problem in mechanics), or a very large number (e.g., statistical thermodynamics). Most systems in the world, at the system level, contain a number of entities that falls somewhere between those extremes. Even where we technically require "the law of large numbers," (e.g., in many queueing theory results), the technical criterion is not strictly met.

Non-isolation: Likewise, classical physics likes to deal with systems that are isolated from interference by irrelevant factors. The systems for which we are interested in finding "general systems laws" are rarely, if ever, isolated in this sense. In fact, behavior governed by outside interference, or behavior at the boundary of a system and its environment, is often precisely what we are trying to explain. As John Muir famously said, "Whenever we try to isolate anything in the universe, we find that it's hitched up to everything else."

Scaling: Most of the interesting systems in this context come in varying sizes: from mouse to elephant, from family to nation, from single-user computer to the Internet, etc. Anything worthy of being a "general systems law" must either be applicable at all scales, or have a component to account for scaling.

Spiral causality and iterative laws: In contrast to the "linear" causality of paradigmatic examples in physics (like the collision of two billiard balls in infinite space), "general systems" typically exhibit a pattern where an effect produced by a on b at time t0 causes b to affect a at time t1. Thus laws explaining such systems must be implicitly or explicitly iterated in order to produce results. The Volterra-Lotka predator-prey equation is a classic example from biology.

Spiral causality is sometimes misnamed "circular causality," implying that it violates the time ordering of cause and effect. Though some interpretations of quantum-mechanical "paradoxes" like Einstein-Podolsky-Rosen and "Schrödinger's cat" may imply violation of time order, these are - hopefully - not relevant at the systems level. If the terms "circular causality" and "causal loop" are used, we must add the caveat that the loop is in space, not time.

Complex causality: Aside from spiral causality, there is another source of complexity. Compared to the simple "linear" causal chains of paradigmatic classical physics (a causes b causes c causes . . .), causal chains in "general systems" tend to be complex graphs in which each effect has many causes, and each cause has many effects. (The combination of this and spiral causality is sometimes called "non-linear causality," but that terminology invites confusion with the use of the terms "linear" and "non-linear" as applied to equations - Though "non-linear" causal graphs are no doubt more likely to require modelling by non-linear equations.)

"Goal-directed" behavior controlled by feedback: This is a higher-level phenomenon built on complex causal graphs. Without sliding into teleology, we can say that many systems of interest are characterized by certain external parameters which the system "acts" to keep within a "goal" range. This is accomplished by causal chains called feedback loops - typically a combination of negative feedback loops, which act to supress variation (maintain homeostasis), and positive feedback loops, acting to amplify variation. (Excessive positive feedback is often a cause of pathology; excessive negative feedback can also be pathological, as in clinical depression, or settling on a local optimum near a global optimum in mathematical programming.)

Sensitivity to initial conditions: In spite of goal-directed behavior, most interesting systems have many salient behavioral parameters that evolve over time, and undergo large changes in response to arbitrarily small changes in initial conditions. (This, and the notion of goal directedness, can be described using terms from the mathematical theory of dynamical systems, like "unstable equilibrium," "attractor," etc. For systems not amenable to mathematical analysis, this must be considered somewhat metaphorical.)

The paragraphs above are just a "laundry list" of observed characteristics, not a taxonomy or a complete, orthogonal set of attributes. It is not clear that "general systems laws" can ever be completely characterized, and certainly not soon, given the amount of active research in systems and related fields such as complexity. There are also other cuts at this, such as the distinction between behaviors that can be modeled with closed-form mathematical equations, and those that require iterative numerical solution, simulation, etc.

Despite what was said above about physics, even the classical physics of Newton provides abundant examples of some of the characteristics of "general systems" mentioned above. Gravitational attraction is no doubt the purest and simplest example of "everything hitched up to everything else," and the first to have been studied with the formal apparatus of mathematics. Even very simple systems involving just a few bodies, such as linked pendulums, exhibit chaotic behavior whose prediction is an intractable problem for Newtonian physics.

Physicists, however, quite naturally chose to occupy themselves (very profitably for them and for the rest of us) with problems they could solve. Having pretty much run out of "simple system" problems, physicists are now active contributors to many things that might fall under the rubric "General Systems."


Links

General systems theory


Complex adaptive systems (CAS)


Networks, natural and manufactured


Systems engineering, design and architecture


Analytic and simulation modeling


Other interesting stuff