Normal Accidents: Living with High-Risk Technologies

Normal Accidents: Living with High-Risk Technologies (1999 ed)


Charles Perrow (Scholar search): An organizational theorist, he is the author of The Radical Attack on Business, Organizational Analysis: A Sociological View, Complex Organizations: A Critical Essay, and Normal Accidents: Living with High Risk Technologies. His interests include the development of bureaucracy in the 19th Century; the radical movements of the 1960s; Marxian theories of industrialization and of contemporary crises; accidents in such high risk systems as nuclear plants, air transport, DNA research and chemical plants; protecting the nation’s critical infrastructure; the prospects for democratic work organizations; and the origins of U.S. capitalism.


This book describes a type of catastrophic accident that emerges in particular types of complex systems called, “Normal”, because they appear to be inevitable, yet unpredictable as to the specifics.

Normal accidents occur in systems that have three common characteristics

  1. They are tightly coupled. Behavior in one part rapidly affects other parts.
  2. They are densely connected. One part may affect many other parts
  3. Their process are opaque in that they cannot be observed directly and as such behavior must be inferred.

In addition, systems that involve the transformation of their input, such as nuclear power, petrochemical plants, and DNA manipulation are more likely to be tightly coupled, densely connected, and opaque and as such more prone to catastrophe.

A sub-theme is that these systems are always “accidents waiting to happen”, and as such an obvious cause can be found in a post-mortem. However, systems that were inspected months before an accident are often found to be in good working order. Looking more closely, the expectation of finding a problem after a catastrophe makes finding the problem more likely. Looking at a running system through the lens of an imaginary catastrophe should be an effective way to tease out potential problems.

My more theoretical thoughts

Like Meltdown, which was based in part on this book, the systems described here look like networks where the degree and stiffness of connections, combined with an overall velocity of the networked system through belief space (described here as production pressure). That implies that modelling them using something like Graph Laplacians might make sense, though the equations that describe the “weight” of the nodes, the “stiffness” of the edges, and the inertial characteristics of the system as a whole are unclear.



  • Rather, I will dwell upon characteristics of high-risk technologies that suggest that no matter how effective conventional safety devices are, there is a form of accident that is inevitable. (page 3)
  • Most high-risk systems have some special characteristics, beyond their toxic or explosive or genetic dangers, that make accidents in them inevitable, even “normal.” This has to do with the way failures can interact and the way the system is tied together. It is possible to analyze these special characteristics and in doing so gain a much better understanding of why accidents occur in these systems, and why they always will. If we know that, then we are in a better position to argue that certain technologies should be abandoned, and others, which we cannot abandon because we have built much of our society around them, should be modified. (Page 4)
  • No one dreamed that when X failed, Y would also be out of order and the two failures would interact so as to both start a fire and silence the fire alarm. (Page 4)
  • This interacting tendency is a characteristic of a system, not of a part or an operator; we will call it the “interactive complexity” of the system. (Page 4)
  • For some systems that have this kind of complexity, such as universities or research and development labs, the accident will not spread and be serious because there is a lot of slack available, and time to spare, and other ways to get things done. But suppose the system is also “tightly coupled,” that is, processes happen very fast and can’t be turned off, the failed parts cannot be isolated from other parts, or there is no other way to keep the production going safely. Then recovery from the initial disturbance is not possible; it will spread quickly and irretrievably for at least some time. Indeed, operator action or the safety systems may make it worse, since for a time it is not known what the problem really is. (Page 5)
  • these systems require organizational structures that have large internal contradictions, and technological fixes that only increase interactive complexity and tighten the coupling; they become still more prone to certain kinds of accidents. (Page 5)
  • If interactive complexity and tight coupling—system characteristics—inevitably will produce an accident, I believe we are justified in calling it a normal accident, or a system accident. The odd term normal accident is meant to signal that, given the system characteristics, multiple and unexpected interactions of failures are inevitable. (Page 5)
  • The cause of the accident is to be found in the complexity of the system. That is, each of the failures—design, equipment, operators, procedures, or environment—was trivial by itself. Such failures are expected to occur since nothing is perfect, and we normally take little notice of them. (Page 7)
  • Most of the time we don’t notice the inherent coupling in our world, because most of the time there are no failures, or the failures that occur do not interact. But all of a sudden, things that we did not realize could be linked (buses and generators, coffee and a loaned key) became linked. The system is suddenly more tightly coupled than we had realized. (Page 8)
    • Comment: This is the central point in the stampeding of things like self-driving cars. The system is more tightly coupled that we realize
  • It is normal in the sense that it is an inherent property of the system to occasionally experience this interaction. Three Mile Island was such a normal or system accident, and so were countless others that we shall examine in this book. We have such accidents because we have built an industrial society that has some parts, like industrial plants or military adventures, that have highly interactive and tightly coupled units. Unfortunately, some of these have high potential for catastrophic accidents. (Page 8)
  • This dependence is known as tight coupling. On the other hand, events in a system can occur independently as we noted with the failure of the generator and forgetting the keys. These are loosely coupled events, because although at this time they were both involved in the same production sequence, one was not caused by the other. (Page 8)
  • In complex industrial, space, and military systems, the normal accident generally (not always) means that the interactions are not only unexpected, but are incomprehensible for some critical period of time. (Page 8)
  • But if, as we shall see time and time again, the operator is confronted by unexpected and usually mysterious interactions among failures, saying that he or she should have zigged instead of zagged is possible only after the fact. Before the accident no one could know what was going on and what should have been done. Sometimes the errors are bizarre. We will encounter “noncollision course collisions,” for example, where ships that were about to pass in the night suddenly turn and ram each other. But careful inquiry suggests that the mariners had quite reasonable explanations for their actions; it is just that the interaction of small failures led them to construct quite erroneous worlds in their minds, and in this case these conflicting images led to collision. (Page 9)
  • Small beginnings all too often cause great events when the system uses a “transformation” process rather than an additive or fabricating one. Where chemical reactions, high temperature and pressure, or air, vapor, or water turbulence is involved, we cannot see what is going on or even, at times, understand the principles. In many transformation systems we generally know what works, but sometimes do not know why. These systems are particularly vulnerable to small failures that “propagate” unexpectedly, because of complexity and tight coupling. (Page 10)
    • Comment: This is a surprisingly apt description of a deep neural network and a good explanation of why they are risky.
  • High-risk systems have a double penalty: because normal accidents stem from the mysterious interaction of failures, those closest to the system, the operators, have to be able to take independent and sometimes quite creative action. But because these systems are so tightly coupled, control of operators must be centralized because there is little time to check everything out and be aware of what another part of the system is doing. An operator can’t just do her own thing; tight coupling means tightly prescribed steps and invariant sequences that cannot be changed. But systems cannot be both decentralized and centralized at the same time; (Page 10)
    • Comment: And the pressures to make systems efficient leads to tighter coupling and regards diversity as a penalty, which it almost always is in the short term. Diversity prevents catastrophes, but inhibits rewards/profits.
  • when we move away from the individual dam or mine and take into account the larger system in which they exist, we find the “eco-system accident,” an interaction of systems that were thought to be independent but are not because of the larger ecology. (Page 14)
    • Comment: I think financial crashes may also be manifestations of an eco-system accident. Credit-default swaps are similar to DDT in that they concentrate risk and move it up the food chain.

Chapter 1: Normal Accident at Three Mile Island

  • We are now, incredibly enough, only thirteen seconds into the “transient,” as engineers call it. (It is not a perversely optimistic term meaning something quite temporary or transient, but rather it means a rapid change in some parameter, in this case, temperature.) In these few seconds there was a false signal causing the condensate pumps to fail, two valves for emergency cooling out of position and the indicator obscured, a PORV that failed to reseat, and a failed indicator of its position. The operators could have been aware of none of these. (Page 21)
    • Comment: This could be equivalent to a navigation system routing a fleet of cars through a recent disaster, where the road is lethal, but the information about that has not entered the system. (examples)
  • What they didn’t know, and couldn’t know, was that with the PORV open and the two feedwater valves blocked, preventing the removal of residual heat, they already had a LOCA, but not from a pipe break. The rise in pressure in the pressurizer was probably due to the steam voids rapidly forming because the core was close to becoming uncovered. They thought they were avoiding a LOCA when they were in one and were making it worse. With the PORV stuck open, the danger of going solid in the pressurizer was reduced because the open valve would provide some relief. But no one knew it was open. (Page 26)
  • We will encounter this man’s dilemma a few more times in this book; it goes to the core of a common organizational problem. In the face of uncertainty, we must, of course, make a judgment, even if only a tentative and temporary one. Making a judgment means we create a “mental model” or an expected universe. (Page 27)
    • Comment: This is belief space, and these models are often created as stories with sequences. Having a headache can create a story of something temporary that will be fixed with a pill, or something serious that requires a trip to the hospital. The type of story that gets told depends very much on the location in the belief space the user is in. Someone who is worried about a stroke is much more likely to think that a headache is serious and build a narrative around that.
  • Despite the fact that this is no proper test of the appropriateness of alternative B rather than A, it serves to “confirm” your decision. In so believing, you are actually creating a world that is congruent with your interpretation, even though it may be the wrong world. It may be too late before you find that out. (Page 27)
  • These are not expected sequences in a production or safety system; they are multiple failures that interacted in an incomprehensible manner (Page 31)
    • Comment: This could be incomprehensible for any intelligent system, not just human.

Chapter 2: Nuclear Power as a High-Risk System: Why We Have Not Had More TMIs–But Will Soon

  • Shoddy construction and inadvertent errors, intimidation and actual deception—these are part and parcel of industrial life. No industry is without these problems, just as no valve can be made failure-proof. Normally, the consequences are not catastrophic. They may be, however, if you build systems with catastrophic potential. (Page 37)
  • A bit more revealing is another discussion of seven “criticality” accidents. If plutonium, which is exceedingly volatile and hard to machine or handle, experiences the proper conditions, it can attain a self-sustaining fission chain reaction. Criticality depends upon the quantity of the plutonium, the size, shape, and material of the vessel that holds it, the nature of any solvents or dilutants, and even adjacent material, which may reflect neutrons back into the plutonium. It is apparently hard to know when these conditions might be just right. (Page 55)

Chapter 3: Complexity, Coupling, and Catastrophe

  • But there are also degrees of disturbance to systems. The rally will not be disturbed in any perceptible way if I show up with a scratch on my ’57 De Soto, or if, ashamed of the state of my car, I do not go at all. But my system might be greatly disturbed if I stayed home rather than meeting or impressing people. The degree of disturbance, then, is related to what we define as the system. If the rally is the system under analysis, there is no accident. If that part of my life concerned with custom cars is the system, there is an accident. A steam generator tube failure in a nuclear plant can hardly be anything other than an accident for that plant and for that utility. Yet it may or may not have an appreciable effect upon the nuclear power “system” in the United States. (page 64)
  • Most of the work concerned with safety and accidents deals, rightly enough, with what I call first-party victims, and to some extent second-party victims. But in this book we are concerned with third- and fourth-party victims. Briefly, first-party victims are the operators; second-party victims are nonoperating personnel or system users such as passengers on a ship; third-party victims are innocent bystanders; fourth-party victims are fetuses and future generations. Generally, as we move from operators to future generations, the number of persons involved rises geometrically, risky activities are less well compensated, and the risks taken are increasingly unknown ones. (Page 67)
  • The following list presents these and adds the definition of the two types of accidents: component failure accidents and system accidents, which we will now take up. (Page 70)
    • Systems are divided into four levels of increasing aggregation: units, parts, subsystems, and system.
    • Incidents involved damage to or failures of parts or a unit only, even though the failure may stop the output of the system or affect it to the extent that it must be stopped.
    • Accidents involve damage to subsystems or the system as a whole, stopping the intended output or affecting it to the extent that it must be halted promptly.
    • Component failure accidents involve one or more component failures (part, unit, or subsystem) that are linked in an anticipated sequence.
    • System accidents involve the unanticipated interaction of multiple failures.
  • A system accident, in our definition, must have multiple failures, and they are likely to be in reasonably independent units or subsystems. But system accidents, as with all accidents, start with a component failure, most commonly the failure of a part, say a valve or an operator error. It is not the source of the accident that distinguishes the two types, since both start with component failures; it is the presence or not of multiple failures that interact in unanticipated ways. (Page 71)
  • Incidents are overwhelmingly the most common untoward system events. Accidents are far less frequent. Among accidents, component failure accidents are far more frequent than system accidents. I have no reliable way to estimate these frequencies. For the systems analyzed in this book the richest body of data comes from the safety-related failures that nuclear plants in the United States are required to report. Roughly 3,000 Licensee Event Reports are filed each year by the 70 or so plants. Based upon the literature discussing these reports, I estimate 300 of the 3,000 events might be called accidents; 15 to 30 of these might be system accidents. (Page 71)
    • Comment: This looks like a power-law relationship. If true, then it would be a nice way of indicating the frequencies of rarer types of accidents based on the more common types.
  • The notion of baffling interactions is increasingly familiar to all of us. It characterizes our social and political world as well as our technological and industrial world. As systems grow in size and in the number of diverse functions they serve, and are built to function in ever more hostile environments, increasing their ties to other systems, they experience more and more incomprehensible or unexpected interactions. They become more vulnerable to unavoidable system accidents. (Page 72)
    • Comment: The central premise of Stampede Theory: This emergent property of tightly coupled systems is not specific to a technology.
  • These are “linear” interactions: production is carried out through a series or sequence of steps laid out in a line. It doesn’t matter much whether there are 1,000 or 1,000,000 parts in the line. It is easy to spot the failure and we know what its effect will be on the adjacent stations. There will be product accumulating upstream and incomplete product going out downstream of the failure point. Most of our planned life is organized that way. (Page 72)
    • Comment: A linear system is a list of transformations. I think that is also the general form of a story, but not a game or a map. This means that linear systems in physical or belief space are inherently understandable. The trick in belief space is to determine what is important at what scale to achieve meaningful understanding.
  • But what if parts, or units, or subsystems (that is, components) serve multiple functions? For example, a heater might both heat the gas in tank A and also be used as a heat exchanger to absorb excess heat from a chemical reactor. If the heater fails, tank A will be too cool for the recombination of gas molecules expected, and at the same time, the chemical reactor will overheat as the excess heat fails to be absorbed. This is a good design for a heater, because it saves energy. But the interactions are no longer linear. The heater has what engineers call a “common-mode” function—it services two other components, and if it fails, both of those “modes” (heating the tank, cooling the reactor) fail. This begins to get more complex. (Page 73)
  • I will refer to these kinds of interactions as complex interactions, suggesting that there are branching paths, feedback loops, jumps from one linear sequence to another because of proximity and certain other features we will explore shortly. The connections are not only adjacent, serial ones, but can multiply as other parts or units or subsystems are reached. (Page 75)
    • Comment: Modeled by graph Laplacians
  • To summarize our work so far: Linear interactions are those interactions of one component in the DEPOSE system (Design, Equipment, Procedures, Operators, Supplies and materials, and Environment) with one or more components that precede or follow it immediately in the sequence of production. Complex interactions are those in which one component can interact with one or more other components outside of the normal production sequence, either by design or not by design. (Page 77)
  • These problems exist in all industrial and transportation systems, but they are greatly magnified in systems with many complex interactions. This is because interactions, caused by proximity, common mode connections, or unfamiliar or unintended feedback loops, require many more probes of system conditions, and many more alterations of the conditions. Much more is simply invisible to the controller. The events go on inside vessels, or inside airplane wings, or in the spacecraft’s service module, or inside computers. Complex systems tend to have elaborate control centers not because they make life easier for the operators, saving steps or time, nor because there is necessarily more machinery to control, but because components must interact in more than linear, sequential ways, and therefore may interact in unexpected ways. (Page 82)
  • In complex systems, where not even a tip of an iceberg is visible, the communication must be exact, the dial correct, the switch position obvious, the reading direct and “on-line.” (Page 84)
    • Comment: True for any stiff, dense system
  • The problem of indirect or inferential information sources is compounded by the lack of redundancy available to complex systems. If we stopped to notice, we would observe that our daily life is full of missed or misunderstood signals and faulty information. A great deal of our speech is devoted to redundancy—saying the same thing over and over, or repeating it in a slightly different way. We know from experience that the person we are talking to may be in a different cognitive framework, framing our remarks to “hear” that which he expects to hear, not what he is being told. The listener suppresses such words as “not” or “no” because he doesn’t expect to hear them. Indeed, he does not “hear” them, in the literal sense of processing in his brain the sounds that enter the ear. All sorts of trivial misunderstandings, and some quite serious ones, occur in normal conversation. We should not be surprised, then, if ambiguous or indirect information sources in complex systems are subject to misinterpretation. (Page 84)
  • accidents continue to plague transformation processes that are fifty years old. These are processes that can be described, but not really understood. They were often discovered through trial and error, and what passes for understanding is really only a description of something that works. (Page 85)
    • Comment: This is also true with machine learning. We don’t know why these algorithms are so effective and what their limits are. Or even their points of greatest sensitivity.
  • To summarize, complex systems are characterized by (Page 86):
    • Proximity of parts or units that are not in a production sequence;
    • many common mode connections between components (parts, units, or subsystems) not in a production sequence;
    • unfamiliar or unintended feedback loops;
    • many control parameters with potential interactions;
    • indirect or inferential information sources; and
    • limited understanding of some processes.
  • Linear systems lack the common-mode connections that require proximity. It is also a design criterion to separate various stages of production for sheer ease of maintenance access or replacement of equipment. Linear systems not only have spatial segregation of separate phases of production, but within production sequences the links are few and sequential, allowing damaged components to be pulled out with minimal disturbance to the rest of the system. (Page 86)
  • Though I don’t want to claim a vast difference between employees in complex and linear systems, the latter appear to have fewer specialized and esoteric skills, allowing more awareness of interdependencies if they appear. The welder in a nuclear plant is more specialized (and specially rated), and presumably more isolated from other personnel, than the welder in a fabrication plant. Specialized personnel tend not to bridge the wide range of possible interactions; generalists, rather than specialists, are perhaps more likely to see unexpected connections and cope with them. (Page 87)
    • Comment: Linear systems are lower dimension. In fact, what is called complex here is actually high dimension sometimes and truly complex at others. Feedback Loops are a sign of true complexity
  • Complex-Linear(page 88)
  • Loosely coupled systems, whether for good or ill, can incorporate shocks and failures and pressures for change without destabilization. Tightly coupled systems will respond more quickly to these perturbations, but the response may be disastrous. Both types of systems have their virtues and vices. (Page 92)
  • Loosely coupled systems are said to have “equifinality”—many ways to skin the cat; tightly coupled ones have “unifinality.” (Page 94)
    • Comment: A tightly coupled system can only behave as a single individual, made up of tightly connected parts (a body). Loosely coupled systems behavior is more like populations. In humans (most animals?), the need for novelty provides velocity to the members of the population and is also a driver for coordination among systems that would otherwise be much more loosely coupled?
  • Tightly coupled systems have little slack. Quantities must be precise; resources cannot be substituted for one another; wasted supplies may overload the process; failed equipment entails a shutdown because the temporary substitution of other equipment is not possible. No organization makes a virtue out of wasting supplies or equipment, but some can do so without bringing the system down or damaging it. In loosely coupled systems, supplies and equipment and human power can be wasted without great cost to the system. Something can be done twice if it is not correct the first time; one can temporarily get by with lower quality in supplies or products in the production line. The lower quality goods may have to be rejected in the end, but the technical system is not damaged in the meantime. (Page 94)
  • In tightly coupled systems the buffers and redundancies and substitutions must be designed in; they must be thought of in advance. In loosely coupled systems there is a better chance that expedient, spur-of-the-moment buffers and redundancies and substitutions can be found, even though they were not planned ahead of time. (page 95)
    • Comment: This is why diversity matters, it loosens the coupling. The question is to determine a) How much friction is needed for a given system, and b) what is the best way to inject diversity or other forms of friction.
  • Tightly coupled systems are not completely devoid of unplanned safety devices. In two of the most famous nuclear plant accidents, Browns Ferry and TMI, imaginative jury-rigging was possible and operators were able to save the systems through fortuitous means. At TMI two pumps were put into service to keep the coolant circulating, even though neither was designed for core cooling. Subjected to intense radiation they were not designed to survive; one of them failed rather quickly, but the other kept going for days, until natural circulation could be established. Something more complex but similar took place at Browns Ferry. The industry claimed that the recovery proved that the safety features worked; but the designed-in ones did not work. (Page 95)
  • The placement of systems is based entirely on subjective judgments on my part; at present there is no reliable way to measure these two variables, interaction and coupling. (Page 96)
    • Comment: Graph Laplacians? The system is a network. That network can be tiny or huge. Characterizing The nodes and edges at scale is difficult.

Chapter 4: Petrochemical Plants

  • Our discussion of nuclear power plants should make the following observations familiar enough. There was organizational ineptitude: they were knowingly short of engineering talent, and the chief engineer had left; there was a hasty decision on the by-pass, a failure to get expert advice, and most probably, strong production pressures. But as was noted in the case of TMI and other nuclear plants, and as will be apparent from other chapters, this is the normal condition for organizations; we should congratulate ourselves when they manage to run close to expectations. Had the pipe held out a short time longer until the reactor was repaired, and a new chief engineer been hired, and had a governmental inquiry board then come around, they could have concluded it was a well-run, safe plant. Once there is an accident, one looks for and easily finds the great causes for the great event. There were unheeded warning signals—the by-pass pipe had been noted to move up and down slightly during operation, surely an irregular bit of behavior; and there were unexplained anomalies in pressure and temperature and hydrogen consumption. But these were warnings only in retrospect. In these complex systems, minor warnings are probably always available for recall once there is an occasion, but if we shut down for every little thing… (Page 112)
  • The report also faults the operators for not closely watching the indicators that monitored the flow of synthesis gas through the heater, which would have disclosed that something was wrong. But the report goes on to acknowledge that “the flow indicators were considered unreliable because there was hardly any indication of flow during both normal operations and in start-up conditions, especially when starting up both converters simultaneously,” as they were now doing. (Note this trivial interdependency of the two converters and their effect upon flow.) The operators, then, were blamed for not monitoring a flow that was so faint it could not be reliably measured. It hardly mattered in any case, since both flow indicators had been set incorrectly, and furthermore the alarms for them, indicating low flow, “had been disarmed since they caused nuisance problems during normal operations.”26 Small wonder, after all this, that the flow indicators “were not closely monitored.” (Page 116)
  • High temperature alarms would occur, but the operators learned to ignore them. (Page 118)
  • After the paper was presented, a discussant noted it is not the newness of the plant that is the problem. Even in the older plants, he said, “We struggle to control it -Runaways will take place and control by these caps is not the answer…. The way it is now we are in difficulties and I don’t think anybody is sophisticated enough to operate the plant safely.”[33] The problems in this mature, but increasingly sophisticated and high volume industry, appear to lie in the nature of the highly interactive, very tightly coupled system itself, not in any design or equipment deficiencies that humans might overcome. (Page 120)

Chapter 5: Aircraft and Airways

  • Thirty-one of the first forty pilots were killed in action, trying to meet the schedules for business and government mail. It may be the first nonmilitary example of a phenomenon that will concern us in this chapter: production pressures in this high-risk system. Though nothing comparable exists today in commercial aviation, such pressures, are, as we shall see, far from negligible. (Page 125)
    • Comment: Production pressure behaves as a global stiffener.
  • Dependence upon multiple function units or subsystems was reduced by the segregation of the traffic (as well as by the use of transponders). More corridors were set up and restricted to certain kinds of flights. Small aircraft with low speeds (and without instrument flying equipment) were excluded from the altitudes where the fast jets fly (though controllers relent on this point, as we have just seen). Military flights were restricted to certain areas; parachutists were controlled. In this way, the system was made more linear. Of course, the density in any one corridor probably also has increased, offsetting the gain to some or even to a great degree. But if we could control for density, we would expect to find a decrease in unexpected interactions. (Page 159)
  • Tight coupling reduces the ability to recover from small failures before they expand into large ones. Loose coupling allows recovery. In ATC processing delays are possible; aircraft are highly maneuverable and in three-dimensional space, so an airplane can be told to hold a pattern, to change course, slow down, speed up, or whatever. The sequence of landing or takeoff or insertion into a long-distance corridor is not invariant, though flexibility here certainly has its limits. The creation of more corridors reduced the coupling as well as the complexity of this system. Time constraints are still tight; the system is not loosely coupled, only moderately tightly coupled. But aircraft are maneuverable. They are also quite small. Near misses generally concern spaces of 200 feet to one mile. Those near misses reported to be under 100 feet are exceedingly rare and the proximity may be exaggerated. Even with 100 feet, though, there are 99 spare feet. If one tried, it would be hard to make two aircraft collide. In other high-risk systems it is comparatively easy to produce an explosion, or to defeat key safety systems and produce a core melt. (Page 160)
  • But in contrast to nuclear power plants and chemical plants (and recombinant DNA research), the system is not a transformation system, with hidden and poorly understood interactions that respond to indirect controls with indirect indicators. (An exception occurs in encountering buffet boundaries.) (Page 167)

Chapter 6: Marine Accidents

  • The problem, it seems to me, lies in the type of system that exists. I will call it an “error-inducing” system; the configuration of its many components induces errors and defeats attempts at error reduction. (Page 172)
    • Comment: I think social networks tend towards “error-inducing”
  • The notion of an error-inducing system itself is derived from the complexity and coupling concepts. It sees some aspects as too loosely coupled (the insurance subsystem and shippers), others as too tightly coupled (shipboard organization); some aspects as too linear (shipboard organizations again, which are highly centralized and routinized), others as too complexly interactive (supertankers, and also the intricate interactions among marine investigations, courts, insurance agencies, and shippers). (Page 175)
  • One major example, that of what I will call “noncollision course collisions,” will raise a problem we have encountered before, the social construction of reality, or building cognitive models of ambiguous situations. Why do two ships that would have passed in the night suddenly turn on one another and collide? (Page 176)
  • The truckers do not want to use low gears and crawl down the other side of the Donner Summit, they want to go as fast as the turns and the highway patrol will allow them. Granted there are irresponsible drivers (irresponsible to themselves and their families as well as to third-party victims), just as all of us are irresponsible at times. But the work truckers do puts even irresponsible drivers in a situation where irresponsibility will have graver consequences than it does for most of us. Again, it is the system that must be analyzed, not the individuals. How should this system be designed to reduce the probability or limit the consequences of situations where irresponsibility can have an effect? (Page 180)
    • Comment: My sense is that this is almost always a user-interface problem, where the interests of the larger population need to be factored in while still providing meaningful freedom (of choice?) to the individual.
  • The encouragement of risk induces the owners and operators of the system to discard the elements of linearity and loose coupling that do exist, and to increase complexity and tight coupling. To some degree this discarding of intrinsic safety features occurs in all systems, but it appears to me to be far more prevalent in the marine system than in, say, nuclear power production and chemical production. I think the difference lies in the technological and environmental aspects of the system (fewer fixes available and a more severe environment), in its social organization (authority structure), and its catastrophic potential (which is less in the marine case, thus inviting less public intervention). (Page 187)
    • Comment: Each one of these components is a different network of weights connecting the same nodes:
      • Improvisational repairability
      • Environment
      • Social Organization
      • Catastrophic potential
  • We are led back to production pressures again. Even if CAS and radiotelephone communications (and inertial guidance systems, et cetera) would appear to reduce accidents in these studies, they also appear to increase speed and risk-taking, because the accident rate is growing steadily. Production pressures defeat the safety ends of safety devices and increase the pressures to use the devices to reduce operating expenses by going faster, or straighter; this makes the maritime system as a whole more complex (proximity, limited understanding) and tightly coupled (time dependent functions, limited slack). (Page 206)
    • Comment: This is a really important externality that will have a huge impact in robotic systems where there is no need to consider another human. This could easily lead to a runaway condition that is completely invisible until too late. Sort of a more industrial, less dramatic SkyNet example.
  • When we do dumb things in our car occasionally, we get an insight into how deck officers might do the same. Why do we, as drivers, or deck officers on ships, zig when we should have zagged, even when we are attentive and can see? I don’t know the many answers, but the following material will suggest that we construct an expected world because we can’t handle the complexity of the present one, and then process the information that fits the expected world, and find reasons to exclude the information that might contradict it. (Page 213)
    • Comment: Is there neurological support for this?
  • As drivers, we all would probably admit that at times we took unnecessary risks; but what we say to ourselves and others is, “I don’t know why; it was silly, stupid of me.” We generally do not do it because it was exciting. Finally, we cannot rule out exhaustion, or inebriation. Both exist. But neither are mentioned in the accident reports, and more important, from my own experience as a driver, skier, sailor, and climber, I know that I do inexplicable things when I am neither exhausted nor inebriated. So, in conclusion, I am arguing that constructing an expected world, while it begs many questions and leaves many things unexplained, at least challenges the easy explanations such as stupidity, inattention, risk taking, and inexperience. (Page 213)
    • Comment: We do inexplicable things online as well, based on constructed, social reality.
  • But what strikes me about this event is how well it exemplifies the easy way in which we can construct an interpretation of an ambiguous situation, process new information in the light of that interpretation—thus making the situation conform to our expectations—and, when distracted by other duties, make a last minute “correction” that fits with the private reality that no one else shares. (Page 217)
    • Comment: This is why we can have so many histories. They are almost as malleable as fiction. E.g. Rashamon effect
  • Why would a ship in a safe passing situation suddenly turn and be impaled by a cargo ship four times its length? For the same reasons the operators of the TMI plant cut back on high pressure injection and uncovered the core. Confronted with ambiguous signals, the safest reality was constructed. In the above marine case this view of reality assumed that the other ship was not a head-on collision threat. (Page 217)
    • Comment: I think that the constructed reality is the one that is most aligned with previous states that takes the least computation to construct. “Things will be fine” is always less computation than planning for catastrophe. And it’s usually the right answer. So why do some people spend their time exploring the space of potential catastrophes? I think this is explore/exploit management at the population level.
  • As complicated as nuclear power and chemical plants are, the complexities are contained within the hardware and the human-machine interface. We may not grasp the functions of superdeheaters, but we know where they are and that they must function on call. In the aircraft and the airways systems, we let more of the environment in. It complicates the situation a bit. The operating envelope of air becomes a problem, as do thunderclouds, windshears, and white-outs. (Page 229)
    • Comment: As long as there are no problems, nuclear reactors are a manufactured social construct, where the environmental reality (the core) is deeply hidden and ignorable. The weather is present and unavoidable. The environment helps to ground the aircraft systems.
  • Marine transport appears to be an error-inducing system, where perverse interconnections defeat safety goals as well as operating efficiencies. Technological improvements did increase output but probably have helped increase accidents; with radar, the ship can go faster; when two ships have radar, they are even more likely to collide. The equivalent of CDTI (cockpit display of traffic information), which is to be introduced into the airliners, already exists at sea, and is useful only a small percent of the time, and may sometimes be counterproductive. Anyway, despite the increasingly sophisticated equipment, captains still inexplicably turn at the last minute and ram each other. We hypothesized that they built perfectly reasonable mental models of world, which work almost all the time, but occasionally turn out to be almost an inversion of what really exists. The authoritarian structure aboard ship, perhaps functional for simpler times, appears to be inappropriate for complex ships in complex situations. Yet it may be sustained by the shipping industry and the insurance industry who need to determine liability almost as much as they need to stem the increase in accidents. It is reinforced by the “technological fix” which says, “Just give the leader more information, more accurately and faster.” (Page 230)

Chapter 7: Earthbound Systems: Dams, Quakes, Mines and Lakes

  • Overall comment: This chapter covers systems that are embedded in a large, external, mostly observable reality. This does seem to make it more difficult for a complex accident to occur.
  • The U.S. Bureau of Mines now calls for multishaft ventilation and for segmented ventilation; this eliminates many common mode failures where the whole air supply is endangered by a collapse of one ventilation shaft or the failure of one set of fans. (Page 250)
    • Comment: An example of legislated diversity injection. There is no idea if a particular ventilation shaft will fail, but the risk of all failing is much lower, since they are not meaningfully connected. The legislation doesn’t have to know about mining per se, it can look at human survival needs (exits, food, air, water) and demand that there are redundant, distributed sources for those.
  • Both dams and mining, and certainly the underground mining and surface drilling that produced the Lake Peigneur accident, have alerted us to the possibility of ecosystem accidents—the unanticipated expansion of the system and thus the scope of failures. Systems not thought to be linked suddenly are. (Page 254)
  • Large dams and reservoirs create a complex new environment, and very little is known of the mutual interactions of the component forces, on a long-term basis. (Page 255)
    • Also applies to global climate forcing

Chapter 8: Exotics: Space, Weapons, and DNA

  • The space missions illustrate that even where the talent and the funds are ample, and errors are likely to be displayed before a huge television audience, system accidents cannot be avoided. I have argued throughout this book that we should give all risky systems more quality control and training than we do, but also that where complexity and coupling lie, it will not be enough. We gave the space missions everything we had, but the system accidents still occurred. This is not a system with catastrophic potential; the victims are first-party victims. Catastrophic potential resides in most, but not all, complex and tightly coupled systems. (Page 257)
  • The centerpiece of this part of the chapter will be an account of an extraordinary mission, Apollo 13, which commenced with a system accident, and ended with a recovery that dramatically illustrates the most exemplary attributes of both humans and their machines. The event will tell us something further about complexity and coupling: the recovery was possible because ground controllers were able to make the system more linear and more loosely coupled, and to put the operators back into the control loop that rarely included them. (Page 257)
  • because of the safety systems involved in a launch-on-warning scenario, it is virtually impossible for well-intended actions to bring about an accidental attack (malevolence or derangement is something else). In one sense this is not all that comforting, since if there were a true warning that the Russian missiles were coming, it looks as if it would also be nearly impossible for there to be an intended launch, so complex and prone to failure is this system. It is an interesting case to reflect upon: at some point does the complexity of a system and its coupling become so enormous that a system no longer exists? Since our ballistic weapons system has never been called upon to perform (it cannot even be tested), we cannot be sure that it really constitutes a viable system. (Page 258)
  • In Chapter 2, I argued that every industrial activity exhibits organizational failures, incompetence, greed, and some criminality. (Page 258)
    • Comment: And this needs to be factored into any design , particularly the design of complex systems
  • The search for the problem was conducted with the well-worn assumption we have been exploring in this book: since the system is safe, or I wouldn’t be here, it must be a minor problem, or the lesser of two possible evils. Just as reactor cores had never been uncovered before, or that ship would never turn this way, or they would never set my course to hit a mountain, the idea that the heart of the spaceship might be broken was inconceivable, particularly for the managers and designers in Houston, all gathered together for the historic flight. Cooper puts it well: (Page 274)
    • …they felt secure in the knowledge that the spacecraft was as safe a machine for flying to the moon as it was possible to devise. Obviously, men would not be sent into space in anything less, and inasmuch as men were being sent into space, the pressure around NASA to have confidence in the spacecraft was enormous. Everyone placed particular faith in the spacecraft’s redundancy: there were two or more of almost everything.
    • Comment: “…almost..“. The complex behavior forms around the stiff links. This can be modeled, I think.
  • The astronauts felt a jolt and this remained an important part of their analysis throughout the process of trying to track down the problem. (Page 277)
    • Comment: Environmental awareness as another, non-social information channel
  • The accident allows us to review some typical behavior associated with system accidents:
    • initial incomprehension about what was indeed failing;
    • failures are hidden and even masked;
    • a search for a de minimus explanation, since a de maximus one is inconceivable;
    • an attempt to maintain production if at all possible;
    • mistrust of instruments, since they are known to fail;
    • overconfidence in ESDs and redundancies, based upon normal experience of smooth operation in the past;
    • ambiguous information is interpreted in a manner to confirm initial (de minimus) hypotheses;
    • tremendous time constraints, in this case involving not only the propagation of failures, but the expending of vital consumables; and
    • invariant sequences, such as the decision to turn off a subsystem that could not be restarted.
  • All this did not just take place with a few high-school graduates with some drilling in reactor procedures, or a crusty old sea captain isolated in his absolute authority, but happened with three brilliant and extremely well-trained test pilots and a gaggle of managers (scientists and engineers all) backed up by the “Great Designers” themselves, all working shifts in Houston and wired to the spacecraft. (page 278)
  • In addition, almost every step they devised could be quickly and realistically tested in a very sophisticated simulator before it was tried out in the capsule—something rarely available to high-risk systems. (Page 278)
    • Comment: In addition to diversity, simulators that afford exploring a space should be a requirement of resilient systems
  • With multiple and independent sources of information, the more detailed the information, the more unlikely an error. (Page 289)
    • Comment: This is a key aspect of diversity injection: diverse, easy to test bits of information that build a larger world view. With respect to the mineshaft legislation earlier, it’s easy to see if ventilation is coming from a shaft.
  • With ecosystem accidents the risk cannot be calculated in advance and the initial event—which usually is not even seen as a component failure at all—becomes linked with other systems from which it was believed to be independent. The other systems are not part of any expected production sequence. The linkage is not only unexpected but once it has occurred it is not even well understood or easily traced back to its source. Knowledge of the behavior of the human-made material in its new ecological niche is extremely limited by its very novelty. Ecosystem accidents illustrate the tight coupling between human made systems and natural systems. There are few or no deliberate buffers inserted between the two systems because the designers never expected them to be connected. At its roots, the ecosystem accident is the result of a design error, namely the inadequate definition of system boundaries. (Page 296)

Chapter 9: Living with High-Risk Systems

  • Ultimately, the issue is not risk, but power; the power to impose risks on the many for the benefit of the few. (Page 306)
  • When societies confront a new or explosively growing evil, the number of risk assessors probably grows—whether they are shamans or scientists. (Page 307)
  • heroic efforts would be needed to educate the general public in the skills needed to decide complex issues of risk. At the basis of this is a quarrel about forms of rationality in human affairs. (Page 315)
    • Comment: A good place for games and simulations?
  • It is convenient to think of three forms of rationality: absolute rationality, which is enjoyed primarily by economists and engineers; “bounded” or limited rationality, which a growing wing of risk assessors emphasize; and what I will call social and cultural rationality, which is what most of us live by, although without thinking that much about it. (Page 315)
    • Comment: These map roughly onto my groupings, though “Absolute” would be “environmental”
      • Absolute rationality – Nomadic
      • Bounded rationality – Flocking
      • Socio-cultural rationality – Stampede
  • One important and unintended conclusion that does come from this work is the overriding importance of the context into which the subject puts the problem. Recall our nuclear power operators, or the crew of the New Zealand DC-10 on its sightseeing trip, or the mariners interpreting ambiguous signals. The decisions made in these cases were perfectly rational; it was just that the operators were using the wrong contexts. Selecting a context (“this can happen only with a small pipe break in the secondary system”) is a pre-decision act, made without reflection, almost effortlessly, as a part of a stream of experience and mental processing. We start “thinking” or making “decisions” based upon conscious, rational effort only after the context has become defined. And defining the context is a much more subtle, self-steering process, influenced by long experience with trials and errors (much as the automatic adjustments made in driving or walking on a busy street). If a situation is ambiguous, without thinking about it or deciding upon it, we sometimes pick what seems to be the most familiar context, and only then do we begin to consciously reason. This is what appears to happen in a great many of the psychological experiments. Without conscious thought (the kind that can be easily and fairly accurately recalled), the subject says, “This is like x: I will do what I usually do then.” The results of these experiments strongly suggest that the context supplied by the subjects is not the context the experimenter expected them to supply. With an ill-defined context, the subject of the experiment may say, “Oh, this is like situation A in real life, and this is what I generally do,” while the experimenter thinks that the subject is assuming it is like situation B in real life, and is surprised by what the subject does. (Page 318)
    • Comment: This aligns with ‘Conflict and Consensus’ and is a form of dimension reduction
  • Finally, heuristics are akin to intuitions. Indeed, they might be considered to be regularized, checked-out intuitions. An intuition is a reason, hidden from our consciousness, for certain apparently unrelated things to be connected in a causal way. Experts might be defined as people who abjure intuitions; it is their virtue to have flushed out the hidden causal connections and subjected them to scrutiny and testing, and thus discarded them or verified them. Intuitions, then, are especially unfortunate forms of heuristics, because they are not amenable to inspection. This is why they are so fiercely held even in the face of contrary evidence; the person insists the evidence is irrelevant to their “insight.” (Page 319)
    • Comment: A nice distinction between Nomad and Stampede framings
  • Finally, bounded rationality is efficient because it avoids an extensive amount of effort. For the citizen, think of the work that could be required to decide just what the TMI accident signified. In the experts’ view, the public should make the effort to answer the following questions, and if they can’t, should accept the experts’ answers. Did the accident fit in the technical fault tree analysis the experts constructed in WASH-1400, the Rasmussen Report (a matter of several volumes of technical writing)? How many times in the past have we come close to an accident of this type? Can we correct the system and thus learn from the accident? Was it accurately reported? Do the experts agree on what happened? Did it fit the base rate of events that led to the prediction that it was a very rare event? And so on. The experts do not have answers to some of these questions, so the public, even were they to devote some months of study to the problem, could not be assured that the answer could be known. (Page 320)
    • Comment: Intelligence is computation, and expensive.
  • The most important factor, which they labeled “dread risk,” was associated with (Page 326):
    • lack of control over the activity;
    • fatal consequences if there were a mishap of some sort;
    • high catastrophic potential;
    • reactions of dread;
    • inequitable distribution of risks and benefits (including the transfer of risks to future generations), and
    • the belief that risks are increasing and not easily reducible.
    • Comment: These clusters are a form of dimension reduction that is emerges spontaneously in a human population. Perceived risk: psychological factors and social implications
  • The dimension of dread—lack of control, high fatalities and catastrophic potential, inequitable distribution of risks and benefits, and the sense that these risks are increasing and cannot be easily reduced by technological fixes—clearly was the best predictor of perceived risk. This is what we might call, after Clifford Geertz, a “thick description” of hazards rather than a “thin description.” A thin one is quantitative, precise, logically consistent, economical, and value-free. It embraces many of the virtues of engineering and the physical sciences, and is consistent with what we have called component failure accidents—failures that are predictable and understandable and in an expected production sequence. A thick description recognizes subjective dimensions and cultural values and, in this example, shows a skepticism about human-made systems and institutions, and emphasizes social bonding and the tentative, ambiguous nature of experience. A thick description reflects the nature of system accidents, where unanticipated, unrecognizable interactions of failures occur, and the system does not allow for recovery. (Page 328)
  • But it is fair to ask whether we have progressed enough as a species to handle the more immediate, short-term problems of DNA, chemical plants, nuclear plants, and nuclear weapons. Recall the major thesis of this book: systems that transform potentially explosive or toxic raw materials or that exist in hostile environments appear to require designs that entail a great many interactions which are not visible and in expected production sequence. Since nothing is perfect—neither designs, equipment, operating procedures, operators, materials, and supplies, nor the environment—there will be failures. If the complex interactions defeat designed-in safety devices or go around them, there will be failures that are unexpected and incomprehensible. If the system is also tightly coupled, leaving little time for recovery from failure, little slack in resources or fortuitous safety devices, then the failure cannot be limited to parts or units, but will bring down subsystems or systems. These accidents then are caused initially by component failures, but become accidents rather than incidents because of the nature of the system itself; they are system accidents, and are inevitable, or “normal” for these systems. (Page 330)
  • Accidents will be avoided if the system is also loosely coupled, (cell 4, universities and R&D units) because loose coupling gives time, resources, and alternative paths to cope with the disturbance and limits its impact. (Page 331)
  • For organizations that are both linear and loosely coupled (cell 3; most manufacturing, and single goal agencies such as a state liquor authority, vehicle registration, or licensing bureau) centralization is feasible because of the linearity, but decentralization is feasible because of the loose coupling. Thus, these organizations have a choice, insofar as organizational structure affects recovery from inevitable failures. The fact that most have opted for centralized structures says a good bit about the norms of elites who design these systems, and perhaps subtle matters such as the “reproduction of the class system” (keeping people in their place). Organizational theorists generally find such organizations overcentralized and recommend various forms of decentralization for both productivity and social rationality reasons. (Page 334)
    • Comment: Designed in inefficiently is good for preventing accidents but bad for profit
  • Organizational theorists have long since given up hope of finding perfect or even exceedingly well-run organizations, even where there is no catastrophic potential. It is an enduring limitation—if it is a limitation—of our human condition. It means that humans do not exist to give their all to organizations run by someone else, and that organizations inevitably will be run, to some degree, contrary to their interests. This is why it is not a problem of “capitalism”; socialist countries, and even the ideal communist system, cannot escape the dilemmas of cooperative, organized effort on any substantial scale and with any substantial complexity and uncertainty. At some point the cost of extracting obedience exceeds the benefits of organized activity. (Page 338)
    • Comment: First point: should we expect any better of machine organizations? Why? Second point: These are curves that cross in a fuzzy way that incorporate short term profit and long term sustainability.
  • What, then, of a less ambitious explanation than capitalism, namely greed (whether due to human nature or structural conditions)—or to put it more analytically, private gain versus the public good? (Page 340)
    • Comment: There is something here. It is not anyone’s private gain, but the gain of the powerful, the amplified. The ability of one person to have the effect of a stampede. In capitalism, this can be achieved using money. In other systems it’s birth, luck, access to power, etc. It can be very good for the few to drive the many over the cliff…
  • Systems that have high status, articulate, and resource-endowed operators are more likely to have the externalities brought to public attention (thus pressuring the system elites) than those with low-status, inarticulate, and impoverished operator groups. To protect their own interests, the operators in the first category can argue the public interest, since both operators and the public will suffer from externalities. The contrast here runs from airline pilots at the favorable extreme, through chemical plant operators, then nuclear plant operators, to miners—the weakest group. (Page 342)
    • Comment: We may only get one 10,000-car pileup. But think about what’s going on in the ML transformation of the justice system
  • The main point of the book is to see these human constructions as systems, not as collections of individuals or representatives of ideologies. From our opening accident with the coffeepot and job interview through the exotics of space, weapons, and microbiology, the theme has been that it is the way the parts fit together, interact, that is important. The dangerous accidents lie in the system, not in the components. The nature of the transformation processes elude the capacities of any human system we can tolerate in the case of nuclear power and weapons; the air transport system works well—diverse interests and technological changes support one another; we may worry much about the DNA system with its unregulated reward structure, less about chemical plants; and though the processes are less difficult and dangerous in mining and marine transport, we find the system of each is an unfortunate concatenation of diverse interests at cross-purposes. (Page 351)
    • Comment: Systems are directed networks. But then ideologies are too. They run at different scales though.
  • These systems are human constructions, whether designed by engineers and corporate presidents, or the result of unplanned, unwitting, crescive, slowly evolving human attempts to cope. (Page 351)
    • Comment: These are the kinds of systems that partially informed agent make. They are mechanisms that populations use to make decisions
  • The catastrophes send us warning signals. This book has attempted to decode these signals: abandon this, it is beyond your capabilities; redesign this, regardless of short-run costs; regulate this, regardless of the imperfections of regulation. But like the operators of TMI who could not conceive of the worst—and thus could not see the disasters facing them—we have misread these signals too often, reinterpreting them to fit our preconceptions. (Page 351)


  • But a little reflection on Bhopal led me to invert “Normal Accident Theory” in order to see how this tragedy was possible. A few hundred plants with this catastrophic potential could quietly exist for forty years without realizing their potential because it takes just the right combination of circumstances to produce a catastrophe, just as it takes the right combination of inevitable errors to produce an accident. I think this deserves to be called the “Union Carbide factor.” (Page 356)
    • Comment: Power law again?
  • It is hard to have a catastrophe; everything has to come together just right (or wrong). When it does we have “negative synergy.” Since catastrophes are rare, elites, I conclude, feel free to populate the earth with these kinds of risky systems. (Page 358)
    • Comment: There is some level of implicit tradeoff in bringing incompletely understood technologies to market. “move fast and break things” is an effective way to get past the valley of death as fast as possible – i.e. “Production Pressure”
  • Both our predictions about the possibilities of accidents and our explanations of them after they occur are profoundly compromised by our act of “social construction.” We do not know what to look for in the first place, and we jump to the most convenient explanation (culture, or bad conditions) in the second place. If an accident has occurred, we will find the most convenient explanations. (Page 359)
    • Comment: “Rush to judgement” has a long history. From the OED – Ld. Erskine Speeches Misc. Subj. (1812) 7 “An attack upon the King is considered to be patricide against the state, and the Jury and the witnesses, and even the Judges, are the children. It is fit, on that account, that there should be a solemn pause before we rush to judgment.”
  • Were we to perform a thought experiment, in which we would go into a plant that was not having an accident but assume there had just been one, we would probably find “an accident waiting to happen” (Page 359)
    • Comment: Pre-mortems are a thing. Should they be used more often?
  • “We might postulate a notion of Organizational Cognitive Complexity,” he writes, “i.e., that the sheer number of nodes and states to be monitored exceeds the organizational capacity to anticipate or comprehend. Complexity Indices for major refinery unit processes varied from 156,000,000 for alkylation/polymerization down to 90 for crude refining. The very use of a logarithmic scale to classify complexity suggests the cognitive challenges facing an organization.” A single component failure thus produces other failures in a (temporarily) incomprehensible manner if the complexity of the system is cognitively challenging. Berniker adds another significant point in his letter: the complexity of the system may not be necessary from a technical point of view, but is mostly the result of poorly planned accretions. (Page 363)
  • More valuable still is Sagan’s development of two organizational concepts that were important for my book: the notion of bounded rationality (the basis of “Garbage Can Theory”), and the notion of organizations as tools to be used for the interests of their masters (the core of a “power theory”). Both garbage can and power theory are noted in passing in my book, but Sagan’s work makes both explicit tools of analysis. Risky systems, I should have stressed more than I did, are likely to have high degrees of uncertainty associated with them and thus one can expect “garbage can” processes, that is, unstable and unclear goals, misunderstanding and mis-learning, happenstance, and confusion as to means. (For the basic work on garbage can theory, see Cohen, March, and Olsen 1988; March and Olsen 1979. For a brief summary, see Perrow 1986, 131–154.) A garbage can approach is appropriate where there is high uncertainty about means and goals and an unstable environment. It invites a pessimistic view of such things as efficiency or commitments to safety. (Page 369)
  • few managers are punished for not putting safety first even after an accident, but will quickly be punished for not putting profits, market share, or agency prestige first. (Page 370)
  • Second, why doesn’t more learning take place? One answer is provided by the title of an interesting piece: “Learning from mistakes is easier said than done …” (Edmonson 1996) in which Edmonson discusses the ample group process barriers to admitting and learning from mistakes. James Reason offers another colorful and more elaborate discussion (1990). Another is that elites do learn, but the wrong things. They learn that disasters are rare and they are not likely to be vulnerable so, in view of the attractions of creating and running risky systems the benefits truly do outweigh the risks for individual calculators. This applies to lowly operators, too; most of the time cutting corners works; only rarely does it “bite back,” in the wonderful terminology that Tenner uses in his delightful book of accident illustrations, though sometimes it will (Tenner 1996). (Page 371)
  • There are important system characteristics that determine the number of inevitable (though often small) failures which might interact unexpectedly, and the degree of coupling which determines the spread and management of the failures. Some of these are: (page 371)
    • Experience with operating scale—did it grow slowly, accumulating experience (fossil fuel utility plants) or rapidly with no experience of the new configurations or volumes (nuclear power plants)?
    • Experience with the critical phases—if starting and stopping are the risky phases, does this happen frequently (take offs and landings) or infrequently (nuclear plant outages)?
    • Information on errors—is this shared within and between the organizations? (Yes, in the Japanese nuclear power industry; barely in the U.S.) Can it be obtained, as in air transport, or does it go to the bottom with the ship?
    • Close proximity of elites to the operating system—they fly on airplanes but don’t ship on rusty bottoms or live next to chemical plants.
    • Organizational control over members—solid with the naval carriers, partial with nuclear defense, mixed with marine transport (new crews each time), absent with chemical and nuclear plants (and we are hopefully reluctant to militarize all risky systems).
    • Organizational density of the system’s environment (vendors, subcontractors, owner’s/operator’s trade associations and unions, regulators, environmental groups, and competitive systems)—if rich, there will be persistent investigations and less likelihood of blaming God or the operators, and more attempts to increase safety features.
  • When the Exxon Valdez accident occurred Clarke took these insights with him to Alaska, where he came upon a truly mammoth social construction, the several contingency plans put out by the oil industry for cleaning up 80 to 90 percent of any spill, plans that necessitated getting permission to operate in the Sound. The plans were so fanciful that it was hard to take them seriously. There was no case on record of a substantial cleanup of a large spill, yet industry and government regulators promised just that. There was such a disjuncture between what they said they could do and what they actually could do that Clarke saw the plans as symbols rather than blueprints, as “fantasy documents.” (Page 374)
    • Comment: This is an attempt to represent a social reality as a physical one
  • Snook makes another contribution. This is, he says, a normal accident without any failures. How can that be? Well, there were minor deviations in procedures that saved time and effort or corrected some problems without disclosing potential ones. Organizations and large systems probably could not function without the lubricant of minor deviations to handle situations no designer could anticipate. Individually they were inconsequential deviations, and the system had over three years of daily safe operations. But they resulted in the slow, steady uncoupling of local practice—the jets, the AWACS, the army helicopters—from the written procedures, a practice he labels “practical drift.” Local adaptation, he says, can lead to large system disaster. The local adaptations are like a fleet of boats sailing to a destination; there is continuous, intended adjustment to the wind and the movement of nearby boats. The disaster comes when, only from high above the fleet, or from hindsight, we see that the adaptive behavior imperceptibly and unexpectedly result in the boats “drifting” into gridlock, collision, or grounding. (Page 378)
  • Instead, she (Diane Vaughn) builds what would be called a “social construction of reality” case that allowed the banality of bureaucracy to create a habit of normalizing deviations from safe procedures. They all did what they were supposed to do, and were used to doing, and the deviations finally caught up with them. (Page 380)
  • this was not the normalization of deviance or the banality of bureaucratic procedures and hierarchy or the product of an engineering “culture”; it was the exercise of organizational power. We miss a great deal when we substitute culture for power. (Page 380)
    • Comment: One thing that I haven’t really thought about is the role of power in forcing a stampede. There may be herding aspects of a “rush to judgement”
  • New financial instruments such as derivatives and hedge funds and new techniques such as programmed trading further increase the complexity of interactions. Breaking up a loan on a home into tiny packages and selling them on a world-wide basis increases interdependency. (Page 385)
    • Comment: This was written in 1999!
  • But another condition of interdependency is not a bunch of pipes with tight fittings and one direction of flow, but a web of connections wherein some pathways are multipurpose, some might not be used, some reversed, some used in unforeseen ways, some spilling excess capacity, and some buffering and isolating disturbances. This is a more “organic” conception of interdependency, consistent with loose coupling. The distinction is an old one in sociology, going back at least to Emile Durkheim (mechanical and organic solidarity) and Ferdinand Toennies (society versus community) and finding echos in most major sociological thinkers. Predictions follow freely. External threats are supposed to heighten organic solidarity, internal economic downturns, mechanical solidarity. Modernization and bureaucracy find their rationale and techniques in mechanical coupling; communitarian, religious, and cult outbreaks find their rationale in organic coupling and solidarity. (Page 392)
  • This points to the basic difference between 1) software problems—billions of lines of code in forgotten programs in computers that have not been manufactured for ten years—which is bad enough, and 2) the embedded chip problem—billions of chips that are in controllers, rheostats, pumps, valves, regulators, safety systems and so on, which are hard-wired and must be taken out and replaced. (Page 401)


Characterizing Online Public Discussions through Patterns of Participant Interactions.

Characterizing Online Public Discussions through Patterns of Participant Interactions



An important paper that lays out mechanisms for relating conversations into navigable spaces. To me, this seems like a first step in being able to map human interaction along the dimensions the humans emphasize. In this case, the dimensions have to do with relatively coarse behavior trajectories: Will a participant block another? Will this be a long threaded discussion among a few people or a set of short links all referring to an initial post?

Rooted in the design affordances of facebook, the data that are readily available influence the overall design of the methods used. For example, a significant amount of the work is focussed on temporal network analytics. I think that these methods are quite generalizable to sites like Twitter and Reddit. The fact that the researchers worked at Facebook and had easy access to the data is a critical part of this studies’ success. For me the implications aren’t that surprising (I found myself saying “Yes! Yes!” several times while reading this), but it is wonderful to see then presented in such a clear, defensible way.

My more theoretical thoughts

Though this study is focussed more on building representations of behaviors, I think that the methods used here (particularly as expanded on in the Future Work section) should be extensible to mapping beliefs

The extensive discussion about how the design affordances of Facebook create the form of the discussion is also quite validating. Although they don’t mention it, Moscovici lays this concept out in Conflict and Consensus, where he describes how even items such as table shape can change a conversation so that the probability of compromise over consensus is increased.

Lastly, I’m really looking forward to checking out the Cornell Conversational Analysis Toolkit, developed for(?) this study.


  • This paper introduces a computational framework to characterize public discussions, relying on a representation that captures a broad set of social patterns which emerge from the interactions between interlocutors, comments and audience reactions. (Page 198:1)
  • we use it to predict the eventual trajectory of individual discussions, anticipating future antisocial actions (such as participants blocking each other) and forecasting a discussion’s growth (Page 198:1)
  • platform maintainers may wish to identify salient properties of a discussion that signal particular outcomes such as sustained participation [9] or future antisocial actions [16], or that reflect particular dynamics such as controversy [24] or deliberation [29]. (Page 198:1)
  • Systems supporting online public discussions have affordances that distinguish them from other forms of online communication. Anybody can start a new discussion in response to a piece of content, or join an existing discussion at any time and at any depth. Beyond textual replies, interactions can also occur via reactions such as likes or votes, engaging a much broader audience beyond the interlocutors actively writing comments. (Page 198:2)
    • This is why JuryRoom would be distinctly different. It’s unique affordances should create unique, hopefully clearer results.
  • This multivalent action space gives rise to salient patterns of interactional structure: they reflect important social attributes of a discussion, and define axes along which discussions vary in interpretable and consequential ways. (Page 198:2)
  • Our approach is to construct a representation of discussion structure that explicitly captures the connections fostered among interlocutors, their comments and their reactions in a public discussion setting. We devise a computational method to extract a diverse range of salient interactional patterns from this representation—including but not limited to the ones explored in previous work—without the need to predefine them. We use this general framework to structure the variation of public discussions, and to address two consequential tasks predicting a discussion’s future trajectory: (a) a new task aiming to determine if a discussion will be followed by antisocial events, such as the participants blocking each other, and (b) an existing task aiming to forecast the growth of a discussion [9]. (Page 198:2)
  • We find that the features our framework derives are more informative in forecasting future events in a discussion than those based on the discussion’s volume, on its reply structure and on the text of its comments (Page 198:2)
  • we find that mainstream print media (e.g., The New York Times, The Guardian, Le Monde, La Repubblica) is separable from cable news channels (e.g., CNN, Fox News) and overtly partisan outlets (e.g., Breitbart, Sean Hannity, Robert Reich)on the sole basis of the structure of the discussions they trigger (Figure 4).(Page 198:2)
  • figure-4
  • These studies collectively suggest that across the broader online landscape, discussions take on multiple types and occupy a space parameterized by a diversity of axes—an intuition reinforced by the wide range of ways in which people engage with social media platforms such as Facebook [25]. With this in mind, our work considers the complementary objective of exploring and understanding the different types of discussions that arise in an online public space, without predefining the axes of variation. (Page 198:3)
  • Many previous studies have sought to predict a discussion’s eventual volume of comments with features derived from their content and structure, as well as exogenous information [893069, inter alia]. (Page 198:3)
  • Many such studies operate on the reply-tree structure induced by how successive comments reply to earlier ones in a discussion rooted in some initial content. Starting from the reply-tree view, these studies seek to identify and analyze salient features that parameterize discussions on platforms like Reddit and Twitter, including comment popularity [72], temporal novelty [39], root-bias [28], reply-depth [41, 50] and reciprocity [6]. Other work has taken a linear view of discussions as chronologically ordered comment sequences, examining properties such as the arrival sequence of successive commenters [9] or the extent to which commenters quote previous contributions [58]. The representation we introduce extends the reply-tree view of comment-to-comment. (Page 198:3)
  • Our present approach focuses on representing a discussion on the basis of its structural rather than linguistic attributes; as such, we offer a coarser view of the actions taken by discussion participants that more broadly captures the nature of their contributions across contexts which potentially exhibit large linguistic variation.(Page 198:4)
  • This representation extends previous computational approaches that model the relationships between individual comments, and more thoroughly accounts for aspects of the interaction that arise from the specific affordances offered in public discussion venues, such as the ability to react to content without commenting. Next, we develop a method to systematically derive features from this representation, hence producing an encoding of the discussion that reflects the interaction patterns encapsulated within the representation, and that can be used in further analyses.(Page 198:4)
  • In this way, discussions are modelled as collections of comments that are connected by the replies occurring amongst them. Interpretable properties of the discussion can then be systematically derived by quantifying structural properties of the underlying graph: for instance, the indegree of a node signifies the propensity of a comment to draw replies. (Page 198:5)
    • Quick responses that reflect a high degree of correlation would be tight. A long-delayed “like” could be slack?
  • For instance, different interlocutors may exhibit varying levels of engagement or reciprocity. Activity could be skewed towards one particularly talkative participant or balanced across several equally-prolific contributors, as can the volume of responses each participant receives across the many comments they may author.(Page 198: 5)
  • We model this actor-focused view of discussions with a graph-based representation that augments the reply-tree model with an additional superstructure. To aid our following explanation, we depict the representation of an example discussion thread in Figure 1 (Page 198: 6)
  • fig1table1
  • Relationships between actors are modeled as the collection of individual responses they exchange. Our representation reflects this by organizing edges into hyperedges: a hyperedge between a hypernode C and a node c ‘ contains all responses an actor directed at a specific comment, while a hyperedge between two hypernodes C and C’ contains the responses that actor C directed at any comment made by C’ over the entire discussion. (Page 198: 6)
    • I think that this  can be represented as a tensor (hyperdimensional or flattened) with each node having a value if there is an intersection. There may be an overall scalar that allows each type of interaction to be adjusted as a whole
  • The mixture of roles within one discussion varies across different discussions in intuitively meaningful ways. For instance, some discussions are skewed by one particularly active participant, while others may be balanced between two similarly-active participants who are perhaps equally invested in the discussion. We quantify these dynamics by taking several summary statistics of each in/outdegree distribution in the hypergraph representation, such as their maximum, mean and entropy, producing aggregate characterizations of these properties over an entire discussion. We list all statistics computed in the appendices (Table 4). (Page 198: 6, 7)
  • table4
  • To interpret the structure our model offers and address potentially correlated or spurious features, we can perform dimensionality reduction on the feature set our framework yields. In particular, let X be a N×k matrix whose N rows each correspond to a thread represented by k features.We perform a singular value decomposition on X to obtain a d-dimensional representation X ˜ Xˆ = USVT where rows of U are embeddings of threads in the induced latent space and rows of V represent the hypergraph-derived features. (Page 198: 9)
    • This lets us find the hyperplane of the map we want to build
  • Community-level embeddings. We can naturally extend our method to characterize online discussion communities—interchangeably, discussion venues—such as Facebook Pages. To this end, we aggregate representations of the collection of discussions taking place in a community, hence providing a representation of communities in terms of the discussions they foster. This higher level of aggregation lends further interpretability to the hypergraph features we derive. In particular, we define the embedding U¯C of a community C containing threads {t1, t2, . . . tn } as the average of the corresponding thread embeddings Ut1 ,Ut2 , . . .Utn , scaled to unit l2 norm. Two communities C1 and C2 that foster structurally similar discussions then have embeddings U¯C1 and U¯C2 that are close in the latent space.(Page 198: 9)
    • And this may let us place small maps in a larger map. Not sure if the dimensions will line up though
  • The set of threads to a post may be algorithmically re-ordered based on factors like quality [13]. However, subsequent replies within a thread are always listed chronologically.We address elements of such algorithmic ranking effects in our prediction tasks (§5). (Page 198: 10)
  • Taken together, these filtering criteria yield a dataset of 929,041 discussion threads.(Page 198: 10)
  • We now apply our framework to forecast a discussion’s trajectory—can interactional patterns signal future thread growth or predict future antisocial actions? We address this question by using the features our method extracts from the 10-comment prefix to predict two sets of outcomes that occur temporally after this prefix. (Pg 198:10)
    • These are behavioral trajectories, though not belief trajectories. Maps of these behaviors could probably be built, too.
  • For instance, news articles on controversial issues may be especially susceptible to contentious discussions, but this should not translate to barring discussions about controversial topics outright. Additionally, in large-scale social media settings such as Facebook, the content spurring discussions can vary substantially across different sub-communities, motivating the need to seek adaptable indicators that do not hinge on content specific to a particular context. (Page 198: 11)
  • Classification protocol. For each task, we train logistic regression classifiers that use our full set of hypergraph-derived features, grid-searching over hyperparameters with 5-fold cross-validation and enforcing that no Page spans multiple folds.13 We evaluate our models on a (completely fresh) heldout set of thread pairs drawn from the subsequent week of data (Nov. 8-14, 2017), addressing a model’s potential dependence on various evolving interface features that may have been deployed by Facebook during the time spanned by the training data. (Page 198: 11)
    • We use logistic regression classifiers from scikit-learn with l2 loss, standardizing features and grid-searching over C = {0.001, 0.01, 1}. In the bag-of-words models, we tf-idf transform features, set a vocabulary size of 5,000 words and additionally grid-search over the maximum document frequency in {0.25, 0.5, 1}. (Page 198: 11, footnote 13)
  • We test a model using the temporal rate of commenting, which was shown to be a much stronger signal of thread growth than the structural properties considered in prior work [9] (Page 198: 12)
  • Table 3 shows Page-macroaveraged heldout accuracies for our prediction tasks. The feature set we extract from our hypergraph significantly outperforms all of the baselines in each task. This shows that interactional patterns occurring within a thread’s early activity can signal later events, and that our framework can extract socially and structurally-meaningful patterns that are informative beyond coarse counts of activity volume, the reply-tree alone and the order in which commenters contribute, along with a shallow representation of the linguistic content discussed. (Page 198: 12)
    • So triangulation from a variety of data sources produces more accurate results in this context, and probably others. Not a surprising finding, but important to show
  • table3
  • We find that in almost all cases, our full model significantly outperforms each subcomponent considered, suggesting that different parts of the hypergraph framework add complementary information across these tasks. (Page 198: 13)
  • Having shown that our approach can extract interaction patterns of practical importance from individual threads, we now apply our framework to explore the space of public discussions occurring on Facebook. In particular, we identify salient axes along which discussions vary by qualitatively examining the latent space induced from the embedding procedure described in §3, with d = 7 dimensions. Using our methodology, we recover intuitive types of discussions, which additionally reflect our priors about the venues which foster them. This analysis provides one possible view of the rich landscape of public discussions and shows that our thread representation can structure this diverse space of discussions in meaningful ways. This procedure could serve as a starting point for developing taxonomies of discussions that address the wealth of structural interaction patterns they contain, and could enrich characterizations of communities to systematically account for the types of discussions they foster. (Page 198: 14) 
    • ^^^Show this to Wayne!^^^
  • The emergence of these groupings is especially striking since our framework considers just discussion structure without explicitly encoding for linguistic, topical or demographic data. In fact, the groupings produced often span multiple languages—the cluster of mainstream news sites at the top includes French (Le Monde), Italian (La Repubblica) and German (SPIEGEL ONLINE) outlets; the “sports” region includes French (L’EQUIPE) as well as English outlets. This suggests that different types of content and different discussion venues exhibit distinctive interactional signatures, beyond lexical traits. Indeed, an interesting avenue of future work could further study the relation between these factors and the structural patterns addressed in our approach, or augment our thread representation with additional contextual information. (Page 198: 15)
  • Taken together, we can use the features, threads and Pages which are relatively salient in a dimension to characterize a type of discussion. (Page 198: 15)
  • To underline this finer granularity, for each examined dimension we refer to example discussion threads drawn from a single Page, The New York Times(, which are listed in the footnotes. (Page 198: 15)
    • Common starting point. Do they find consensus, or how the dimensions reduce?
  • Focused threads tend to contain a small number of active participants replying to a large proportion of preceding comments; expansionary threads are characterized by many less-active participants concentrating their responses on a single comment, likely the initial one. We see that (somewhat counterintuitively) meme-sharing discussion venues tend to have relatively focused discussions. (Page 198: 15)
    • These are two sides of the same dimension-reduction coin. A focused thread should be using the dimension-reduction tool of open discussion that requires the participants to agree on what they are discussing. As such it refines ideas and would produce more meme-compatible content. Expansive threads are dimension reducing to the initial post. The subsequent responses go in too many directions to become a discussion.
  • Threads at one end (blue) have highly reciprocal dyadic relationships in which both reactions and replies are exchanged. Since reactions on Facebook are largely positive, this suggests an actively supportive dynamic between actors sharing a viewpoint, and tend to occur in lifestyle-themed content aggregation sub-communities as well as in highly partisan sites which may embody a cohesive ideology. In threads at the other end (red), later commenters tend to receive more reactions than the initiator and also contribute more responses. Inspecting representative threads suggests this bottom-heavy structure may signal a correctional dynamic where late arrivals who refute an unpopular initiator are comparatively well-received. (Page 198: 17)
  • This contrast reflects an intuitive dichotomy of one- versus multi-sided discussions; interestingly, the imbalanced one-sided discussions tend to occur in relatively partisan venues, while multi-sided discussions often occur in sports sites (perhaps reflecting the diversity of teams endorsed in these sub-communities). (Page 198: 17)
    • This means that we can identify one-sided behavior and use that then to look at they underlying information. No need to look in diverse areas, they are taking care of themselves. This is ecosystem management 101, where things like algae blooms and invasive species need to be recognized and then managed
  • We now seek to contrast the relative salience of these factors after controlling for community: given a particular discussion venue, is the content or the commenter more responsible for the nature of the ensuing discussions? (Page 198: 17)
  • This suggests that, perhaps somewhat surprisingly, the commenter is a stronger driver of discussion type. (Page 198: 18)
    • I can see that. The initial commenter is kind of a gate-keeper to the discussion. A low-dimension, incendiary comment that is already aligned with one group (“lock her up”), will create one kind of discussion, while a high-dimensional, nuanced post will create another.
  • We provide a preliminary example of how signals derived from discussion structure could be applied to forecast blocking actions, which are potential symptoms of low-quality interactions (Page 198: 18)
  • The nature of the discussion may also be shaped by the structure of the underlying social network, such that interactions between friends proceed in contrasting ways from interactions between complete strangers.  (Page 198: 19)
    • Yep, design matters. Diversity injection matters.
  • For instance, as with the bulk of other computational studies, our work relies heavily on indicators of interactional dynamics which are easily extracted from the data, such as replies or blocks. Such readily available indicators can at best only approximate the rich space of participant experiences, and serve as very coarse proxies for interactional processes such as breakdown or repair [27, 62]. As such, our implicit preference for computational expedience limits the granularity and nuance of our analyses. (Page 198: 20)
    • Another argument for funding a platform that is designed to provide these nuances
  • One possible means of enriching our model to address this limitation could be to treat nodes as high-dimensional vectors, such that subsequent responses only act on a subset of these dimensions. (Page 198: 21)
    • Agreed. A set of matrices that represent an aspect of each node should have a rich set of capabilities
  • Accounting for linguistic features of the replies within a discussion necessitates vastly enriching the response types presently considered, perhaps through a model that represents the corresponding edges as higher-dimensional vectors rather than as discrete types. Additionally, linguistic features might identify replies that address multiple preceding comments or a small subset of ideas within the target(s) of the reply, offering another route to move beyond the atomicity of comments assumed by our present framework. (Page 198: 21)
    • Exactly right. High dimensional representations that can then be analyzed to uncover the implicit dimensions of interaction is the way to go, I think.
  • Important references

Similar neural responses predict friendship

Similar neural responses predict friendship

Authors and related work


A detailed, lay overview has been written up in the New York Times: You Share Everything With Your Bestie. Even Brain Waves.

The study took a cohort (N = 279) of graduate students in a graduate program. Students were asked to list who their friends were, from which a social network was constructed. A subset (N = 42) of these students were then asked to watch a series of videos while their brains were being monitored by an fMRI machine. The timings of brain activations across 80 regions of the brain were compared to see if there were similarities that correlated with social distance. Statistically significant similarities exist such that friends could be identified by firing patterns and timing. Particularly, individuals with one degree of separation were strongly resonant(?), while individuals with three or more degrees of separation could not be discriminated by fMRI.

My more theoretical thoughts:

This is more support for the idea that groups of people “flock” in latent belief space. If everyone fired in the same way to the videos, then the environmental influence would have been dominant – a video of a sloth or a volcano is “objectively” interpreted across a population. Instead, the interpretation of the videos is clustered around individuals with high levels of social connection. Humans spontaneously form groups of preferred sizes organized in a geometrical series approximating 3–5, 9–15, 30–45, etc. This is remarkably similar to the numbers found in social organizations such as flocks of starlings (seven). As we’ve seen in multiple studies, a certain amount of social cohesion is beneficial as away of finding resources in a noisy environment (Grunbaum), so this implies that belief space is noisy, but that beneficial beliefs can be found using similar means.  Grunbaum also finds that excessive social cohesion (stampedes) decrease the ability to find resources. Determining the balance of explore/exploit with respect to depending on your neighbors/friends is uncomputable, but exploration is computationally more expensive than exploitation, so the pressure is always towards some level of stampede.

This means that in physical and belief spaces, the density and stiffness of connections controls the behavior of the social network. By adjusting the dial on the similarity aspect (increasing/decreasing stiffness of the links) should result in nomadic, flocking and stampeding behavior in belief space.


  • Research has borne out this intuition: social ties are forged at a higher-than expected rate between individuals of the same age, gender, ethnicity, and other demographic categories. This assortativity in friendship networks is referred to as homophily and has been demonstrated across diverse contexts and geographic locations, including online social networks [2345(Page 2)
  • When humans do forge ties with individuals who are dissimilar from themselves, these relationships tend to be instrumental, task-oriented (e.g., professional collaborations involving people with complementary skill sets [7]), and short-lived, often dissolving after the individuals involved have achieved their shared goal. Thus, human social networks tend to be overwhelmingly homophilous [8]. (Page 2)
    • This means that groups can be more efficient, but prone to belief stampede
  • Remarkably, social network proximity is as important as genetic relatedness and more important than geographic proximity in predicting the similarity of two individuals’ cooperative behavioral tendencies [4] (Page 2)
  • how individuals interpret and respond to their environment increases the predictability of one another’s thoughts and actions during social interactions [14], since knowledge about oneself is a more valid source of information about similar others than about dissimilar others. (Page 2)
    • There is a second layer on top of this which may be more important. How individuals respond to social cues (which can have significant survival value in a social animal) may be more important than day-to-day reactions to the physical environment.
  • Here we tested the proposition that neural responses to naturalistic audiovisual stimuli are more similar among friends than among individuals who are farther removed from one another in a real-world social network. Measuring neural activity while people view naturalistic stimuli, such as movie clips, offers an unobtrusive window into individuals’ unconstrained thought processes as they unfold [16(page 2)
  • Social network proximity appears to be significantly associated with neural response similarity in brain regions involved in attentional allocation, narrative interpretation, and affective responding (Page 2)
  • We first characterized the social network of an entire cohort of students in a graduate program. All students (N = 279) in the graduate program completed an online survey in which they indicated the individuals in the program with whom they were friends (see Methods for further details). Given that a mutually reported tie is a stronger indicator of the presence of a friendship than an unreciprocated tie, a graph consisting only of reciprocal (i.e., mutually reported) social ties was used to estimate social distances between individuals. (Page 2)
    • I wonder if this changes as people age. Are there gender differences?
  • The videos presented in the fMRI study covered a range of topics and genres (e.g., comedy clips, documentaries, and debates) that were selected so that they would likely be unfamiliar to subjects, effectively constrain subjects’ thoughts and attention to the experiment (to minimize mind wandering), and evoke meaningful variability in responses across subjects (because different subjects attend to different aspects of them, have different emotional reactions to them, or interpret the content differently, for example). (Page 3)
    • I think this might make the influence more environmental than social. It would be interesting to see how a strongly aligned group would deal with a polarizing topic, even something like sports.
  • Mean response time series spanning the course of the entire experiment were extracted from 80 anatomical regions of interest (ROIs) for each of the 42 fMRI study subjects (page 3)
    • 80 possible dimensions. It would be interesting to see this in latent space. That being said, there is no dialog here, so no consensus building, which implies no dimension reduction.
  • To test for a relationship between fMRI response similarity and social distance, a dyad-level regression model was used. Models were specified either as ordered logistic regressions with categorical social distance as the dependent variable or as logistic regression with a binary indicator of reciprocated friendship as the dependent variable. We account for the dependence structure of the dyadic data (i.e., the fact that each fMRI subject is involved in multiple dyads), which would otherwise underestimate the standard errors and increase the risk of type 1 error [20], by clustering simultaneously on both members of each dyad [2122].
  • For the purpose of testing the general hypothesis that social network proximity is associated with more similar neural responses to naturalistic stimuli, our main predictor variable of interest, neural response similarity within each student dyad, was summarized as a single variable. Specifically, for each dyad, a weighted average of normalized neural response similarities was computed, with the contribution of each brain region weighted by its average volume in our sample of fMRI subjects. (Page 3)
  • To account for demographic differences that might impact social network structure, our model also included binary predictor variables indicating whether subjects in each dyad were of the same or different nationalities, ethnicities, and genders, as well as a variable indicating the age difference between members of each dyad. In addition, a binary variable was included indicating whether subjects were the same or different in terms of handedness, given that this may be related to differences in brain functional organization [23]. (page 3)
  • Logistic regressions that combined all non-friends into a single category, regardless of social distance, yielded similar results, such that neural similarity was associated with a dramatically increased likelihood of friendship, even after accounting for similarities in observed demographic variables. More specifically, a one SD increase in overall neural similarity was associated with a 47% increase in the likelihood of friendship(logistic regression: ß = 0.388; SE = 0.109; p = 0.0004; N = 861 dyads)Again, neural similarity improved the model’s predictive power above and beyond observed demographic similarities, χ2(1) = 7.36, p = 0.006. (Page 4)
  • To gain insight into what brain regions may be driving the relationship between social distance and overall neural similarity, we performed ordered logistic regression analyses analogous to those described above independently for each of the 80 ROIs, again using cluster-robust standard errors to account for dyadic dependencies in the data. This approach is analogous to common fMRI analysis approaches in which regressions are carried out independently at each voxel in the brain, followed by correction for multiple comparisons across voxels. We employed false discovery rate (FDR) correction to correct for multiple comparisons across brain regions. This analysis indicated that neural similarity was associated with social network proximity in regions of the ventral and dorsal striatum … Regression coefficients for each ROI are shown in Fig. 6, and further details for ROIs that met the significance threshold of p < 0.05, FDR-corrected (two tailed) are provided in Table 2. (Page 4)
    • So the latent space that matters involves something on the order of 7 – 9 regions? I wonder if the actions across regions are similar enough to reduce further. I need to look up what each region does.
  • Table 2Figure6
  • Results indicated that average overall (weighted average) neural similarities were significantly higher among distance 1 dyads than dyads belonging to other social distance categories … distance 4 dyads were not significantly different in overall neural response similarity from dyads in the other social distance categories. All reported p-values are two-tailed. (Page 4)
  • Within the training data set for each data fold, a grid search procedure [24] was used to select the C parameter of a linear support vector machine (SVM) learning algorithm that would best separate dyads according to social distance. (Page 5)
  • As shown in Fig. 8, the classifier tended to predict the correct social distances for dyads in all distance categories at rates above the accuracy level that would be expected based on chance alone (i.e., 25% correct), with an overall classification accuracy of 41.25%. Classification accuracies for distance 1, 2, 3, and 4 dyads were 48%, 39%, 31%, and 47% correct, respectively. (Page 6)
  • where the classifier assigned the incorrect social distance label to a dyad, it tended to be only one level of social distance away from the correct answer: when friends were misclassified, they were misclassified most often as distance 2 dyads; when distance 2 dyads were misclassified, they were misclassified most often as distance 1 or 3 dyads, and so on. (Page 6)
  • The results reported here are consistent with neural homophily: people tend to be friends with individuals who see the world in a similar way. (Page 7)
  • Brain areas where response similarity was associated with social network proximity included subcortical areas implicated in motivation, learning, affective processing, and integrating information into memory, such as the nucleus accumbens, amygdala, putamen, and caudate nucleus [27, 28, 29]. Social network proximity was also associated with neural response similarity within areas involved in attentional allocation, such as the right superior parietal cortex [30,31], and regions in the inferior parietal lobe, such as the bilateral supramarginal gyri and left inferior parietal cortex (which includes the angular gyrus in the parcellation scheme used [32]), that have been implicated in bottom-up attentional control, discerning others’ mental states, processing language and the narrative content of stories, and sense-making more generally [3334, 35]. (Page 7)
  • However, the current results suggest that social network proximity may be associated with similarities in how individuals attend to, interpret, and emotionally react to the world around them. (Page 7)
    • Both the environmental and social world
  • A second, not mutually exclusive, possibility pertains to the “three degrees of influence rule” that governs the spread of a wide range of phenomena in human social networks [43]. Data from large-scale observational studies as well as lab-based experiments suggest that wide-ranging phenomena (e.g., obesity, cooperation, smoking, and depression) spread only up to three degrees of geodesic distance in social networks, perhaps due to social influence effects decaying with social distance to the extent that the they are undetectable at social distances exceeding three, or to the relative instability of long chains of social ties [43]. Although we make no claims regarding the causal mechanisms behind our findings, our results show a similar pattern. (Page 8)
    • Does this change with the level of similarity in the group?
  • pre-existing similarities in how individuals tend to perceive, interpret, and respond to their environment can enhance social interactions and increase the probability of developing a friendship via positive affective processes and by increasing the ease and clarity of communication [1415]. (Page 8)

Meltdown: Why our systems fail and What we can do about it

Meltdown: Why our systems fail and What we can do about it

Authors and related work

  • Chris Clearfield 
    • Chris is the founder of System Logic, an independent research and consulting firm focusing on the challenges posed by risk and complexity. He previously worked as a derivatives trader at Jane Street, a quantitative trading firm, in New York, Tokyo, and Hong Kong, where he analyzed and devised mitigations for the financial and regulatory risks inherent in the business of technologically complex high-speed trading. He has written about catastrophic failure, technology, and finance for The Guardian, Forbes, the Harvard Kennedy School Review, the popular science magazine Nautilus, and the Harvard Business Review blog.
  • András Tilcsik
    • András holds the Canada Research Chair in Strategy, Organizations, and Society at the University of Toronto’s Rotman School of Management. He has been recognized as one of the world’s top forty business professors under forty and as one of thirty management thinkers most likely to shape the future of organizations. The United Nations named his course on organizational failure as the best course on disaster risk management in a business school. 
  • How to Prepare for a Crisis You Couldn’t Possibly Predict
    • Over the past five years, we have studied dozens of unexpected crises in all sorts of organizations and interviewed a broad swath of people — executives, pilots, NASA engineers, Wall Street traders, accident investigators, doctors, and social scientists — who have discovered valuable lessons about how to prepare for the unexpected. Here are three of those lessons.


This book looks at the underlying reasons for accidents that emerge from complexity and how diversity is a fix. It’s based on Charles Perrow’s concept of Normal Accidents being a property of high-risk systems.

Normal Accidents are unpredictable, yet inevitable combinations of small failures that build upon each other within an unforgiving environment. Normal accidents include catastrophic failures such as reactor meltdowns, airplane crashes, and stock market collapses. Though each failure is unique, all these failures have common properties:

    • The system’s components are tightly coupled. A change in one place has rapid consequences elsewhere
    • The system is densely connected, so that the actions of one part affects many others
    • The system’s internals are difficult to observe, so that failure can appear without warning

What happens in all these accidents is that there is misdirected progress in a direction that makes the problem worse. Often, this is because the humans in the system are too homogeneous. They all see the problem from the same perspective, and they all implicitly trust each other (Tight coupling and densely connected).

The addition of diversity is a way to solve this problem. Diversity does three things:

    • It provides additional perspectives into the problem. This only works if there is large enough representation of diverse groups so that they do not succumb to social pressure.
    • It lowers the amount of trust within the group, so that proposed solutions are exposed to a higher level of skepticism.
    • It slows the process down, making the solution less reflexive and more thoughtful.

By designing systems to be transparent, loosely coupled and sparsely connected, the risk of catastrophe is reduced. If that’s not possible, ensure that the people involved in the system are diverse.

My more theoretical thoughts:

There are two factors that affect the response of the network: The level of connectivity and the stiffness of the links. When the nodes have a velocity component, then a sufficiently stiff network (either many somewhat stiff or a few very stiff links) has to move as a single entity. Nodes with sparse and slack connections are safe systems, but not responsive. Stiff, homogeneous (similarity is implicit stiff coupling)  networks are prone to stampede. Think of a ball rolling down a hill as opposed to a lump of jello.

When all the nodes are pushing in the same direction, then the network as a whole will move into more dangerous belief spaces. That’s a stampede. When some percentage of these connections are slack connections to diverse nodes (e.g. moving in other directions), the structure as a whole is more resistant to stampede.

I think that dimension reduction is inevitable in a stiffening network. In physical systems, where the nodes have mass, a stiff structure really only has two degrees of freedom, its direction of travel and its axis of rotation. Which means that regardless of the number of initial dimensions, a stiff body’s motion reduces to two components. Looking at stampedes and panics, I’d say that this is true for behaviors as well, though causality could run in either direction. This is another reason that diversity helps keep away from dangerous conditions, but at the expense of efficiency.


  • Such a collision should have been impossible. The entire Washington Metro system, made up of over one hundred miles of track, was wired to detect and control trains. When trains got too close to each other, they would automatically slow down. But that day, as Train 112 rounded a curve, another train sat stopped on the tracks ahead—present in the real world, but somehow invisible to the track sensors. Train 112 automatically accelerated; after all, the sensors showed that the track was clear. By the time the driver saw the stopped train and hit the emergency brake, the collision was inevitable. (Page 2)
  • The second element of Perrow’s theory (of normal accidents) has to do with how much slack there is in a system. He borrowed a term from engineering: tight coupling. When a system is tightly coupled, there is little slack or buffer among its parts. The failure of one part can easily affect the others. Loose coupling means the opposite: there is a lot of slack among parts, so when one fails, the rest of the system can usually survive. (Page 25)
  • Perrow called these meltdowns normal accidents. “A normal accident,” he wrote, “is where everyone tries very hard to play safe, but unexpected interaction of two or more failures (because of interactive complexity) causes a cascade of failures (because of tight coupling).” Such accidents are normal not in the sense of being frequent but in the sense of being natural and inevitable. “It is normal for us to die, but we only do it once,” he quipped. (Page 27)
    • This is exactly what I see in my simulations and in modelling with graph Laplacians. There are two factors that affect the response of the network: The level of connectivity and the stiffness of the links. When the nodes have a velocity component, then a sufficiently stiff network (either many somewhat stiff or a few very stiff links) has to behave as a single entity.
  • These were unintended interactions between the glitch in the content filter, Talbot’s photo, other Twitter users’ reactions, and the resulting media coverage. When the content filter broke, it increased tight coupling because the screen now pulled in any tweet automatically. And the news that Starbucks had a PR disaster in the making spread rapidly on Twitter—a tightly coupled system by design. (Page 30)
  • This approach—reducing complexity and adding slack—helps us escape from the danger zone. It can be an effective solution, one we’ll explore later in this book. But in recent decades, the world has actually been moving in the opposite direction: many systems that were once far from the danger zone are now in the middle of it. (Page 33)
  • Today, smartphone videos create complexity because they link things that weren’t always connected (Page 37)
  • For nearly thirty minutes, Knight’s trading system had gone haywire and sent out hundreds of unintended orders per second in 140 stocks. Those very orders had caused the anomalies that John Mueller and traders across Wall Street saw on their screens. And because Knight’s mistake roiled the markets in such a visible way, traders could reverse engineer its positions. Knight was a poker player whose opponents knew exactly what cards it held, and it was already all in. For thirty minutes, the company had lost more than $15 million per minute. (Page 41)
  • Though a small software glitch caused Knight’s failure, its roots lay much deeper. The previous decade of technological innovation on Wall Street created the perfect conditions for the meltdown. Regulation and technology transformed stock trading from a fragmented, inefficient, relationship-based activity to a tightly connected endeavor dominated by computers and algorithms. Firms like Knight, which once used floor traders and phones to execute trades, had to adapt to a new world. (Page 42)
    • This is an important point. There is short-term survival value in becoming homogeneous and tightly connected. Diversity only helps in the long run.
  • As the crew battled the blowout, complexity struck again. The rig’s elaborate emergency systems were just too overwhelming. There were as many as thirty buttons to control a single safety system, and a detailed emergency handbook described so many contingencies that it was hard to know which protocol to follow. When the accident began, the crew was frozen. The Horizon’s safety systems paralyzed them. (Page 49)
    • I think that this may argue opposite to the authors’ point. The complexity here is a form of diversity. The safety system was a high-dimensional system that required an effective user to be aligned with it, like a free climber on a cliff face. A user highly educated in the system could probably have made it work, even better than a big STOP button. But expecting that user is a mistake. The authors actually discuss this later when they describe how safety training was reduced to simple practices that ignored perceived unlikely catastrophic events.
  • “The real threat,” Greenberg explained, “comes from malicious actors that connect things together. They use a chain of bugs to jump from one system to the next until they achieve full code execution.” In other words, they exploit complexity: they use the connections in the system to move from the software that controls the radio and GPS to the computers that run the car itself. “As cars add more features,” Greenberg told us, “there are more opportunities for abuse.” And there will be more features: in driverless cars, computers will control everything, and some models might not even have a steering wheel or brake pedal. (Page 60)
    • In this case it’s not the stiffness of the connections, its the density of connections
  • Attacks on cars, ATMs, and cash registers aren’t accidents. But they, too, originate from the danger zone. Complex computer programs are more likely to have security flaws. Modern networks are rife with interconnections and unexpected interactions that attackers can exploit. And tight coupling means that once a hacker has a foothold, things progress swiftly and can’t easily be undone. In fact, in all sorts of areas, complexity creates opportunities for wrongdoing, and tight coupling amplifies the consequences. It’s not just hackers who exploit the danger zone to do wrong; it’s also executives at some of the world’s biggest companies. (Page 62)
  • By the year 2000, Fastow and his predecessors had created over thirteen hundred specialized companies to use in these complicated deals. “Accounting rules and regulations and securities laws and regulation are vague,” Fastow later explained. “They’re complex. . . . What I did at Enron and what we tended to do as a company [was] to view that complexity, that vagueness . . . not as a problem, but as an opportunity.” Complexity was an opportunity. (Page 69)
    • I’m not sure how to fit this in, but I think there is something here about high-dimensional spaces being essentially invisible. This is the same thing as the safety system on the Deepwater Horizon.
  • But like the core of a nuclear power plant, the truth behind such writing is difficult to observe. And research shows that unobservability is a key ingredient to news fabrications. Compared to genuine articles, falsified stories are more likely to be filed from distant locations and to focus on topics that lend themselves to the use of secret sources, such as war and terrorism; they are rarely about big public events like baseball games. (Page 77)
    •  More heuristics for map building
  • Charles Perrow once wrote that “safety systems are the biggest single source of catastrophic failure in complex, tightly coupled systems.” (Page 85)
    • Dimensions reduce through use, which is a kind of conversation between the users and the designers. Safety systems are rarely used, so this conversation doesn’t happen.
  • Perrow’s matrix is helpful even though it doesn’t tell us what exactly that “crazy failure” will look like. Simply knowing that a part of our system—or organization or project—is vulnerable helps us figure out if we need to reduce complexity and tight coupling and where we should concentrate our efforts. It’s a bit like wearing a seatbelt. The reason we buckle up isn’t that we have predicted the exact details of an impending accident and the injuries we’ll suffer. We wear seatbelts because we know that something unforeseeable might happen. We give ourselves a cushion of time when cooking an elaborate holiday dinner not because we know what will go wrong but because we know that something will. “You don’t need to predict it to prevent it,” Miller told us. “But you do need to treat complexity and coupling as key variables whenever you plan something or build something.” (Page 88)
  • A fundamental feature of complex systems is that we can’t find all the problems by simply thinking about them. Complexity can cause such strange and rare interactions that it’s impossible to predict most of the error chains that will emerge. But before they fall apart, complex systems give off warning signs that reveal these interactions. The systems themselves give us clues as to how they might unravel. (Page 141)
  • Over the course of several years, Rerup conducted an in-depth study of global pharmaceutical powerhouse Novo Nordisk, one of the world’s biggest insulin producers. In the early 1990s, Rerup found, it was difficult for anyone at Novo Nordisk to draw attention to even serious threats. “You had to convince your own boss, his boss, and his boss that this was an issue,” one senior vice president explained. “Then he had to convince his boss that it was a good idea to do things in a different way.” But, as in the childhood game of telephone—where a message gets more and more garbled as it passes between people—the issues became oversimplified as they worked their way up the chain of command. “What was written in the original version of the report . . . and which was an alarm bell for the specialist,” the CEO told Rerup, “was likely to be deleted in the version that senior management read.” (Page 146)
    • Dimension reduction, leading to stampede
  • Once an issue has been identified, the group brings together ad hoc teams from different departments and levels of seniority to dig into how it might affect their business and to figure out what they can do to prevent problems. The goal is to make sure that the company doesn’t ignore weak signs of brewing trouble.  (Page 147)
    • Environmental awareness as a deliberate counter to dimension reduction
  • We show that a deviation from the group opinion is regarded by the brain as a punishment,” said the study’s lead author, Vasily Klucharev. And the error message combined with a dampened reward signal produces a brain impulse indicating that we should adjust our opinion to match the consensus. Interestingly, this process occurs even if there is no reason for us to expect any punishment from the group. As Klucharev put it, “This is likely an automatic process in which people form their own opinion, hear the group view, and then quickly shift their opinion to make it more compliant with the group view.” (Page 154)
    • Reinforcement Learning Signal Predicts Social Conformity
      • Vasily Klucharev
      • We often change our decisions and judgments to conform with normative group behavior. However, the neural mechanisms of social conformity remain unclear. Here we show, using functional magnetic resonance imaging, that conformity is based on mechanisms that comply with principles of reinforcement learning. We found that individual judgments of facial attractiveness are adjusted in line with group opinion. Conflict with group opinion triggered a neuronal response in the rostral cingulate zone and the ventral striatum similar to the “prediction error” signal suggested by neuroscientific models of reinforcement learning. The amplitude of the conflict-related signal predicted subsequent conforming behavioral adjustments. Furthermore, the individual amplitude of the conflict-related signal in the ventral striatum correlated with differences in conforming behavior across subjects. These findings provide evidence that social group norms evoke conformity via learning mechanisms reflected in the activity of the rostral cingulate zone and ventral striatum.
  • When people agreed with their peers’ incorrect answers, there was little change in activity in the areas associated with conscious decision-making. Instead, the regions devoted to vision and spatial perception lit up. It’s not that people were consciously lying to fit in. It seems that the prevailing opinion actually changed their perceptions. If everyone else said the two objects were different, a participant might have started to notice differences even if the objects were identical. Our tendency for conformity can literally change what we see. (Page 155)
    • Gregory Berns
      • Dr. Berns specializes in the use of brain imaging technologies to understand human – and now, canine – motivation and decision-making.  He has received numerous grants from the National Institutes of Health, National Science Foundation, and the Department of Defense and has published over 70 peer-reviewed original research articles.
    • Neurobiological Correlates of Social Conformity and Independence During Mental Rotation
      • Background: When individual judgment conflicts with a group, the individual will often conform his judgment to that of the group. Conformity might arise at an executive level of decision making, or it might arise because the social setting alters the individual’s perception of the world.
      • Methods: We used functional magnetic resonance imaging and a task of mental rotation in the context of peer pressure to investigate the neural basis of individualistic and conforming behavior in the face of wrong information.
      • Results: Conformity was associated with functional changes in an occipital-parietal network, especially when the wrong information originated from other people. Independence was associated with increased amygdala and caudate activity, findings consistent with the assumptions of social norm theory about the behavioral saliency of standing alone.
      • Conclusions: These findings provide the first biological evidence for the involvement of perceptual and emotional processes during social conformity.
      • The Pain of Independence: Compared to behavioral research of conformity, comparatively little is known about the mechanisms of non-conformity, or independence. In one psychological framework, the group provides a normative influence on the individual. Depending on the particular situation, the group’s influence may be purely informational – providing information to an individual who is unsure of what to do. More interesting is the case in which the individual has definite opinions of what to do but conforms due to a normative influence of the group due to social reasons. In this model, normative influences are presumed to act through the aversiveness of being in a minority position
    • A Neural Basis for Social Cooperation
      • Cooperation based on reciprocal altruism has evolved in only a small number of species, yet it constitutes the core behavioral principle of human social life. The iterated Prisoner’s Dilemma Game has been used to model this form of cooperation. We used fMRI to scan 36 women as they played an iterated Prisoner’s Dilemma Game with another woman to investigate the neurobiological basis of cooperative social behavior. Mutual cooperation was associated with consistent activation in brain areas that have been linked with reward processing: nucleus accumbens, the caudate nucleus, ventromedial frontal/orbitofrontal cortex, and rostral anterior cingulate cortex. We propose that activation of this neural network positively reinforces reciprocal altruism, thereby motivating subjects to resist the temptation to selfishly accept but not reciprocate favors.
  • These results are alarming because dissent is a precious commodity in modern organizations. In a complex, tightly coupled system, it’s easy for people to miss important threats, and even seemingly small mistakes can have huge consequences. So speaking up when we notice a problem can make a big difference. (Page 155)
  • KRAWCHECK: I think when you get diverse groups together who’ve got these different backgrounds, there’s more permission in the room—as opposed to, “I can’t believe I don’t understand this and I’d better not ask because I might lose my job.” There’s permission to say, “I come from someplace else, can you run that by me one more time?” And I definitely saw that happen. But as time went on, the management teams became less diverse. And in fact, the financial services industry went into the downturn white, male and middle aged. And it came out whiter, maler and middle-aged-er. (Page 176)
  • “The diverse markets were much more accurate than the homogeneous markets,” said Evan Apfelbaum, an MIT professor and one of the study’s authors. “In homogeneous markets, if someone made a mistake, then others were more likely to copy it,” Apfelbaum told us. “In diverse groups, mistakes were much less likely to spread.” (Page 177)
  • Having minority traders wasn’t valuable because they contributed unique perspectives. Minority traders helped markets because, as the researchers put it, “their mere presence changed the tenor of decision making among all traders.” In diverse markets, everyone was more skeptical. (Page 178)
  • In diverse groups, we don’t trust each other’s judgment quite as much, and we call out the naked emperor. And that’s very valuable when dealing with a complex system. If small errors can be fatal, then giving others the benefit of the doubt when we think they are wrong is a recipe for disaster. Instead, we need to dig deeper and stay critical. Diversity helps us do that. (Page 180)
  • Ironically, lab experiments show that while homogeneous groups do less well on complex tasks, they report feeling more confident about their decisions. They enjoy the tasks they do as a group and think they are doing well. (Page 182)
    • Another stampede contribution
  • The third issue was the lack of productive conflict. When amateur directors were just a small minority on a board, it was hard for them to challenge the experts. On a board with many bankers, one CEO told the researchers, “Everybody respects each other’s ego at that table, and at the end of the day, they won’t really call each other out.” (Page 193)
    • Need to figure out what productive conflict is and how to measure it
  • Diversity is like a speed bump. It’s a nuisance, but it snaps us out of our comfort zone and makes it hard to barrel ahead without thinking. It saves us from ourselves. (Page 197)
  • A stranger is someone who is in a group but not of the group. Simmel’s archetypal stranger was the Jewish merchant in a medieval European town—someone who lived in the community but was different from the insiders. Someone close enough to understand the group, but at the same time, detached enough to have an outsider’s perspective. (Page 199)
    • Can AI be trained to be a stranger?
  • But Volkswagen didn’t just suffer from an authoritarian culture. As a corporate governance expert noted, “Volkswagen is well known for having a particularly poorly run and structured board: insular, inward-looking, and plagued with infighting.” On the firm’s twenty-member supervisory board, ten seats were reserved for Volkswagen workers, and the rest were split between senior managers and the company’s largest shareholders. Both Piëch and his wife, a former kindergarten teacher, sat on the board. There were no outsiders. This kind of insularity went well beyond the boardroom. As Milne put it, “Volkswagen is notoriously anti-outsider in terms of culture. Its leadership is very much homegrown.” And that leadership is grown in a strange place. Wolfsburg, where Volkswagen has its headquarters, is the ultimate company town. “It’s this incredibly peculiar place,” according to Milne. “It didn’t exist eighty years ago. It’s on a wind-swept plain between Hanover and Berlin. But it’s the richest town in Germany—thanks to Volkswagen. VW permeates everything. They’ve got their own butchers, they’ve got their own theme park; you don’t escape VW there. And everybody comes through this system.” (Page 209)
  • Most companies have lots of people with different skills. The problem is, when you bring people together to work on the same problem, if all they have are those individual skills . . . it’s very hard for them to collaborate. What tends to happen is that each individual discipline represents its own point of view. It basically becomes a negotiation at the table as to whose point of view wins, and that’s when you get gray compromises where the best you can achieve is the lowest common denominator between all points of view. The results are never spectacular but, at best, average. (Page 236)
    • The idea here is that there is either total consensus and groupthink, or grinding compromise. The authors are focussing too much on the ends of the spectrum. The environmentally aware, social middle is the sweet spot where flocking occurs.
  • Or think about driverless cars. They will almost certainly be safer than human drivers. They’ll eliminate accidents due to fatigued, distracted, and drunk driving. And if they’re well engineered, they won’t make the silly mistakes that we make, like changing lanes while another car is in our blind spot. At the same time, they’ll be susceptible to meltdowns—brought on by hackers or by interactions in the system that engineers didn’t anticipate. (Page 242)
  • We can design safer systems, make better decisions, notice warning signs, and learn from diverse, dissenting voices. Some of these solutions might seem obvious: Use structured tools when you face a tough decision. Learn from small failures to avoid big ones. Build diverse teams and listen to skeptics. And create systems with transparency and plenty of slack. (Page 242)

The Evolution of Cooperation

The Evolution of Cooperation

Robert Axelrod (Scholar)

  • Robert Axelrod is the Walgreen Professor for the Study of Human Understanding at the University of Michigan. He has appointments in the Department of Political Science and the Gerald R. Ford School of Public Policy. He is best known for his interdisciplinary work on the evolution of cooperation which has been cited more than 30,000 times. His current research interests include international security and sense-making.

Quick Takeaway

  • One of the first computer analysis of using computers to analyze game theory, in this case the Iterated Prisoner’s Dilemma (IPD). The book covers a playoff between contributed algorithms for the IPD, where the victor, surprisingly, was TIT-FOR-TAT (TFT). TFT simply responds back with the last response from the other player, and won out of considerably more complicated algorithms.
  • Based on the outcome of the first playoff, a second competition was held. TFT won this competition as well. The book spends most of its time either bringing the reader up to speed on the IPD or discussing the ramifications. The main point of the book is the surprising robustness of TFT, as examined in a variety of contexts.
  • A thing that strikes me is that once a TFT successfully takes over, then it becomes computationally easier to ALWAYS-COOPERATE. That could evolve to become dominant and be completely vulnerable to ALWAYS-DEFECT


  • By the final chapter, the discussion has developed from the study of the emergence of cooperation among egoists without central authority to an analysis of what happens when people actually do care about each other and what happens when there is central authority. But the basic approach is always the same: seeing how individuals operate in their own interest reveals what happens to the whole group. This approach allows more than the understanding of the perspective of a single player. It also provides an appreciation of what it takes to promote the stability of mutual cooperation in a given setting. The most promising finding is that if the facts of Cooperation Theory are known by participants with foresight, the evolution of cooperation can be speeded up. (Page 24)
  • A computer tournament for the study of effective choice in the iterated Prisoner’s Dilemma meets these needs. In a computer tournament, each entrant writes a program that embodies a rule to select the cooperative or noncooperative choice on each move. The program has available to it the history of the game so far, and may use this history in making a choice. If the participants are recruited primarily from those who are familiar with the Prisoner’s Dilemma, the entrants can be assured that their decision rule will be facing rules of other informed entrants. Such recruitment would also guarantee that the state of the art is represented in the tournament. (Page 30)
  • The sample program sent to prospective contestants to show them how to make a submission would in fact have won the tournament if anyone had simply clipped it and mailed it in! But no one did. The sample program defects only if the other player defected on the previous two moves. It is a more forgiving version of TIT FOR TAT in that it does not punish isolated defections. The excellent performance of this TIT FOR TWO TATS rule highlights the fact that a common error of the contestants was to expect that gains could be made from being relatively less forgiving than TIT FOR TAT, whereas in fact there were big gains to be made from being even more forgiving. The implication of this finding is striking, since it suggests that even expert strategists do not give sufficient weight to the importance of forgiveness. (Page 39)
  • The second round was also a dramatic improvement over the first round in sheer size of the tournament. The response was far greater than anticipated. There was a total of sixty-two entries from six countries. The contestants were largely recruited through announcements in journals for users of small computers. The game theorists who participated in the first round of the tournament were also invited to try again. The contestants ranged from a ten-year-old computer hobbyist to professors of computer science,.physics, economics, psychology, mathematics, sociology, political science, and evolutionary biology. The countries represented were the United States, Canada, Great Britain, Norway, Switzerland, and New Zealand. (Page 41)
    • A nice argument for and example of diversity.
  • A good way to examine this question is to construct a series of hypothetical tournaments, each with a very different distribution of the types of rules participating. The method of constructing these drastically modified tournaments is explained in appendix A. The results were that TIT FOR TAT won five of the six major variants of the tournament, and came in second in the sixth. This is a strong test of how robust the success of TIT FOR TAT really is. (Page 48)
  • EoC pg51.PNG
  • This simulation provides an ecological perspective because there are no new rules of behavior introduced. It differs from an evolutionary perspective, which would allow mutations to introduce new strategies into the environment. In the ecological perspective there is a changing distribution of given types of rules. The less successful rules become less common and the more successful rules proliferate. (Page 51)
  • This approach imagines the existence of a whole population of individuals employing a certain strategy, and a single mutant individual employing a different strategy. The mutant strategy is said to invade the population if the mutant can get a higher payoff than the typical member of the population gets. Put in other terms, the whole population can be imagined to be using a single strategy, while a single individual enters the population with a new strategy. The newcomer will then be interacting only with individuals using the native strategy. Moreover, a native will almost certainly be interacting with another native since the single newcomer is a negligible part of the population. Therefore a new strategy is said to invade a native strategy if the newcomer gets a higher score with a native than a native gets with another native. Since the natives are virtually the entire population, the concept of invasion is equivalent to the single mutant individual being able to do better than the population average. This leads directly to the key concept of the evolutionary approach. A strategy is collectively stable if no strategy can invade it. (Page 56)
    • We need to be careful here as this is a very simple fitness test. Add more conditions, like computational cost, and the fitness landscape becomes multidimensional and potentially more rugged.
  • Proposition 2. TIT FOR TAT is collectively stable if and only if, w is large enough. This critical value of w is a function of the four payoff parameters, T (temptation), R (reward), P (punishment), and S (sucker’s) . The significance of this proposition is that if everyone in a population is cooperating with everyone else because each is using the TIT FOR TAT strategy, no one can do better using any other strategy providing that the future casts a large enough shadow onto the present. In other words, what makes it impossible for TIT FOR TAT to be invaded is that the discount parameter, w, is high enough relative to the requirement determined by the four payoff parameters. For example, suppose that T=S, R=3, P= 1, and S=O as in the payoff matrix shown in figure 1. Then TIT FOR TAT is collectively stable if the next move is at least 2/3 as important as the current move. Under these conditions, if everyone else is using TIT FOR TAT, you can do no better than to do the same, and cooperate with them. On the other hand, if w falls below this critical value, and everyone else is using TIT FOR TAT, it will pay to defect on alternative moves. If w is less than 1/2, it even pays to always defect. (Page 59)
    • From Wikipedia: If both players cooperate, they both receive the reward R for cooperating. If both players defect, they both receive the punishment payoff P. If Blue defects while Red cooperates, then Blue receives the temptation payoff T, while Red receives the “sucker’s” payoff, S. Similarly, if Blue cooperates while Red defects, then Blue receives the sucker’s payoff S, while Red receives the temptation payoff T.
  • One specific implication is that if the other player is unlikely to be around much longer because of apparent weakness, then the perceived value of w falls and the reciprocity of TIT FOR TAT is no longer stable. We have Caesar’s explanation of why Pompey’s allies stopped cooperating with him. “They regarded his (Pompey’s] prospects as hopeless and acted according to the common rule by which a man’s friends become his enemies in adversity” (Page 59)
  • Proposition 4. :For a nice strategy to be collectively stable, it must be provoked by the very first defection of the other player. The reason is simple enough. If a nice strategy were not provoked by a defection on move n, then it would not be collectively stable because it could be invaded by a rule which defected only on move n. (Page 62)
  • Many of the benefits sought by living things are disproportionately available to cooperating groups. While there are considerable differences in what is meant by the terms “benefits” and “sought,” this statement, insofar as it is true, lays down a fundamental basis for all social life. The problem is that while an individual can benefit from mutual cooperation, each one can also do even better by exploiting the cooperative efforts of others. Over a period of time, the same individuals may interact again, allowing for complex patterns of strategic interactions. As the earlier chapters have shown, the Prisoner’s Dilemma allows a formalization of the strategic possibilities inherent in such situations. (Page 92)
    • And how groups are structured and connected affect this tremendously.
  • Non-zero-sum games, such as the Prisoner’s Dilemma, are not like this. Unlike the clouds, the other player can respond to your own choices. And unlike the chess opponent, the other player in a Prisoner’s Dilemma should not be regarded as someone who is out to defeat you. The other player will be watching your behavior for signs of whether you will reciprocate cooperation or not, and therefore your own behavior is likely to be echoed back to you. (Page 121)
    • There is a fundamental difference between interacting with the external environment. The environment is somewhat immutable, and the individual must adapt to it. This adaptation can be simple (don’t jump off the cliff) or complicated (free-climbing the same cliff). The more complicated the adaptation, the more difficult it is for the agent to do anything else. Each problem we encounter, social or environmental has a cognitive cost that we pay out of our limited cognitive budget.
  • Having a firm reputation for using TIT FOR TAT is advantageous to a player, but it is not actually the best reputation to have. The best reputation to have is the reputation for being a bully. The best kind of bully to be is one who has a reputation for squeezing the most out of the other player while not tolerating any defections at all from the other. The way to squeeze the most out of the other is to defect so often that the other player just barely prefers cooperating all the time to defecting all the time. And the best way to encourage cooperation from the other is to be known as someone who will never cooperate again if the other defects even once. (Page 152)
    • This is the most interesting description of Donald Trump that I have ever read.
  • The “neighbors” of this soft drink are other drinks on the market with a little more or less sugar, or a little more or less caffeine. Similarly, a Political candidate might take a position on a liberal/conservative dimension and a position on an internationalism/isolationism dimension. If there are many candidates vying with each other in an election, the “neighbors” of the candidate are those with similar positions. Thus territories can be abstract spaces as well as geographic spaces. (Page 159)
    • Look! Belief spaces!

Thinking slow, acting reflexively

I just finished the cover story in Communications of the ACM on Human-Level Intelligence or Animal-Like Abilities?. Overall interesting and insightful, but what really caught my eye was Adnan Darwiche‘s discussion of models and maps:

  • “In his The Book of Why: The New Science of Cause and Effect, Judea Pearl explained further the differences between a (causal) model and a function, even though he did not use the term “function” explicitly. In Chapter 1, he wrote: “There is only one way a thinking entity (computer or human) can work out what would happen in multiple scenarios, including some that it has never experienced before. It must possess, consult, and manipulate a mental causal model of that reality.” He then gave an example of a navigation system based on either reasoning with a map (model) or consulting a GPS system that gives only a list of left-right turns for arriving at a destination (function). The rest of the discussion focused on what can be done with the model but not the function. Pearl’s argument particularly focused on how a model can handle novel scenarios (such as encountering roadblocks that invalidate the function recommendations) while pointing to the combinatorial impossibility of encoding such contingencies in the function, as it must have a bounded size.”
  • This is a Lists and Maps argument, and it leaves out stories, but it also implies something powerful that I need to start to think about. There is another interface, and it’s one that bridges human and machine, The dynamic model. What follows is a bunch of (at the moment – 10.8.18) incomplete thoughts. I think that models/games are another sociocultural interface, one that may be as affected by computers as the Ten Blue Links. So I’m using this as a staging area.
  • Games
    • Games and play are probably the oldest form of a dynamic model. Often, and particularly in groups, they are abstract simulations of conflict of some kind. It can be a simple game of skill such as Ringing the Bull, or a complex a wargame, such as chess:
      • “Historically chess must be classed as a game of war. Two players direct a conflict between two armies of equal strength upon a field of battle, circumscribed in extent, and offering no advantage of ground to either side. The players have no assistance other than that afforded by their own reasoning faculties, and the victory usually falls to the one whose strategical imagination is the greater, whose direction of his forces is the more skilful, whose ability to foresee positions is the more developed.” Murray, H.J.R.. A History of Chess: The Original 1913 Edition (Kindle Locations 576-579). Skyhorse Publishing. Kindle Edition.
    • Recently, video games afford games that can follow narrative templates:
      • Person vs. Fate/God
      • Person vs. Self
      • Person vs. Person
      • Person vs Society
      • Person vs. Nature
      • Person vs. Supernatural
      • Person vs. Technology
    • More on this later, because I think that this sort of computer-human interaction is really interesting, because it seems to open up spaces that would not be accessible to humans because of the data manipulation requirements (would flight simulators exist without non-human computation?).
  • Moving Maps
    • I would argue that the closer to interactive rates a model is, the more dynamic it is. A map is a static model, a snapshot of the current geopolitical space. Maps are dynamic because the underlying data is dynamic. Borders shift. Counties come into and go out of existence. Islands are created, and the coastline is eroded. And the next edition of map will incorporate these changes.
    • Online radar weather maps are an interesting case, since they reflect a rapidly changing environment and often now support playback of the last few hours (and prediction for the next few hours) of imagery at variable time scales.
  • Cognition
    • Traditional simulation and humans
      • Simulations provide a mechanism for humans to explore a space of possibilities that larger than what can be accomplished by purely mental means. Further, these simulations create artifacts that can be examined independently by other humans.
        • Every model is a theory—a very-well specified theory. In the case of simulations, the models are theories expressed in so much detail that their consequences can be checked by execution on a computer [Bryson, 2015]
      • The assumptions that provide the basis for the simulation are the model. The computer provides the dynamics. The use of simulation allows users to explore the space in the same way that one would explore the environment. Discoveries can be made that exist outside of the social constructs that led to the construction of the simulator and the assumptions that the simulator is based on.
      • What I think this means is that humans bring meaning to the outputs of the simulation. But it also means that there is a level of friction required to get from the outputs as they are computed to a desired level of meaningfulness to the users. In other words, if you have a theory of galaxy formation, but the results of the simulation only match observations if you have to add something new, like negative gravity, this could reflect a previously undiscovered component in the current theory of the formation of the universe.
      • I think this is the heart of my thinking. Just as maps allow the construction of trajectories across a physical (or belief) spaces, dynamic models such as simulation support ways of evaluating potential (and simplified/general) spaces that exist outside the realms of current understanding. This can be in the form of alternatives not yet encountered (a hurricane will hit the Florida panhandle on Thursday), or systems not yet understood (protein folding interactive simulators)
      • From At Home in the Universe: Physicists roll out this term, “universality class,” to refer to a class of models all of which exhibit the same robust behavior. So the behavior in question does not depend on the details of the model. Thus a variety of somewhat incorrect models of the real world may still succeed in telling us how the real world works, as long as the real world and the models lie in the same universality class. (Page 283)
    • Traditional simulation and ML(models and functions)
      • Darwiche discusses how the ML community has focused on “functional” AI at the expense of  “model-based” AI. I think his insight that functional AI is closer to reflex, and how there is an analogical similarity between it and “thinking fast“. Similarly, he believes that model-based AI may more resemble “thinking slow“.
      • I would contend that building simulators may be the slowest possible thinking. And I wonder if using simulators to train functional AI that can then be evaluated against real-world data, which is then used to modify the model in a “round trip” approach might be a way to use the fundamental understandability of simulation with the reflexive speed of trained NN systems.
      • What this means is that “slow” AI explicitly includes building testable models. The tests are not always going to be confirmation of predictions because of chaos theory. But there can be predictions of the characteristics of a model. For example, I’m working with using agent-based simulation moving in belief space to generate seeds for RNNs to produce strings that resemble conversations. Here, the prediction would be about the “spectral” characteristics of the conversation – how words change over time when compared to actual conversations where consensus evolves over time.

At Home in the Universe: The Search for the Laws of Self-Organization and Complexity

Kauffman’s NK model K large K medium K small fitness distanceFitness landscapes

At Home in the Universe: The Search for the Laws of Self-Organization and Complexity (Kindle Edition)

Stuart Kauffman (Wikipedia)

Quick takeaway:

  • The book’s central thesis is that complexity in general (and life in particular) is an inevitable consequence of self organizing principles that come into play with non-equilibrium systems. He explores the underlying principles in a variety of ways including binary networks, autocatalytic sets, NK models, and fitness landscapes, both static and co-evolving.
  • When I was reading this 20 year old book, I had the impression that his work, particularly on how fitness landscapes are explored have direct relevance to the construction of complex systems today. In particular I was struck by how applicable his work with fitness landscapes and NK models would be to the evaluation of the hyperparameter space associated with building Neural Networks.
  • Another point that I found particularly compelling is his descriptions of the incalculable size of the high-dimension spaces of combinatorial possibility. The number of potential combinations on even a smallish binary network would take more time in than the universe has to calculate. As such, there need to be mechanisms that allow for a faster, “good enough” evaluation of the space. That’s why we have historical narratives. They describe a path through this space that has worked. As an example, compare tic-tac-toe to chess. In the former, the every possibility in the game space can be known. Chess has too many possibilities, so instead there are openings, gambits, and endgames, discovered by chess masters that come to us as stories.


  • Chapter 1: At home in the Universe
    • In all these cases, the order that emerges depends on robust and typical properties of the systems, not on the details of structure and function. Under a vast range of different conditions, the order can barely help but express itself. (page 19)
    • Nonequilibrium ordered systems like the Great Red Spot are sustained by the persistent dissipation of matter and energy, and so were named dissipative structures by the Nobel laureate Ilya Prigogine some decades ago. These systems have received enormous attention. In part, the interest lies in their contrast to equilibrium thermodynamic systems, where equilibrium is associated with collapse to the most probable, least ordered states. In dissipative systems, the flux of matter and energy through the system is a driving force generating order. In part, the interest lies in the awareness that free-living systems are dissipative structures, complex metabolic whirlpools. (page 21).
    • The theory of computation is replete with deep theorems. Among the most beautiful are those showing that, in most cases by far, there exists no shorter means to predict what an algorithm will do than to simply execute it, observing the succession of actions and states as they unfold. The algorithm itself is its own shortest description. It is, in the jargon of the field, incompressible. (page 22)
    • And yet, even if it is true that evolution is such an incompressible process, it does not follow that we may not find deep and beautiful laws governing that unpredictable flow. For we are not precluded from the possibility that many features of organisms and their evolution are profoundly robust and insensitive to details. (page 23)
    • Strikingly, such coevolving systems also behave in an ordered regime, a chaotic regime, and a transition regime. (page 27)
      • Note that this reflects our Nomadic (chaotic), Flocking (transition) and Stampeding (ordered) states
    • This seemingly haphazard process also shows an ordered regime where poor compromises are found quickly, a chaotic regime where no compromise is ever settled on, and a phase transition where compromises are achieved, but not quickly. The best compromises appear to occur at the phase transition between order and chaos. (page 28)
  • Chapter 4: Order for Free
    • But evolution requires more than simply the ability to change, to undergo heritable variation. To engage in the Darwinian saga, a living system must first be able to strike an internal compromise between malleability and stability. To survive in a variable environment, it must be stable, to be sure, but not so stable that it remains forever static. Nor can it be so unstable that the slightest internal chemical fluctuation causes the whole teetering structure to collapse. (page 73)
    • To survive in a variable environment, it must be stable, to be sure, but not so stable that it remains forever static. Nor can it be so unstable that the slightest internal chemical fluctuation causes the whole teetering structure to collapse. (pg 73)
    • It is now well known that in most cells, such molecular feedback can give rise to complex chemical oscillations in time and space. (page 74)
      • Olfati-Saber and graph laplacians!
    • The point in using idealizations in science is that they help capture the main issues. Later one must show that the issues so captured are not altered by removing the idealizations. (page 75)
      • Start with observation, build initial simulation and then measure the difference and modify
    • If started in one state, over time the system will flow through some sequence of states. This sequence is called a trajectory (page 77)
      • I wonder if this can be portrayed as a map? You have to go through one state to get to the next. In autocatalytic systems there may be multiple systems that may be similar and yet have branch points (plant cells, animal cells, bacteria)
    • To answer these questions we need to understand the concept of an attractor. More than one trajectory can flow into the same state cycle. Start a network with any of these different initial patterns and, after churning through a sequence of states, it will settle into the same state cycle, the same pattern of blinking. In the language of dynamical systems, the state cycle is an attractor and the collection of trajectories that flow into it is called the basin of attraction. We can roughly think of an attractor as a lake, and the basin of attraction as the water drainage flowing into that lake. (page 78)
      • Also applicable to social and socio-technical systems. The technology changes the connectivity which could change the shape of the landscape
    • One feature is simply how many “inputs” control any lightbulb. If each bulb is controlled by only one or two other lightbulbs, if the network is “sparsely connected,” then the system exhibits stunning order. If each bulb is controlled by many other light-bulbs, then the network is chaotic. So “tuning” the connectivity of a network tunes whether one finds order or chaos. The second feature that controls the emergence of order or chaos is simple biases in the control rules themselves. Some control rules, the AND and OR Boolean functions we talked about, tend to create orderly dynamics. Other control rules create chaos. (page 80)
      • In our more velocity-oriented system, this is Social Influence Horizon) and is dynamic over time
    • Consider networks in which each lightbulb receives input from only one other. In these K = 1 networks, nothing very interesting happens. They quickly fall into very short state cycles, so short that they often consist of but a single state, a single pattern of illumination. Launch such a K = 1 network and it freezes up, saying the same thing over and over for all time. (page 81)
    • At the other end of the scale, consider networks in which K = N, meaning that each lightbulb receives an input from all lightbulbs, including itself. One quickly discovers that the length of the networks’ state cycles is the square root of the number of states. Consider the implications. For a network with only 200 binary variables—bulbs that can be on or off—there are 2200 or 1060 possible states. (page 81)
    • Such K = N networks do show signs of order, however. The number of attractors in a network, the number of lakes, is only N/e, where e is the basis of the natural logarithms, 2.71828. So a K = N network with 100,000 binary variables would harbor about 37,000 of these attractors. Of course, 37,000 is a big number, but very very much smaller than 2100,000, the size of its state space. (page 82)
      • Need to look into if there is some kind of equivalent in the SIH settings
    • The order arises, sudden and stunning, in K = 2 networks. For these well-behaved networks, the length of state cycles is not the square root of the number of states, but, roughly, the square root of the number of binary variables. Let’s pause to translate this as clearly as we can. Think of a randomly constructed Boolean network with N = 100,000 lightbulbs, each receiving K = 2 inputs. The “wiring diagram” would look like a madhatterly scrambled jumble, an impenetrable jungle. Each lightbulb has also been assigned at random a Boolean function. The logic is, therefore, a similar mad scramble, haphazardly assembled, mere junk. The system has 2100,000 or 1030,000 states—megaparsecs of possibilities—and what happens? The massive network quickly and meekly settles down and cycles among the square root of 100,000 states, a mere 317. (page 83)
    • The reason complex systems exist on, or in the ordered regime near, the edge of chaos is because evolution takes them there. (page 89)
  • Chapter 5: The Mystery of Ontology
    • Another way to ensure orderly behavior is to construct networks using what are called canalyzing Boolean functions. These Boolean rules have the easy property that at least one of the molecular inputs has one value, which might be 1 or 0, which by itself can completely determine the response of the regulated gene. The OR function is an example of a canalyzing function (Figure 5.3a). An element regulated by this function is active at the next moment if its first, or its second, or both inputs are active at the current moment. Thus if the first input is active, then the regulated element is guaranteed to be active at the next moment, regardless of the activity of the second input. (page 103)
      • This is max pooling
    • For most perturbations, a genomic system on any attractor will exhibit homeostatic return to the same attractor. The cell types are fundamentally stable. But for some perturbations, the system flows to a different attractor. So differentiation occurs naturally. And the further critical property is this: from any one attractor, it is possible to undergo transitions to only a few neighboring attractors, and from them other perturbations drive the system to still other attractors. Each lake, as it were, is close to only a few other lakes. (page 110)
  • Chapter 6: Noah’s Vessel
    • That we eat our meals rather than fusing with them marks, I believe, a profound fact. The biosphere itself is supracritical. Our cells are just subcritical. Were we to fuse with the salad, the molecular diversity this fusion would engender within our cells would unleash a cataclysmic supracritical explosion. The explosion of molecular novelty would soon be lethal to the unhappy cells harboring the explosion. The fact that we eat is not an accident, one of many conceivable methods evolution might have alighted on to get new molecules into our metabolic webs. Eating and digestion, I suspect, reflect our need to protect ourselves from the supracritical molecular diversity of the biosphere. (page 122)
    • We may be discovering a universal in biology, a new law: if our cells are subcritical, then, presumably, so too are all cells—bacteria, bracken, fern, bird, man. Throughout the supracritical explosion of the biosphere, cells since the Paleozoic, cells since the start, cells since 3.45 billion years ago must have remained subcritical. If so, then this subcritical–supracritical boundary must have always set an upper limit on the molecular diversity that can be housed within one cell. A limit exists, then, on the molecular complexity of the cell. (page 126)
    • If local ecosystems are metabolically poised at the subcritical–supracritical boundary, while the biosphere as a whole is supracritical? Then what a new tale we tell, of life cooperating to beget ever new kinds of molecules, and a biosphere where local ecosystems are poised at the boundary, but have collectively crept slowly upward in total diversity by the supracritical character of the whole planet. The whole biosphere is broadly collectively autocatalytic, catalyzing its own maintenance and ongoing molecular exploration. (page 130)
  • Chapter 8: High-Country Adventures
    • what would happen if, in addition to attempting to evolve such a computer program, we were more ambitious and attempted to evolve the shortest possible program that will carry out the task? Such a “shortest program” is one that is maximally compressed; that is, all redundancies have been squeezed out of it. Evolving a serial computer program is either very hard or essentially impossible because it is incredibly fragile. Serial computer programs contain instructions such as “compare two numbers and do such and such depending on which is larger” or “repeat the following action 1,000 times.” The computation performed is extremely sensitive to the order in which actions are carried out, the precise details of the logic, numbers of iterations, and so forth. The result is that almost any random change in a computer program produces “garbage.” Familiar computer programs are precisely the kind of complex systems that do not have the property that small changes in structure yield small changes in behavior. Almost all small changes in structure lead to catastrophic changes in behavior. (page 152)
      • This is the inherent problem we are grappling with in our “barely controlled systems”. All the elements involved are brittle and un-evolvable
    • It seems likely that there is no way to evolve a maximally compressed program in less time than it would take to exhaustively generate all possible programs, testing each to see if it carries out the desired task. When all redundancy has been squeezed from a program, virtually any change in any symbol would be expected to cause catastrophic variation in the behavior of the algorithm. Thus nearby variants in the program compute very different algorithms. (page 154)
    • because the program is maximally compressed, any change will cause catastrophic alterations in the computation performed. The fitness landscape is entirely random. The next fact is this: the landscape has only a few peaks that actually perform the desired algorithm. In fact, it has recently been shown by the mathematician Gregory Chaitin that for most problems there is only one or, at most, a few such minimal programs. It is intuitively clear that if the landscape is random, providing no clues about good directions to search, then at best the search must be a random or systematic search of all the 10300 possible programs to find the needle in the haystack, the possibly unique minimal program. This is just like finding Mont Blanc by searching every square meter of the Alps; the search time is, at best, proportional to the size of the program space. (page 155)
      • I’ve been thinking about hyperparameter tuning in the wrong way. There need(?) to be two approaches – one that works in evolvable spaces where there can be gradualism. The other approach cas to work in discontinuous regions, such as what activation function to use.
    • The question of what kinds of complex systems can be assembled by an evolutionary search process not only is important for understanding biology, but may be of practical importance in understanding technological and cultural evolution as well. The sensitivity of our most complex artifacts to catastrophic failure from tiny causes—for example, the Challenger disaster, the failed Mars Observer mission, and power-grid failures affecting large regions—suggests that we are now butting our heads against a problem that life has nuzzled for enormously longer periods: how to produce complex systems that do not teeter on the brink of collapse. Perhaps general principles governing search in vast spaces of possibilities cover all these diverse evolutionary processes, and will help us design—or even evolve—more robust systems. (page 157)
    • Once we understand the nature of these random landscapes and evolution on them, we will better appreciate what it is about organisms that is different, how their landscapes are nonrandom, and how that nonrandomness is critical to the evolutionary assembly of complex organisms. We will find reasons to believe that it is not natural selection alone that shapes the biosphere. Evolution requires landscapes that are not random. The deepest source of such landscapes may be the kind of principles of self-organization that we seek. (page 165)
    • On random landscapes, finding the global peak by searching uphill is totally useless; we have to search the entire space of possibilities. But even for modestly complex genotypes, or programs, that would take longer than the history of the universe. (page 167)
    • Things capable of evolving—metabolic webs of molecules, single cells, multicellular organisms, ecosystems, economic systems, people—all live and evolve on landscapes that themselves have a special property: they allow evolution to “work.” These real fitness landscapes, the types that underlie Darwin’s gradualism, are “correlated.” Nearby points tend to have similar heights. The high points are easier to find, for the terrain offers clues about the best directions in which to proceed. (page 169)
    • In short, the contribution to overall fitness of the organism of one state of one trait may depend in very complex ways on the states of many other traits. Similar issues arise if we think of a haploid genotype with N genes, each having two alleles. The fitness contribution of one allele of one gene to the whole organism may depend in complex ways on the alleles of other genes. Geneticists call this coupling between genes epistasis or epistatic coupling, meaning that genes at other places on the chromosomes affect the fitness contribution of a gene at a given place. (page 170)
    • The NK model captures such networks of epistatic couplings and models the complexity of the coupling effects. It models epistasis itself by assigning to each trait, or gene, epistatic “inputs” from K other traits or genes. Thus the fitness contribution of each gene depends on the gene’s own allele state, plus the allele states of the K other genes that affect that gene. (page 171)
    • I find the NK model fascinating because of this essential point: altering the number of epistatic inputs per gene, K, alters the ruggedness and number of peaks on the landscape. Altering K is like twisting a control knob. (page 172)
      •  This is really important and should also work with graph laplacians. In other words, not only can we model the connectivity, we can model the stiffness
    • our model organism, with its network of epistatic interactions among its genes, is caught in a web of conflicting constraints. The higher K is—the more interconnected the genes are—the more conflicting constraints exist, so the landscape becomes ever more rugged with ever more local peaks. (page 173)
      • This sounds oddly like how word2vec is calculated. Which implies that all connected neural networks are correlated and epistatic.
    • It is these conflicting constraints that make the landscape rugged and multipeaked. Because so many constraints are in conflict, there is a large number of rather modest compromise solutions rather than an obvious superb solution. (page 173)
      • Dimension reduction and polarization are a social solution to this problem
    • landscapes with moderate degrees of ruggedness share a striking feature: it is the highest peaks that can be scaled from the greatest number of initial positions! This is very encouraging, for it may help explain why evolutionary search does so well on this kind of landscape. On a rugged (but not random) landscape, an adaptive walk is more likely to climb to a high peak than a low one. If an adapting population were to “jump” randomly into such a landscape many times and climb uphill each time to a peak, we would find that there is a relationship between how high the peak is and how often the population climbed to it. If we turned our landscapes upside down and sought instead the lowest valleys, we would find that the deepest valleys drain the widest basins. (page 177)
    • The property that the highest peaks are the ones to which the largest fraction of genotypes can climb is not inevitable. The highest peaks could be very narrow but very high pinnacles on a low-lying landscape with modest broad hilltops. If an adapting population were released at a random spot and walked uphill, it would then find itself trapped on the top of a mere local hilltop. The exciting fact we have just discovered is that for an enormous family of rugged landscapes, the NK family, the highest peaks “drain” the largest basins. This may well be a very general property of most rugged landscapes reflecting complex webs of conflicting constraints. (page 177)
      •  I think this may be a function of how the landscapes are made. The K in NK somewhat dictates the amount of correlation
    • Recall another striking feature of random landscapes: with every step one takes uphill, the number of directions leading higher is cut by a constant fraction, one-half, so it becomes ever harder to keep improving. As it turns out, the same property shows up on almost any modestly rugged or very rugged landscape. Figure 8.9 shows the dwindling fraction of fitter neighbors along adaptive walks for different K values (Figure 8.9a) and the increased waiting times to find fitter variants for different K values (Figure 8.9b). Once K is modestly large, about K = 8 or greater, at each step uphill the number of directions uphill falls by a constant fraction, and the waiting time or number of tries to find that way uphill increases by a constant fraction. This means that as one climbs higher and higher, it becomes not just harder, but exponentially harder to find further directions uphill. So if one can make one try per unit time, the rate of improving slows exponentially. (page 178)
      • This is very important in understanding how hyperparameter space needs to be explored
    • Optimal solutions to one part of the overall design problem conflict with optimal solutions to other parts of the overall design. Then we must find compromise solutions to the joint problem that meet the conflicting constraints of the different subproblems. (page 179)
    • Selection, in crafting the kinds of organisms that exist, may also help craft the kinds of landscapes over which they evolve, picking landscapes that are most capable of supporting evolution—not only by mutation alone, but by recombination as well. Evolvability itself is a triumph. To benefit from mutation, recombination, and natural selection, a population must evolve on rugged but “well-correlated” landscapes. In the framework of NK landscapes, the “K knob” must be well tuned. (page 182)
      • This is going to be the trick for machine learning
    • even if the population is released on a local peak, it may not stay there! Simply put, the rate of mutation is so high that it causes the population to “diffuse” away from the peak faster than the selective differences between less fit and more fit mutants can return the population to the peak. An error catastrophe, first discovered by Nobel laureate Manfred Eigen and theoretical chemist Peter Schuster, has occurred, for the useful genetic information built up in the population is lost as the population diffuses away from the peak. (page 184)
    • Eigen and Schuster were the first to emphasize the importance of this error catastrophe, for it implies a limit to the power of natural selection. At a high enough mutation rate, an adapting population cannot assemble useful genetic variants into a working whole; instead, the mutation-induced “diffusion” over the space overcomes selection, pulling the population toward adaptive peaks. (page 184)
    • This limitation is even more marked when seen from another vantage point. Eigen and Schuster also emphasized that for a constant mutation rate per gene, the error catastrophe will arise when the number of genes in the genotype increases beyond a critical number. Thus there appears to be a limit on the complexity of a genome that can be assembled by mutation and selection! (page 184)
    • We are seeking a new conceptual framework that does not yet exist. Nowhere in science have we an adequate way to state and study the interleaving of self-organization, selection, chance, and design. We have no adequate framework for the place of law in a historical science and the place of history in a lawful science. (page 185)
      • This is the research part of the discussion in the iConference paper. Use the themes in the following paragraphs (self organization, selection, etc. ) to build up the areas that need to be discussed and researched.
    • The inevitability of historical accident is the third theme. We can have a rational morphology of crystals, because the number of space groups that atoms in a crystal can occupy is rather limited. We can have a periodic table of the elements because the number of stable arrangements of the subatomic constituents is relatively limited. But once at the level of chemistry, the space of possible molecules is vaster than the number of atoms in the universe. Once this is true, it is evident that the actual molecules in the biosphere are a tiny fraction of the space of the possible. Almost certainly, then, the molecules we see are to some extent the results of historical accidents in this history of life. History arises when the space of possibilities is too large by far for the actual to exhaust the possible. (page 186)
    • Here is a firm foothold: an evolutionary process, to be successful, requires that the landscapes it searches are more or less correlated. (page 186)
      • This is a meta design constraint that needs to be discussed (iConference? Antonio’s workshop?)
    • Nonequilibrium systems can be robust as well. A whirlpool dissipative system is robust in the sense that a wide variety of shapes of the container, flow rates, kinds of fluids, and initial conditions of the fluids lead to vortices that may persist for long periods. So small changes in the construction parameters of the system, and initial conditions, lead to small changes in behavior. (page 187)
    • Whirlpools are attractors in a dynamical system. Attractors, however, can be both stable and unstable. Instability arises in two senses. First, small changes in the construction of the system may dramatically alter the behavior of the system. Such systems are called structurally unstable. In addition, small changes in initial conditions, the butterfly effect, can sharply change subsequent behavior. Conversely, stable dynamical systems can be stable in both senses. Small changes in construction may typically lead to small changes in behavior. The system is structurally stable. And small changes in initial conditions can lead to small changes in behavior. (page 187)
    • We know that there is a clear link between the stability of the dynamical system and the ruggedness of the landscape over which it adapts. Chaotic Boolean networks, and many other classes of chaotic dynamical systems, are structurally unstable. Small changes wreak havoc on their behavior. Such systems adapt on very rugged landscapes. In contrast, Boolean networks in the ordered regime are only slightly modified by mutations to their structure. These networks adapt on relatively smooth fitness landscapes. (page 187)
    • We know from the NK landscape models discussed in this chapter that there is a relationship between the richness of conflicting constraints in a system and the ruggedness of the landscape over which it must evolve. We plausibly believe that selection can alter organisms and their components so as to modify the structure of the fitness landscapes over which those organisms evolve. By taking genomic networks from the chaotic to the ordered regime, selection tunes network behavior to be sure. By tuning epistatic coupling of genes, selection also tunes landscape structure from rugged to smooth. Changing the level of conflicting constraints in the construction of an organism from low to high tunes how rugged a landscape such organisms explore. (page 188)
    • And so we return to a tantalizing possibility: that self-organization is a prerequisite for evolvability, that it generates the kinds of structures that can benefit from natural selection. It generates structures that can evolve gradually, that are robust, for there is an inevitable relationship among spontaneous order, robustness, redundancy, gradualism, and correlated landscapes. Systems with redundancy have the property that many mutations cause no or only slight modifications in behavior. Redundancy yields gradualism. But another name for redundancy is robustness. Robust properties are ones that are insensitive to many detailed alterations. The robustness of the lipid vesicle, or of the cell type attractors in genomic networks in the ordered regime, is just another version of redundancy. Robustness is precisely what allows such systems to be molded by gradual accumulation of variations. Thus another name for redundancy is structural stability—a folded protein, an assembled virus, a Boolean network in the ordered regime. The stable structures and behaviors are ones that can be molded. (page 188)
      • This is why evolution may be the best approach for machine learning hyperparameter tuning
    • If this view is roughly correct, then precisely that which is self-organized and robust is what we are likely to see preeminently utilized by selection. (page 188)
    • The more rare and improbable the forms that selection seeks, the less typical and robust they are and the stronger will be the pressure of mutations to revert to what is typical and robust. (page 189)
  • Chapter 9: Organisms and Artifacts
    •  Might the same general laws govern major aspects of biological and technological evolution? Both organisms and artifacts confront conflicting design constraints. As shown, it is those constraints that create rugged fitness landscapes. Evolution explores its landscapes without the benefit of intention. We explore the landscapes of technological opportunity with intention, under the selective pressure of market forces. But if the underlying design problems result in similar rugged landscapes of conflicting constraints, it would not be astonishing if the same laws governed both biological and technological evolution. (page 192)
    • I begin by describing a simple, idealized kind of adaptive walk—long-jump adaptation—on a correlated but rugged landscape. We have already looked at adaptive walks that proceed by generating and selecting single mutations that lead to fitter variants. Here, an adaptive walk proceeds step-by-step in the space of possibilities, marching steadfastly uphill to a local peak. Suppose instead that we consider simultaneously making a large number of mutations that alter many features at once, so that the organism takes a “long jump” across its fitness landscape. Suppose we are in the Alps and take a single normal step. Typically, the altitude where we land is closely correlated with the altitude from which we started. There are, of course, catastrophic exceptions; cliffs do occur here and there. But suppose we jump 50 kilometers away. The altitude at which we land is essentially uncorrelated with the altitude from which we began, because we have jumped beyond what is called the correlation length of the landscape(page 192)
    • A very simple law governs such long-jump adaptation. The result, exactly mimicking adaptive walks via fitter single-mutant variants on random landscapes is this: every time one finds a fitter long-jump variant, the expected number of tries to find a still better long-jump variant doubles! (page 193)
      • Intelligence is computation, and expensive
    • As the number of genes increases, long-jump adaptations becomes less and less fruitful; the more complex an organism, the more difficult it is to make and accumulate useful drastic changes through natural selection. (Page 194)
    • The germane issue is this: the “universal law” governing long-jump adaptation suggests that adaptation on a correlated landscape should show three time scales—an observation that may bear on the Cambrian explosion. Suppose that we are adapting on a correlated, but rugged NK landscape, and begin evolving at an average fitness value. Since the initial position is of average fitness, half of all nearby variants will be better. But because of the correlation structure or shape of the landscape, those nearby variants are only slightly better. In contrast, consider distant variants. Because the initial point is of average fitness, again half the distant variants are fitter. But because the distant variants are far beyond the correlation length of the landscape, some of them can be very much fitter than the initial point. (By the same token, some distant variants can be very much worse.) Now consider an adaptive process in which some mutant variants change only a few genes, and hence search the nearby vicinity, while other variants mutate many genes, and hence search far away. Suppose that the fittest of the variants will tend to sweep through the population the fastest. Thus early in such an adaptive process, we might expect the distant variants, which are very much fitter than the nearby variants, to dominate the process. If the adapting population can branch in more than one direction, this should give rise to a branching process in which distant variants of the initial genotype, differing in many ways from one another as well, emerge rapidly. Thus early on, dramatically variant forms should arise from the initial stem. Just as in the Cambrian explosion, the species exhibiting the different major body plans, or phyla, are the first to appear. (Page 195)
    • Because the fraction of fitter nearby variants dwindles very much more slowly than in the long-jump case. In short, in the mid term of the process, the adaptive branching populations should begin to climb local hills. (Page 195)
    • The implication is this: when fitness is average, the fittest variants will be found far away. As fitness improves, the fittest variants will be found closer and closer to the current position. (Page 196)
      • So with hyperparameter tuning, change many variables initially, and reduce as the fitness results level out and proceed up the local hill
    • Uniting these two features of rugged but correlated landscapes, we should find radiation that initially both is bushy and occurs among dramatically different variants, and then quiets to scant branching among similar variants later on as fitness increases. (page 198)
    • Despite the fact that human crafting of artifacts is guided by intent and intelligence, both processes often confront problems of conflicting constraints. (Page 202)
      • Dimension reduction is a way of reducing those constraints, but the cost is ignoring the environment. Ideologies must be simple to allow for dense connection without conflict
    • As better designs are found, it becomes progressively harder to find further improvements, so variations become progressively more modest. Insofar as this is true, it is obviously reminiscent of the claims for the Cambrian explosion, where the higher taxa filled in from the top down. (Page 202)
      • This is a design trap. Since designing for more constraints limits hill climbing, designing for individuals and cultures could make everything grind to a halt. Designing for cultures needs to have a light footprint
    • There is something very familiar about this in the context of technological trajectories and learning effects: the rate of finding fitter variants (that is, making better products or producing them more cheaply) slows exponentially, and then ceases when a local optimum is found. This is already almost a restatement of two of the well-known aspects of learning effects. First, the total number of “tries” between finding fitter variants increases exponentially; thus we expect that increasingly long periods will pass with no improvements at all, and then rapid improvements as a fitter variant is suddenly found. Second, adaptive walks that are restricted to search the local neighborhood ultimately terminate on local optima. Further improvement ceases. (Page 204)
    • it seems worthwhile to consider seriously the possibility that the patterns of branching radiation in biological and technological evolution are governed by similar general laws. Not so surprising, this, for all these forms of adaptive evolution are exploring vast spaces of possibilities on more or less rugged “fitness” or “cost” landscapes. If the structures of such landscapes are broadly similar, the branching adaptive processes on them should also be similar. (Page 205)
  • Chapter 10: An Hour upon the Stage
    • The vast puzzle is that the emergent order in communities—in community assembly itself, in coevolution, and in the evolution of coevolution—almost certainly reflects selection acting at the level of the individual organism. (Page 208)
    • Models like those of Lotka and Volterra have provided ecologists with simple “laws” that may govern predator-prey relationships. Similar models study the population changes, or population dynamics, when species are linked into more complex communities with tens, hundreds, or thousands of species. Some of these links are “food webs,” which show which species eat which species. But communities are more complex than food webs, for two species may be mutualists, may be competitors, may be host and parasite, or may be coupled by a variety of other linkages. In general, the diverse populations in such model communities might exhibit simple steady-state patterns of behavior, complex oscillations, or chaotic behavior. (Page 211)
      • Building an ecology for intelligent machines means doing this. I guess we’ll find out what it’s like to build the garden of eden
    • Pimm and his colleagues have struggled to understand these phenomena and have arrived at ideas deeply similar to the models of fitness landscapes we discussed in Chapter 8 and 9. Different communities are imagined as points on a community landscape. Change the initial set of species, and the community will climb to a different peak, a different stable community. (Page 212)
    • In these models, Pimm and friends toss randomly chosen species into a “plot” and watch the population trajectories. If any species goes to zero population, hence extinct, it is “removed” from the plot. The results are both fascinating and still poorly understood. What one finds is that, at first, it is easy to add new species, but as more species are introduced, it becomes harder and harder. That is, more randomly chosen species must be tossed into the plot to find one that can survive with the rest of the assembling community. Eventually, the model community is saturated and stable; no further species can be added. (Page 212)
    • The community-assembly simulation studies are fascinating for a number of reasons beyond the distribution of extinction events. In particular, it is not obvious why model communities should “saturate,” so that it becomes increasingly difficult and finally impossible to add new species. If one constructs a “community landscape,” in which each point of the terrain represents a different combination of species, then the peaks will represent points of high fitness—combinations that are stable. While a species navigates a fitness landscape by mutating genes, a community navigates a community landscape by adding or deleting a species. Pimm argues that as communities climb higher and higher toward some fitness peak, the ascension becomes harder and harder. As the climb proceeds, there are fewer directions uphill, and hence it is harder to add new species. At a peak, no new species can be added. Saturation is attained. And from one initial point the community can climb to different local peaks, each representing a different stable community. (Page 214)
      • In belief spaces, this could help to explain the concept of velocity. It is mechanism for stumbling into new parts of the fitness landscape. And there is something about how ideas go stale.
    • In a coevolutionary arms race, when the Red Queen dominates, all species keep changing and changing their genotypes indefinitely in a never-ending race merely to sustain their fitness level. (Page 216)
      • This should also apply to belief spaces
    • Two main behaviors are imagined. The first image is of Red Queen behavior, where all organisms keep changing their genotypes in a persistent “arms race,” and hence the coevolving population never settles down to an unchanging mixture of genotypes. The second main image is of coevolving populations within or between species that reach a stable ratio of genotypes, an evolutionary stable strategy, and then stop altering genotypes. Red Queen behavior is, as we will soon see, a kind of chaotic behavior. ESS behavior, when all species stop changing, is a kind of ordered regime. (Page 221)
    • Just as we can use the NK model to show how genes interact with genes or how traits interact with traits within one organism, we can also use it to show how traits interact with traits between organisms in an ecosystem. (Page 225)
    • The ecosystem tends to settle into the ordered, evolutionary stable strategies regime if either epistatic connections, K, within each species are high, so that there are lots of peaks to become trapped on, or if couplings between species, C, is low, so landscapes do not deform much at the adaptive moves of the partners. Or an ESS regime might result when a third parameter, S, the number of species each species interacts with, is low, so that moves by one do not deform the landscapes of many others. (Page 226)
    • There is also a chaotic Red Queen regime where the species virtually never stop coevolving (Figure 10.4c). This Red Queen regime tends to occur when landscapes have few peaks to get trapped on, thus when K is low; when each landscape is deformed a great deal by adaptive moves of other species, thus when C is high; or when S is high so that each species is directly affected by very many other species. Basically, in this case, each species is chasing peaks that move away faster than the species can chase them. (Page 228)
    • At first, it might seem surprising that low K leads to chaotic ecosystems; in the NK Boolean networks, high K led to chaos. The more inter-couplings, the more likely a small change was to propagate throughout and cause the Boolean system to veer off into butterfly behavior. But with coupled landscapes it is the interconnectedness between the species that counts. When intercoupling, C, is high, moves by one species strongly deform the fitness landscapes of its partners. If any trait in the frog is affected by many traits in the fly, and vice versa, then a small change in traits of one species alters the landscape of the other a lot. The system will tend to be chaotic. Conversely, the ecosystem will tend to be in the ordered regime when the couplings between species, C, is sufficiently low. For much the same reason, if we were to keep K and C the same, but change the number of species S any one species directly interacts with, we would find that if the number is low the system will tend to be ordered, while if the number is high the ecosystem will tend to be chaotic. (Page 228)
      • There is something about Tajfel’s opposition identity that might lead to Red Queen scenarios. This would also help to explain the differences between left and right wing behaviours. Right wing is driven by “liberal tears” more than the opposition.
    • In fact, the results of our simulations suggest that the very highest fitness occurs precisely between ordered and chaotic behavior! (Page 228)
    • EpistasisTuning
    • Tuning an ecosystem. As the richness of epistatic connections between species, K, is increased, tuning the ecosystem from the chaotic to the orderly regime, average fitness at first increases and then decreases. It reaches the highest value midway between the extremes. The experiment is based on a 5 × 5 square lattice ecosystem, in which each of 25 species interacts with at most four other species. (Species on the corners of the lattice interact with two neighbors [CON = 2]; species on the edges interact with three neighbors [CON = 3]; and interior species interact with four neighbors [CON = 4]. N = 24, C = 1, S =25.) (page 229)
    • One might start the system with all species having very high K values, coevolving on very rugged landscapes, or all might have very low K values, coevolving on smooth landscapes. If K were not allowed to change, then deep within the high-K ordered regime, species would settle to ESS rapidly; that is, species would climb to poor local peaks and cling to them. In the second, low K, Red Queen chaotic regime, species would never attain fitness peaks. The story no longer stops there, however, for the species can now evolve the ruggedness of their landscapes, and the persistent attempts by species to invade new niches, when successful, will insert a new species into an old niche and may disrupt any ESS attained. (Page 232)
    • CoevolvingLandscapes
    • Figures 10.7 and 10.8 show these results. Each species has N = 44 traits; hence epistatic coupling can be as high as 43, creating random landscapes, or as low as 0, creating Fujiyama landscapes. As generations pass, the average value of K in the coevolving system converges onto an intermediate value of K, 15 to 25, and stays largely within this narrow range of intermediate landscape ruggedness (Above)). Here fitness is high, and the species do reach ESS equilibria where all genotypes stop changing for considerable periods of time, before some invader or invaders disrupt the balance by driving one or more of the coadapted species extinct. (Page 232)
    • When K is held high or low, deep in the ordered regime or deep in the chaotic regime, huge extinction avalanches rumble through the model ecosystems. The vast sizes of these events reflect the fact that fitness is low deep in the ordered regime because of the high-K conflicting constraints, and fitness is low deep in the chaotic regime because of the chaotic rising and plunging fitness values. In either case, low fitness of a species makes it very vulnerable to invasion and extinction. The very interesting result is that when the coevolving system can adjust its K range, it self-tunes to values where average fitness is as high as possible; therefore, the species are least vulnerable to invasion and extinction, so extinction avalanches appear to be as rare as possible. This shows up in Figure 10.8, which compares the size distribution and total number of extinction events deep in the ordered regime and after the system has self-tuned to optimize landscape ruggedness, K, and fitness. After the ecosystem self-tunes, the avalanches of extinction events remain a power law—the slope is about the same as when deep in the ordered regime. But over the same total number of generations, far fewer extinction events of each size occur. The self-tuned ecosystem also has far fewer extinction events than does an ecosystem deep in the chaotic regime. In short, the ecosystem self-tunes to minimize the rate of extinction! As if by an invisible hand, all the coevolving species appear to alter the rugged structures of the landscapes over which they evolve such that, on average, all have the highest fitness and survive as long as possible. (Page 234)
  • Chapter 11: In Search of Excellence
    • Organisms, artifacts, and organizations all evolve and coevolve on rugged, deforming, fitness landscapes. Organisms, artifacts, and organizations, when complex, all face conflicting constraints. So it can be no surprise if attempts to evolve toward good compromise solutions and designs must seek peaks on rugged landscapes. Nor, since the space of possibilities is typically vast, can it be a surprise that even human agents must search more or less blindly. Chess, after all, is a finite game, yet no grand master can sit at the board after two moves and concede defeat because the ultimate checkmate by the opponent 130 moves later can now be seen as inevitable. And chess is simple compared with most of real life. We may have our intentions, but we remain blind watchmakers. We are all, cells and CEOs, rather blindly climbing deforming fitness landscapes. If so, then the problems confronted by an organization—cellular, organismic, business, governmental, or otherwise—living in niches created by other organizations, is preeminently how to evolve on its deforming landscape, to track the moving peaks. (Page 247)
    • Evolution is a search procedure on rugged fixed or deforming landscapes. No search procedure can guarantee locating the global peak in an NP-hard problem in less time than that required to search the entire space of possibilities. And that, as we have repeatedly seen, can be hyperastronomical. Real cells, organisms, ecosystems, and, I suspect, real complex artifacts and real organizations never find the global optima of their fixed or deforming landscapes. The real task is to search out the excellent peaks and track them as the landscape deforms. Our “patches” logic appears to be one way complex systems and organizations can accomplish this. (Page 248)
    • The basic idea of the patch procedure is simple: take a hard, conflict-laden task in which many parts interact, and divide it into a quilt of nonoverlapping patches. Try to optimize within each patch. As this occurs, the couplings between parts in two patches across patch boundaries will mean that finding a “good” solution in one patch will change the problem to be solved by the parts in the adjacent patches. Since changes in each patch will alter the problems confronted by the neighboring patches, and the adaptive moves by those patches in turn will alter the problem faced by yet other patches, the system is just like our model coevolving ecosystems. Each patch is the analogue of what we called a species in Chapter 10. Each patch climbs toward fitness peaks on its own landscape, but in doing so deforms the fitness landscapes of its partners. As we saw, this process may spin out of control in Red Queen chaotic behavior and never converge on any good overall solution. Here, in this chaotic regime, our system is a crazy quilt of ceaseless changes. Alternatively, in the analogue of the evolutionary stable strategy (ESS) ordered regime, our system might freeze up, getting stuck on poor local peaks. Ecosystems, we saw, attained the highest average fitness if poised between Red Queen chaos and ESS order. We are about to see that if the entire conflict-laden task is broken into the properly chosen patches, the coevolving system lies at a phase transition between order and chaos and rapidly finds very good solutions. Patches, in short, may be a fundamental process we have evolved in our social systems, and perhaps elsewhere, to solve very hard problems. (Page 253)
    • It is the very fact that patches coevolve with one another that begins to hint at powerful advantages of patches compared with the Stalinist limit of a single large patch. What if, in the Stalinist limit, the entire lattice settles into a “bad” local minimum, one with high energy rather than an excellent low-energy minimum? The single-patch Stalinist system is stuck forever in the bad minimum. Now let’s think a bit. If we break the lattice up into four 5 × 5 patches just after the Stalinist system hits this bad minimum, what is the chance that this bad minimum is not only a local minimum for the lattice as a whole, but also a local minimum for each of the four 5 × 5 patches individually? You see, in order for the system broken into four patches to “stay” at the same bad minimum, it would have to be the case that the same minimum of the entire lattice happens also to be a minimum for all four of the 5 × 5 patches individually. If not, one or more of the patches will be able to flip a part, and hence begin to move. Once one patch begins to move, the entire lattice is no longer frozen in the bad local minimum. (Page 256)
    • Breaking large systems into patches allows the patches literally to coevolve with one another. Each climbs toward its fitness peaks, or energy minima, but its moves deform the fitness landscape or energy landscape of neighboring patches. (Page 257)
    • In the chaotic Leftist Italian limit, the average energy achieved by the lattice is only a slight bit less, about 0.47. In short, if the patches are too numerous and too small, the total system is in a disordered, chaotic regime. Parts keep flipping between their states, and the average energy of the lattice is high. (Page 258)
    • The answer depends on how rugged the landscape is. Our results suggest that if K is low so the landscape is highly correlated and quite smooth, the best results are found in the Stalinist limit. For simple problems with few conflicting constraints, there are few local minima in which to get trapped. But as the landscape becomes more rugged, reflecting the fact that the underlying number of conflicting constraints is becoming more severe, it appears best to break the total system into a number of patches such that the system is near the phase transition between order and chaos. (Page 258)
    • Here, then, is the first main and new result. It is by no means obvious that the lowest total energy of the lattice will be achieved if the lattice is broken into quilt patches, each of which tries to lower its own energy regardless of the effects on surrounding patches. Yet this is true. It can be a very good idea, if a problem is complex and full of conflicting constraints, to break it into patches, and let each patch try to optimize, such that all patches coevolve with one another. (Page 262)
    • But what, if anything, characterizes the optimum patch-size distribution? The edge of chaos. Small patches lead to chaos; large patches freeze into poor compromises. When an intermediate optimum patch size exists, it is typically very close to a transition between the ordered and the chaotic regime. (Page 262)
      • I’m pretty sure that this can be determined iteratively and within a desired epsilon. It should resemble the way a neural net converges on an accuracy.
    • I find it fascinating that hard problems with many linked variables and loads of conflicting constraints can be well solved by breaking the entire problem into nonoverlapping domains. Further, it is fascinating that as the conflicting constraints become worse, patches become ever more helpful. (Page 264)
    • I suspect that analogues of patches, systems having various kinds of local autonomy, may be a fundamental mechanism underlying adaptive evolution in ecosystems, economic systems, and cultural systems. (Page 254)
    • We are constructing global communication networks, and whipping off into space in fancy tin cans powered by Newton’s third law. The Challenger disaster, brownouts, the Hubble trouble, the hazards of failure in vast linked computer networks—our design marvels press against complexity boundaries we do not understand. (Page 265)
    • Patching systems so that they are poised on the edge of chaos may be extremely useful for two quite different reasons: not only do such systems rapidly attain good compromise solutions, but even more essentially, such poised systems should track the moving peaks on a changing landscape very well. The poised, edge-of-chaos systems are “nearly melted.” Suppose that the total landscape changes because external conditions alter. Then the detailed locations of local peaks will shift. A rigid system deep in the ordered regime will tend to cling stubbornly to its peaks. Poised systems should track shifting peaks more fluidly. (Page 266)
    • Misspecification arises all the time. Physicists and biologists, trying to figure out how complex biopolymers such as proteins fold their linear sequence of amino acids into compact three-dimensional structures, build models of the landscape guiding such folding and solve for the deep energy minima. Having done so, the scientists find that the real protein does not look like the predicted one. The physicists and biologists have “guessed” the wrong potential surface; they have guessed the wrong landscape and hence have solved the wrong hard problem. They are not fools, for we do not know the right problem. (Page 266)
      • Same for Hyperparameter tuning
    • We must learn how to learn in the face of persistent misspecification. Suppose we model the production facility, and learn from that model that a particular way to break it into patches is optimal, allowing the system to converge on a suggested solution. If we have misspecified the problem, the detailed solution is probably of little value. But it may often be the case that the optimal way to break the problem into patches is itself very insensitive to misspecifications of the problem. In the NK lattice and patch model we have studied, a slight change in the NK landscape energies will shift the locations of the minima substantially, but may not alter the fact that the lattice should still be broken into 6 × 6 patches. Therefore, rather than taking the suggested solution to the misspecified problem and imposing it on the real facility, it might be far smarter to take the suggested optimal patching of the misspecified problem, impose that on the real production facility, and then try to optimize performance within each of the now well-defined patches. In short, learning how to optimize the misspecified problem may not give us the solution to the real problem, but may teach us how learn about the real problem, how to break it into quilt patches that coevolve to find excellent solutions. (Page 267)
      • This is really worth looking at, because it can apply to round tripping simulation and real world systems as well. And a fitness test could be the time to divergence
    • receiver-based communication is roughly this: all the agents in a system that is trying to coordinate behavior let other agents know what is happening to them. The receivers of this information use it to decide what they are going to do. The receivers base their decisions on some overall specification of “team” goal. (Page 268)
    • ReceiverAttention
    • This observation suggests that it might be useful if, in our receiver-based communication system, we allowed sites to ignore some of their customers. Let’s say that each site pays attention to itself and a fraction, P, of its customers, and ignores 1 – P of them. What happens if we “tune” P? What happens is shown in Figure 11.8. The lowest energy for the entire lattice occurs when a small fraction of customers is ignored! As Figure 11.8 shows, if each site tries to help itself and all its customers, the system does less well than if each site pays attention, on average, to about 95 percent of its customers. In the actual numerical simulation, we do this by having each site consider each of its customers and pay attention to that customer with a 95 percent probability. In the limit where each site pays attention to no customers, of course, energy of the entire lattice is very high, and hence bad. (Page 268)
  • Chapter 12: An Emerging Global Civilization
    • Catalytic closure is not mysterious. But it is not a property of any single molecule; it is a property of a system of molecules. It is an emergent property. (Page 275)
    • But Fontana found a second type of reproduction. If he “disallowed” general copiers, so they did not arise and take over the soup, he found that he evolved precisely what I might have hoped for: collectively autocatalytic sets of Lisp expressions. That is, he found that his soup evolved to contain a “core metabolism” of Lisp expressions, each of which was formed as the product of the actions of one or more other Lisp expressions in the soup. (Page 278)
    • Fontana called copiers “level-0 organizations” and autocatalytic sets “level-1 organizations (Page 279)
    • The ever-transforming economy begins to sound like the ever-transforming biosphere, with trilobites dominating for a long, long run on Main Street Earth, replaced by other arthropods, then others again. If the patterns of the Cambrian explosion, filling in the higher taxa from the top down, bespeak the same patterns in early stages of a technological trajectory when many strong variants of an innovation are tried until a few dominant designs are chosen and the others go extinct, might it also be the case that the panorama of species evolution and coevolution, ever transforming, has its mirror in technological coevolution as well? Maybe principles deeper than DNA and gearboxes underlie biological and technological coevolution, principles about the kinds of complex things that can be assembled by a search process, and principles about the autocatalytic creation of niches that invite the innovations, which in turn create yet further niches. It would not be exceedingly strange were such general principles to exist. Organismic evolution and coevolution and technological evolution and coevolution are rather similar processes of niche creation and combinatorial optimization. While the nuts-and-bolts mechanisms underlying biological and technological evolution are obviously different, the tasks and resultant macroscopic features may be deeply similar. (Page 281)
    • The difficulty derives from the fact that economists have no obvious way to build a theory that incorporates what they call complementarities. The automobile and gasoline are consumption complementarities. You need both the car and the gas to go anywhere. (Page 282)
    • The use, I claim, is that we can discover the kinds of things that we would expect in the real world if our “as if” mock-up of the true laws lies in the same universality class. Physicists roll out this term, “universality class,” to refer to a class of models all of which exhibit the same robust behavior. So the behavior in question does not depend on the details of the model. Thus a variety of somewhat incorrect models of the real world may still succeed in telling us how the real world works, as long as the real world and the models lie in the same universality class. (Page 283)
    • An “enzyme” might be a symbol string in the same pot with a “template matching” (000) site somewhere in it. Here the “enzyme match rule” is that a 0 on the enzyme matches a 1 on the substrate, rather like nucleotide base-pairing. Then given such a rule for “enzymatic sites,” we can allow the symbol strings in the pot to act on one another. One way is to imagine two randomly chosen symbol strings colliding. If either string has an “enzymatic site” that matches a “substrate site” on the other, then the enzymatic site “acts on” the substrate site and carries out the substitution mandated in the corresponding row (Page 285)
    • Before we turn to economic models, let us consider some of the kinds of things that can happen in our pot of symbol strings as they act on one another, according to the laws of substitution we might happen to choose. A new world of possibilities lights up and may afford us clues about technological and other forms of evolution. Bear in mind that we can consider our strings as models of molecules, models of goods and services in an economy, perhaps even models of cultural memes such as fashions, roles, and ideas. Bear in mind that grammar models give us, for the first time, kinds of general “mathematical” or formal theories in which to study what sorts of patterns emerge when “entities” can be both the “object” acted on and transformed and the entities that do the acting, creating niches for one another in their unfolding. Grammar models, therefore, help make evident patterns we know about intuitively but cannot talk about very precisely. (Page 287)
    • These grammar models also suggest a possible new factor in economic takeoff: diversity probably begets diversity; hence diversity may help beget growth. (Page 292)

        Diversity begets growth opportunities. Pure growth is fastest in a monoculture of simple items with short maturity cycles

    • DiversityThe number of renewable goods with which an economy is endowed is plotted against the number of pairs of symbol strings in the grammar, which captures the hypothetical “laws of substitutability and complementarity.” A curve separates a subcritical regime below the curve and a supracritical regime above the curve. As the diversity of renewable resources or the complexity of the grammar rules increases, the system explodes with a diversity of products. (Page 193)
    • Friend, you cannot even predict the motions of three coupled pendula. You have hardly a prayer with three mutually gravitating objects. We let loose pesticides on our crops; the insects become ill and are eaten by birds that sicken and die, allowing the insects to proliferate in increased abundance. The crops are destroyed. So much for control. Bacon, you were brilliant, but the world is more complex than your philosophy. (Page 302)

When Worlds Collide


Charlottesville demonstrations, summer 2017 (link)

I’ve been thinking about this picture a lot recently. My research explores how extremist groups can develop using modern computer-mediated communication, particularly recommender systems. This picture lays the main parts like a set of nested puzzle pieces.

This is a picture of a physical event. In August 2017, various “Alt-Right” online communities came to Charlottesville Virginia to ostensibly protest the removal of confederate statues, which in turn was a response to the Charleston South Carolina church shooting of 2015. From August 11th through 12th, sanctioned and unsanctioned protests and counter protests happened in and around Emancipation Park.

Although this is not a runway in Paris, London or New York, this photo contains what I can best call “fashion statements”, in the most serious use of the term. They are mechanisms for signifying and conveying identity,  immediately visible. What are they trying to say to each other and to us, the public behind the camera?

Standing on the right hand of the image is a middle-aged white man, wearing a type of uniform: On his cap and shirt are images of the confederate “battle flag”. He is wearing a military-style camouflage vest and is carrying an AR-15 rifle and a 9mm handgun. These are archetypal components of the Alt-right identity.

He is yelling at a young black man moving in from the left side of the photo, who is also wearing a uniform of a sort. In addition to the black t-shirt and the dreadlocks, he is carrying multiple cameras – the sine qua non of credibility for young black men in modern America. Lastly, he is wearing literal chains and shackles, ensuring that no one will forget the slave heritage behind these protests.

Let’s consider these carried items, the cameras and the guns. The fashion accessories, if you will.

Cameras exist to record a selected instant of reality. It may be framed, with parts left out and others enhanced, but photographs and videos are a compelling document that something in the world happened. Further, these are internet-connected cameras, capable of sharing their content widely and quickly. These two elements, photographic evidence and distribution are a foundation of the #blacklivesmatter movement, which is a response to the wide distribution of videos where American police killed unarmed black men. These videos changed the greater social understanding of a reality encountered by a minority that was incomprehensible by the majority before these videos emerged.

Now to the other accessory, the guns. They are mechanisms “of violence to compel our opponent to fulfil our will”. Unlike cameras, which are used to provide a perspective of reality , these weapons are used to create a reality through their display and their threatened use. They also reflect a perception  of those that wield them that the world has become so threatening that battlefield weapons make sense at a public event.

Oddly, this is may also be a picture of an introduction of sorts. Alt-right and #blacklivesmatter groups almost certainly interact significantly. In fact, it is doubtful that, even though they speak in a common language , one group can comprehend the other. The trajectories of their defining stories are so different, so misaligned, that the concepts of one slide off the brain of the other.

Within each group, it is a different story. Each group shares a common narrative, that is expressed in words, appearance, and belief. And within each group, there is discussion and consensus. These are the most extreme examples of the people that we see in the photo. I don’t see anyone else in the image wearing chains or openly carrying guns. The presence of these individuals within their respective groups exerts a pull on the overall orientation and position of the group in the things that they will accept. Additionally, the individuals in one group can cluster in opposition to a different group, which is a pressure that drives each group further apart.

Lastly, we come to the third actor in the image, the viewer. The photo is taken by Shelby Lum, an award-winning staff photographer for the Richmond Times-Dispatch. Through framing, focus and timing, she captures this frame that tells this story.  Looking at this photo, we the audience feel that we understand the situation. But photographs are inherently simplifying. The audience fills in the gaps – what’s happened before, the backstory of the people in the image. This image can mean many things to many people. And as such, it’s what we do with that photo – what we say about it and what we connect with it that makes the image as much about us as it is about the characters within the frame.

It is those interactions that I focus on, the ways that we as populations interact with information that supports, expands, or undermines our beliefs. My theory is that humans move through belief space like animals move on the planes of the Serengeti. And just as the status of the ecosystem can be inferred through the behaviors of its animal population, the health and status of our belief spaces can be determined by our digital behaviors.

Using this approach, I believe that we may be able look at populations at scale to determine the “health” of the underlying information. Wildebeest behave differently in risky environments. Their patterns of congregation are different. They can stampede, particularly when the terrain is against them, such as a narrow water crossing. Humans can behave in similar ways for example when their core beliefs about their identity is challenged, such as when Galileo was tried by the church for essentially moving man from the literal center of the universe..

I think that this sort of approach can be used to identify at-risk (stampeding) groups and provide avenues for intervention that can “nudge” groups off of dangerous trajectories. It may also be possible to recognize the presence of deliberate actors attempting to drive groups into dangerous terrain, like native Americans driving buffalo off of pishkun cliffs, or more recently the Russian Internet Research Agency instigating and coordinating a #bluelivesmatter and a #blacklivesmatter demonstration to occur at the same time and place in Texas.

This theory is based on simulations that are based on the assumption that people coordinate in high-dimensional belief spaces based on orientation, velocity and social influence. Rather than coming to a static consensus, these interactions are dynamic and follow intuitions of belief movement across information terrain. That dynamic process is what I’ll be discussing over the next several posts.

The Radio in Fascist Italy

The Radio in Fascist Italy

  • Philip Cannistraro
  • Journal of European Studies
  • scholars have generally agreed that the control of the mass media by the state is a fundamental prerequisite for the establishment and maintenance of totalitarian dictatorships (pg 127)
  • It is not so widely acknowledged, however, that contemporary totalitarian governments have been largely responsible for the initial growth of the mass media-particularly films and the radio-in their respective countries. (pg 127)
  • In their efforts to expose entire populations to official propaganda, totalitarian regimes encouraged and sponsored the development of the mass media and made them available to every· citizen on a large scale basis. (pg 127)
  • Marconi shrewdly reminded Mussolini that it would be politically wise to place control of the radio in the hands of the state, pointing out the radio’s great potential for propaganda purposes (pg 128)
  • “How many hearts recently beat with emotion when hearing the very voice of the Duce! All this means but one thing: the radio must be extended and extended rapidly. It will contribute much to the general culture of the people” (pg 129)
  • … to insure that EIAR’s programmes conformed to the requirements of the regime’s cultural and political policies. The High Commission included government representatives from each major area of culture: literature, journalism., the fine arts, music, poetry, theatre, and films. The programmes Commission screened the transcripts and plans of all and censored the content of all broadcasts. (pg 130)
  • His broadcast, ‘The Bombardment of Adrianople’, was awaited by the public with great interest and was heralded by critics as the most significant cultural event of the Italian radio.ts Marinetti’s colourful language and emotion-packed presentation blasted un expected life into the Italian radio. His flam.boyant style introduced the concept of the ‘radio personality’ in Fascist Italy, and the success of his talk encouraged those who, like Marinetti himself, hoped to make the radio a new art form. Broadcasts by Marinetti, most of which were lectures on Futurism, continued to be heard on Italian radio each month for more than a decade. (pg 131)
  • The regime quickly recognized the effectiveness of this technique in· arousing listener interest, and it was an easy matter to transfer microphones to mass rallies from where the enthusiastic cheers of the spectators could be heard by radio audiences. (pg 132)
  • The popular announcer Cesare Ferri created the characters ‘Nonno Radio’ (Grandfather Radio) and ‘Zia Radio’ (Aunt Radio), speaking to Italian youth with unprecedented familiarity in terms they easily understood. (pg 132)
  • In order to popular arouse interest in its EIAR sought to stimulate indirect audience participation through public contests for short stories, poems, songs, In and children’s fairy tales. addition, surveys were conducted among listeners to discover trends in popular taste. (pg 133)
  • The radio had an important task to fulfil in the totalitarian state, that of binding the Italians together into one nation through common ideals and a common cultural experience inspired by Fascism. (pg 134)
  • Mussolini proclaimed Radio Rurale a great achievement of the Fascist revolution, for contemporary observers saw it as a new instrument with which to integrate rural existence into the mainstream. of national life. (pg 135)
  • The measures taken by the regime to overcome cultural and political provincialism by creating a mass radio audience in the countryside met with qualified success. (pg 137)
  • Regarded by many as an important step towards the creation of a truly popular culture, Radio Btdilla’s purpose was to give the working classes of the city and the countryside the means of acquiring a radio at a modest cost. Through the radio art, instruction, music, poetry-all the cultural masterworks–cease to become the privilege and unjust monopoly of a few elitist groups’. (pg 139)
  • ‘The ministry, in carrying out its delicate functions of vigilance over radio broadcasting, must guide itself by criteria that are essentially of a political and cultural nature.’ (pg 140)
  • Once the radio had been integrated into the structure of the Ministry of Popular Culture, the Fascists began to develop m.ore effective ways of using broadcasting as a cultural medium. While the number and variety of programmes had begun to increase by the beginning of the decade, it was only after 1934 that they became politically sophisticated. (pg 141)
  • Fascist racial doctrines became a major theme of radio propaganda during World War II. An Italo-German accord signed in 1940 to co-ordinate radio propaganda between the two countries included measures to ‘intensify anti-Jewish propaganda’ on the Italian radio as well as in foreign broadcasts.78 The Inspectorate for Radio Broadcasting organized an important series of anti-Semitic that centred around the ‘Protocols of Zion’, and talks such as ‘Judaism. versus Western Culture’, the ‘Jewish International’, and ‘Judaism. Wanted this War’, were broadcast from 1941 to 1943. (pg 143)
  • information received from the Vatican radio during World War II was generally regarded more accurate than the obvious propaganda broadcasts of the Allies (pg 147)
  • On the radio he astutely employed direct, forceful language, shouting short and vivid sentences to create a sense of drama and arouse emotional reactions. ‘This ‘maniera forte’ that characterized Appelius’ radio talks had a great appeal for many Italians, especially for the ‘little man’ who wanted to be talked to on his own level in terms he could readily understand.121 In his broadcasts Appelius screamed insults and ranted and raved at the foul enemies of Fascism. with a powerful barrage of verbal abuse, inciting his audiences to unmitigated hatred and scorn against the evil ‘anglo-sassoni’ and their allies. (pg 150)
  • In the broad context of Fascist cultural aspirations, all the media aimed at similar goals: the diffusion of standard images and themes that reflected the ideological values of Fascism.; the creation of a mass culture that conformed the needs of the Fascist state in its capacity as a totalitarian to government. (pg 154)

Karl Marx and the Tradition of Western Political Thought – The Broken Thread of Tradition

Hanna Arendt – Thinking Without a Banister

These two connected statements had already been torn asunder by a tradition that translated the one by declaring that man is a social being, a banality for which one would not have needed Aristotle, and the other by defining man as the animal rationale, the reasoning animal. (pg 23)

What Aristotle had seen as one and the same human quality, to live together with others in the modus of speaking, now became two distinct characteristics, to have reason and to be social. And these two characteristics, almost from the beginning, were not thought merely to be distinct, but antagonistic to each other: the conflict between man’s rationality and his sociability can be seen throughout our tradition of political thought (pg 23)

The law was now no longer the boundary (which the citizens ought to defend like the walls of the city, because it had the same function for the citizens’ political life as the city’s wall had for their physical existence and distinctness, as Heraclitus had said), but became a yardstick by which rule could be measured. Rule now either conformed to or overruled the law, and in the latter case the rule was called tyrannical usually, although not necessarily, exerted by one man-and therefore a kind of perverted monarchy. From then on, law and power became the two conceptual pillars of all definitions of government, and these definitions hardly changed during the more than two thousand years that separate Aristotle from Montesquieu. (pg 28)

But bureaucracy should not be mistaken for totalitarian domination. If the October Revolution had been permitted to follow the lines prescribed by Marx and Lenin, which was not the case, it would probably have resulted in bureaucratic rule. The rule of nobody, not anarchy, or disappearance of rule, or oppression, is the ever present danger of any society based on universal equality. (pg 33)

In Marx’s own opinion, what made socialism scientific and distinguished it from that of his predecessors, the “utopian socialists,” was not an economic theory with its scientific insights as well as its errors, but the discovery of a law of movement that ruled matter and, at the same time, showed itself in the reasoning capacity of man as “consciousness,” either of the self or of a class. (pg 35)

The logic of dialectal movement enables Marx to combine nature with history, or matter with man; man becomes the author of a meaningful, comprehensible history because his metabolism with nature, unlike an animal’s, is not merely consumptive but requires an activity, namely, labor. For Marx labor is the uniting link between matter and man, between nature and history. He is a “materialist” insofar as the specifically human form of consuming matter is to him the beginning of everything (pg 35)

Politics, in other words, is derivative in a twofold sense: it has its origin in the pre-political data of biological life, and it has its end in the post-political, highest possibility of human destiny (pg 40)

the fact that the multitude, whom the Greeks called hoi polloi, threatens the existence of every single person, runs like a red thread throughout the centuries that separate Plato from the modern age. In this context it is irrelevant whether this attitude expresses itself in secular terms, as in Plato and Aristotle, or if it does so in the terms of Christianity. (pg 40)

true Christians wohnen fern voneinander, that is, dwell far from each other and are as forlorn among the multitude as were the ancient philosophers. (pg 41)

Each new birth endangers the continuity of the polis because with each new birth a new world potentially comes into being. The laws hedge in these new beginnings and guarantee the preexistence of a common world, the permanence of a a continuity that transcends the individual life span of e each generation, and in which each single man in his mortality can hope to leave a trace of permanence behind him. (pg 46)

introduced the terms nomo and physei, by law or by nature. Thus, the order of the universe, the kosmos of natural things, was differentiated from the world of human affairs, whose order is laid down by men since it is an order of things made and done by men This distinction too, survives in the beginning of our tradition, where Aristotle expressly States that political science deals with things that are nomo and not physei. (pg 47)