Emergent Behaviors
Emergent Behaviors are system behaviors that are not due to a single part or component, but instead due to the interaction of multiple components[1]. Some illustrative examples of emergent behaviors include:
- Vibrations produced by moving parts in a system can cause unwanted effected such as loosening bolts and nuts, or frictional wear.
- Upon opening of the London Millennium Bridge, it was discovered that the very small movements of the bridge produced by the natural strides of pedestrians were of the correct frequency to influence the strides of other pedestrians. The result was that most pedestrians ended up walking in step with each other at a frequency of resonance for the bridge. This positive feedback loop resulted in significant deflections that necessitated alterations to the bridge for safety.
- Even the interactions of insects, can be emergent systems. Bees are able to form a complex system that can construct hives and form complex social systems. Meanwhile individual bees themselves have no large scale understanding of the system as a whole and simply react to external stimulus and instinct for their particular role.
Failures arising from emergent behaviors are particularly difficult to track down even using systems engineering processes such as FMEA or FTA because while those methods pick either a component or system failure and trace them to the respective system failure or causes, they assume knowledge of all potential failures. On the other hand, emergent behaviors are likely unexpected, and due to internal system interactions that are not apparent during design, and as a result may cause failure modes that were not previously considered.
Since there will be emergent behaviors in a system that may cause undesired results, it is important that safety engineering processes are maintained throughout the life cycle of the system and emergent behaviors are recorded and if necessary, the design or implementation altered to account for newly discovered behaviors. As systems become more complex, there will be a more and more opportunities for emergent behaviors that may cause failure. While redundancy in a design can be helpful as a preventative measure, in many cases it assumes that there will not be failure of all redundant features at the same time. Indeed this is unlikely, but as noted in the book Normal Accidents[2] by Charles Parrow,
"(...) we load our complex systems with safety devices in the form of buffers, redundancies, circuit breakers, alarms, bells, and whistles (...) however, two or more failures, none of them devastating in themselves in isolation, come together in unexpected ways and defeat the safety devices."
References and Resources
- DoD. 2008. Systems Engineering Guide for Systems of Systems. Washington, DC, USA: US Department of Defense (DoD).
- Perrow, C. (2000). Normal Accidents : Living with High Risk Technologies - Updated Edition