Episode 6 – The Structure of Failure: MCAS Drifts
This is a pivotal moment in the story. A system (with a modest risk rating) already in place to address a very rare flight maneuver was then applied and modified to a new and more common situation without much debate about the implications of this change. A benign fix for one situation was applied to another as if they were identical.
There was a striking lack of imagination about how this new design could go wrong. How could this have happened?
Its design wasn’t really a rush job or an intentionally hidden flaw, but instead a failure of perception and process, a rote following of the rules and the “path of least resistance,” combined with a strong hesitation to ask too many questions—all driven by the program pressures.
Please share your views, insights, and opinions through the MAX8 Podcast Comments form. Episode 12 will be dedicated to feedback from listeners such as you.
-
EPISODE HIGHLIGHTS:
(0:51) – Summary of the story so far.
(2:25) – The interim period in MCAS’s design.
(5:28) – A new problem emerges.
(7:49) – New MCAS requirements and its design.
(15:55) – MCAS safety testing.
(22:49) – More weak signals.
(27:50) – The pilot training matter and the Mark Forkner story.
(37:23) – Conclusions using the Structure of Failure.
(41:08) – Three useful concepts driving failure.
(44:52) – Closing thoughts and outstanding questions.
KEY POINTS:
Early “weak signals.”
In 2012, a test pilot in a simulator took more than 10 seconds to diagnose and respond to uncommanded MCAS actions. On July 8, 2015, another test pilot took over 10 seconds to do the same. Then, on December 17, 2015, a Boeing engineer asked, “Are we vulnerable to single AOA sensor failure with the MCAS implementation, or is there some checking that occurs?”
These and other similar events represent what is called a “weak signal,” an indicator of a potentially emerging issue that may matter in the future. These should be viewed as “gifts,” not inconvenient problems, and carefully examined, even if they appear minor. Unfortunately, Boeing did not dig deep. Instead, they were reviewed per established procedures and dismissed, most likely due to MCAS’s modest risk rating.
A New Problem Emerges.
In March 2016, a new “handing” or “feel” issue with the control stick appeared during early live flight tests, similar to the first wind-up turn situation. Here, the problem appeared in a low-altitude, low-speed situation early after take-off while the airplane was accelerating and in a steep climb.
The aerodynamic cause here was different from the first situation. However, in both cases, unexpected extra lift forward on the airplane created the stick-feel problem.
While those two situations now appear different in retrospect, they were viewed at the time as similar problems—a straightforward compliance issue with the control stick’s feel.
Design changes in MCAS.
To address the new low-altitude situation, MCAS’s technical design evolved from an AOA sensor combined with a G-force sensor as triggering inputs to MCAS to a single AOA sensor acting alone. It also evolved from a maximum “authority” on the horizontal stabilizer per activation of 0.6 degrees occurring over 5 seconds to 2.4 degrees occurring for almost 10 seconds and from allowing only a single activation to allowing multiple ones.
Plus, other hidden problems appeared. First, by design, the pilot could not override automatic trimming with the control stick while MCAS was activating. This is different from the NG. The second problem was accidental - the AOA Disagree indicator light wasn’t working for most MAX airplanes.
So, we have a design for MCAS that served one situation altered to serve a substantially different situation. Along the way, the nose-down movement could be much more aggressive. Multiple activations were now possible. Only a single sensor now triggered MCAS. And all of this could happen much closer to the ground. What didn’t change but became more critical was the assumption that pilots would react during MCAS failure as if experiencing a runaway trim and do so within three seconds.
Why? Because MCAS was not viewed as a risky technology but as a proven fix.
Safety evaluation.
Only two potential failures were evaluated: the low-speed situation where MCAS activates but freezes (in other words, where it fails to perform in this new situation) and a high-speed wind-up turn in which MCAS doesn’t trigger (perhaps because of the removal of the g-force sensor input).
Even with the increased potential of a 2.5-degree stabilizer movement per activation, the analysis affirmed the original “major” risk rating in the low-speed situation and “hazardous” in the wind-up turn. Further, the study concluded that a low-speed failure's aerodynamic impact was less than a high-speed failure because MCAS was judged to be “less powerful” than in a wind-up turn.
And there was no test in which MCAS activated due to a faulty angle-of-attack sensor.
The pilot training story within the story.
In May 2013, the MAX’s chief project engineer, Michael Teal, emailed Boeing managers to express strong concern about MCAS upsetting the Level B training goal. This became a mantra.
Then came March 30, 2016, a highly visible milestone in the MCAS story. At 8:00 AM Pacific Time, Mr. Teal presented the decision to utilize MCAS for the low-altitude situation to Keith Leverkuhn, Boeing’s 737 MAX program General Manager. The recommendation was accepted. Three hours later, Mark Forkner, a Boeing manager responsible for airline training standards, emailed an FAA counterpart in the AEG unit to remove references to MCAS in a fight manual draft. This removal meant that an FAA body that determines specific required pilot training would not be aware of MCAS and could not include it in its evaluations. Was this connected to the executive decision that morning? Some feel it is a “smoking gun.” Others say evidence shows these events could not have been connected.
The story of Mark Forkner and his efforts to minimize or obscure MCAS is complicated and could be an entire podcast. This episode describes the first part of the story (and Episode 10 describes Mr. Forkner's subsequent trial).
Three concepts.
In reflecting on the design process (structure of failure), we see three macro forces were at work:
“Design drift.” This is not a failure to engineer a component but about taking a successfully created and built design for one situation and modifying it incrementally to serve a new situation without identifying unanticipated impacts and outcomes.
“Structural organizational ignorance.” This is a widespread state of painfully bad knowledge sharing, critical thinking, initiative, and collaboration across work groups that inhibit people from anticipating and managing difficult and unexpected issues.
“Positive asymmetry.” This is the notion that people have a strong, culturally driven preference for best-case scenarios. As a result, they pay less attention to worst-case scenarios. This is where risk gets underappreciated.
USEFUL EPISODE RESOURCES:
Robison, Peter. Flying Blind: The 737 MAX Tragedy and the Fall of Boeing. Doubleday, 2021.
THING YOU CAN DO:
Let me know your thoughts.
Please share your views, insights, and opinions. Episode 12 will be dedicated to feedback from listeners such as you.
You can contact me through the MAX8 Podcast Comments form. While I may not be able to respond to all comments, I will read each one carefully. I’m very interested in your thoughts.
Download my Framework of Failure description.
The Framework of Failure is summarized in a six-page PDF that can be downloaded. Access is at the bottom of the Home Page at BradIvie.com.
Subscribe for updates and announcements.
Please sign up to receive periodic email communications from me, primarily announcements of new podcast episodes and (in the future) blog posts. The signup form is at the bottom of all web pages on BradIvie.com.
Share this episode with friends and colleagues.
This podcast is created for many audiences: business professionals, management consultations, aeronautics industry professionals, aviation enthusiasts, policymakers, and the general public. Please share this episode.
with those who you feel would be interested in this story and benefit from the information provided and the analytic approach taken. Or perhaps the video trailer for the series.
