Tuesday, May 01, 2012

Plane Tests Must Use Average Pilots, National Transportation Safety BoardSays After 737 MAX Crashes: Safety board says FAA should embrace data-driven approach to assumptions about pilot responses


The Wall Street Journal
By Andy Pasztor
September 26, 2019 10:00 am ET

Federal accident investigators called for broad changes in decades-old engineering principles and design assumptions related to pilot emergency responses, the first formal U.S. safety recommendations stemming from two fatal Boeing 737 MAX crashes.

As part of lessons learned from the crashes that took 346 lives and grounded the global MAX fleet, the National Transportation Safety Board suggested that Boeing Co. and the Federal Aviation Administration used unrealistic tests to initially certify the aircraft to carry passengers. The board also urged the plane maker and the FAA to pay more attention to interactions between humans and cockpit computers to ensure safety. The board wants Boeing and the FAA to reassess—and potentially jettison—what senior investigators portrayed as overly optimistic assumptions about the speed and effectiveness of cockpit-crew reactions to complex automation failures.

Five of the NTSB’s seven recommendations, released Thursday, called for the use of more-objective methods to predict likely responses of airline pilots in such cases when automation goes haywire. The board’s announcement challenged long-held industry and FAA practices that largely use the nearly instantaneous responses of highly trained test pilots—rather than those of average pilots, who typically have less experience—to verify the safety of new jetliner models. Some of the recommendations cover future airliner designs, not just the MAX.

The Wall Street Journal previously reported that mistaken assumptions by Boeing and the FAA about pilot response were at the core of last year’s Lion Air MAX crash as well as the Ethiopian Airlines MAX crash in March. The safety board is assisting local authorities in the Lion Air and Ethiopian Airlines probes.

Both planes went down because pilots—seemingly confused and distracted by sometimes contradictory warnings prompted by faulty sensor readings—failed to cope with a powerful automated flight-control feature, called MCAS, that pushed down the noses of the jets and ultimately put them into unrecoverable dives.

NTSB officials told reporters that before the MAX began commercial service Boeing failed to test—and the FAA never asked to see demonstrated—the full range of alerts, warnings and related system failures that could result from an MCAS misfire. Pilots of the ill-fated jets were overwhelmed by multiple alerts caused by a single malfunctioning sensor, leading to what safety experts call task saturation.

The pilots “did not react in the ways Boeing and the FAA assumed,” said NTSB Chairman Robert Sumwalt, a retired pilot who flew older 737 models. “We have found a gap between the assumptions” used to certify the MAX, he said, “and the real-world experiences of these crews.”

Amid a flurry of outside investigations, including a Justice Department criminal probe of the certification process, FAA officials already are moving to bring some procedures and regulations in line with the NTSB recommendations. In consultation with its foreign counterparts, for example, the agency intends to use a range of airline pilots—with varying degrees of flying expertise, experience and training—to verify the safety of pending software fixes and changes to flight-control computers on the MAX.

Dana Schulze, the NTSB’s top aviation investigator, said it is important to discount the bias that test pilots have “because they know the airplane like the back of their hand,” and instead seek new ways to determine “what the average pilot would do.” She also said additional recommendations are possible.

Ms. Schulze also told reporters the all-important certification tests “did not look at all potential flight-deck alerts and indications that pilots might see” in a real-world MCAS emergency.

The nonbinding recommendations also call on the FAA to help its foreign counterparts avoid such missteps when certifying aircraft in their countries. Eventually, the safety board wants Boeing and other U.S. plane manufacturers to devise onboard diagnostic systems capable on their own of sorting through a jumble of emergency alerts, to help pilots prioritize and speed through the correct checklist and alleviate the danger.

The FAA said it would carefully review these and all other recommendations as part of ongoing work to safely return the MAX to service. In a statement, it said that it is committed to a philosophy of improvement and that lessons learned from the two crashes will be a springboard to an even greater level of safety.

Boeing said it is “committed to working with the FAA in reviewing the NTSB recommendations.” Previously, the Chicago plane maker said the MAX was certified according to accepted industry standards and FAA requirements.

NTSB recommendations usually refrain from mandating specific technical solutions, so those involving the 737 MAX predictably leave the door open to a combination of design and training changes.

But in a teleconference laying out the recommendations, senior NTSB investigators questioned historical assumptions that pilots can be counted on to identify certain in-flight emergencies and respond to them within seconds.

Instead of the subjective validation of those assumptions by using test pilots, Ms. Schulze said the board is advocating for a “more data-driven, more scientific” evaluation of pilot performance. “We’re asking the FAA to improve the human-factors aspects” of aircraft certification, she said.

In designing the flight controls for the 737 MAX, Boeing assumed that, regardless of experience or background, pilots trained on existing safety procedures should be able to sift through a jumble of contradictory warnings and take the proper action 100% of the time within four seconds. The FAA based part of its certification on that assumption, though it is currently considering lengthening that time requirement.

But now the safety board is explicitly calling on Boeing and the FAA to identify and correct for potential “pilot actions that are inconsistent with manufacturer assumptions.” In addition, the recommendations urge the creation of robust tools and methods, with the help of human-factors experts, to validate “assumptions about pilot recognition and response” to significant safety risks or system malfunctions.

The board, according to people familiar with the details, initially drafted language calling for a review of the way the FAA gives industry officials authority to conduct certain safety reviews on behalf of the government. But the final document doesn’t include such a recommendation.

No comments:

Post a Comment