No crystal ball is needed to envision a future that engineers have in mind, one in which air taxis and other flying vehicles ferry passengers between urban locations, avoiding the growing gridlock on the ground below. Companies are already prototyping and testing such hybrid electric "flying cars" that take off and land vertically but soar through the air like winged aircraft to enable efficient flight over longer distances.

Naturally, one of the key areas of concern for these aerial vehicles is safety. The aircraft must not only stay airborne but also remain in control regardless of problems that could arise during flight—anything from gusts of wind to objects flying in their path to failing propellers.

Now, a Caltech team has developed an onboard Machine Learning-based control method to help such aircraft detect and compensate for disturbances so they can keep on flying. The engineers describe the new method, which they call "Neural-Fly for Fault Tolerance" (NFFT), in a paper accepted for publication in the journal IEEE Robotics and Automation Letters.

"In order to realize the full potential of these electric fliers, you need an intelligent control system that improves their robustness and especially their resilience against a variety of faults," says Soon-Jo Chung, Bren Professor of Control and Dynamical Systems at Caltech and Senior Research Scientist at JPL, which Caltech manages for NASA.

"We have developed such a fault-tolerant system crucial for safety-critical autonomous systems, and it introduces the idea of virtual sensors for the detection of any failure using machine learning and adaptive control methods."

Multiple rotors mean many possible points of failure

Engineers are building these hybrid-electric aircraft with multiple propellers, or rotors, in part for redundancy: If one rotor fails, enough functional motors remain to stay airborne. However, to reduce the energy required to make flights between urban locations—say, 10 or 20 miles—the craft also needs fixed wings.

Having both rotors and wings, though, creates many points of possible failure in each aircraft. And that leaves engineers with the question of how best to detect when something has gone wrong with any part of the vehicle.

Engineers could include sensors for each rotor, but even that would not be enough, says Chung. For example, an aircraft with nine rotors would need more than nine sensors since each rotor might need one sensor to detect a failure in the rotor structure, another to notice if its motor stops running, and still another to alert when a signal wiring problem occurs.

"You could eventually have a highly redundant distributed system of sensors," says Chung, but that would be expensive, difficult to manage, and would increase the weight of the aircraft. The sensors themselves could also fail.

With NFFT, Chung's group has proposed an alternative, novel approach. Building on previous efforts, the team has developed a deep-learning method that can not only respond to strong winds but also detect, on the fly, when the aircraft has suffered an onboard failure.

The system includes a neural network that is pre-trained on real-life flight data and then learns and adapts in real-time based on a limited number of changing parameters, including an estimation of how effective each rotor on the aircraft is functioning at any given time.

"This doesn't require any additional sensors or hardware for fault detection and identification," says Chung. "We just observe the behaviors of the aircraft—its attitude and position as a function of time. If the aircraft is deviating from its desired position from point A to point B, NFFT can detect that something is wrong and use the information it has to compensate for that error."

And the correction happens extremely quickly—in less than a second. "Flying the aircraft, you can really feel the difference NFFT makes in maintaining controllability of the aircraft when a motor fails," says Staff Scientist Matthew Anderson, an author on the paper and pilot who helped conduct the flight tests. "The real-time control redesign makes it feel as though nothing has changed, even though you've just had one of your motors stop working."

Introducing Virtual Sensors

The NFFT method relies on real-time control signals and algorithms to detect where a failure is, so Chung says it can give any type of vehicle essentially free virtual sensors to detect problems.

The team has primarily tested the control method on the aerial vehicles they are developing, including the Autonomous Flying Ambulance, a hybrid electric vehicle designed to transport injured or ill people to hospitals quickly. But Chung's group has tested a similar fault-tolerant control method on ground vehicles and has plans to apply NFFT to boats.