In 2009, an Air France jet crashed into the ocean, leaving no survivors. The plane's autopilot system shut down and the pilots, having become reliant on their computerized assistant, were unable to correct the situation manually.

In 2015, a bus driver in Europe typed the wrong destination into his GPS device and cheerfully took a group of Belgian tourists on a 1,200 kilometer detour in the wrong direction.

In 2017, in a decision later overturned on appeal, US prosecutors who had agreed to release a teenager on probation abruptly changed their minds because an algorithm ruled the defendant "high risk".

These are dramatic examples, but they are far from isolated. When we outsource cognitive tasks to technology—such as flying a plane, navigating, or making a judgment—research shows we may lose the ability to perform those tasks ourselves. There is even a term for our tendency to forget information that is available through online search engines: the Google effect.

As new AI technologies promise to automate an increasing range of activities, the risk of "skill erosion" is growing. Our research shows how it can happen—and suggests ways to keep hold of the expertise you need, even when you don't need it every day.

Skill erosion can cripple an organization

My research shows the risk of skill erosion is easily overlooked. In a recent study, my team and I examined skill erosion in an accounting company.

The company had recently stopped using software that automated much of its fixed-asset accounting service. However, the accountants found themselves unable to carry out the task without it. Years of over-reliance on the software had eroded their expertise, and ultimately, they had to relearn their fixed-asset accounting skills.

While the software was rule-based (it did not use machine learning or "AI"), it was "smart" enough to track depreciation and produce reports for many tax and financial purposes. These are tasks that human accountants found very complex and tedious.

The company only became aware of skill erosion after a client found errors in the accounting team's manual reports. With its accountants lacking sufficient expertise, the company had to commission the software provider to fix the errors.

How skill erosion happens

We found that a lack of mindfulness about the automation-supported task had led to skill erosion. The old saying, "use it or lose it", applies to cognitively intense work as much as to anything else.

The accountants were not concerned about outsourcing their thinking to the software, since it operated almost flawlessly. In other words, they fell prey to "automation complacency": the assumption that "all is well" while ignoring potential risks.

This had three major consequences:

1. they lost their awareness of what automation was doing
2. they lost the incentive to maintain and update relevant knowledge (such as tax legislation), because the vendor and software did that for them
3. as the software was reliable, they no longer bothered to check the outgoing reports for accuracy.

This article is republished from The Conversation under a Creative Commons license. Read the original article.