Collaboration between Norfolk Southern Corporation and the Georgia Tech Research Institute (GTRI) has led to the development of digital train inspection portals that use advanced machine vision and artificial intelligence to examine trains moving at speeds of up to 60 miles per hour to identify mechanical defects that may exist.

Machine vision technology in the portals produces images of key components located on the front and back, top, bottom, and sides of train cars, providing a 360-degree view of the complete train. Images produced by the portal are analyzed within minutes of a train's passage, allowing any issues identified to be reported immediately.

Two train portals are currently in operation on adjacent tracks in Leetonia, Ohio, and the company plans to have as many as a dozen in service by the end of 2024. Among them will be a train portal already under construction near Jackson, Georgia, which is located south of Atlanta.

"Norfolk Southern is deploying Digital Train Inspection Portals to enhance rail safety across the company's 22-state network," said Mabby Amouie, chief data scientist for the company. "The portals feature cutting-edge machine vision inspection technology developed in partnership with GTRI, which engineered the hardware, and Norfolk Southern's Data Science/Artificial Intelligence and Mechanical teams, which built the brains behind the program."

The machine vision portion uses 38 high-resolution cameras consisting of a mix of area and line scan cameras to photograph critical components of each rail car moving through the portals. Powerful lights comparable to those used in sports stadiums allow the cameras to take approximately a thousand photographs of each moving rail car.

"Being able to look at the train while it's moving at 60 miles per hour provides visibility into defects that would be difficult to see otherwise," said Gary McMurray, division chief of GTRI's Intelligent Sustainable Technologies Division. "You want to be able to look at a train while it's in motion because that's when components are stressed, and you can see other dynamic faults."

To reduce the amount of data that must be analyzed, each camera is aimed at a specific area of the train and takes photographs only when components of interest are visible. "The high-speed cameras are strategically placed at angles to capture things that are difficult to detect with the human eye during stationary inspections," said Amouie.

Sensors at each portal determine the speed of each train passing through and use that information to precisely control when the photographs are taken.

"Even with a train traveling 60 miles per hour, we are able to calculate in real time when to tell each camera to take a picture," said Colin Usher, a GTRI senior research scientist who led development of the machine vision system. "Only images of critical components are taken and the other areas of the train that are inconsequential to identifying defects are not captured. That optimizes the image capture and saves space in the computer system."

The images produced by the system are analyzed by artificial intelligence algorithms developed by Norfolk Southern. The algorithms were designed to provide a combination of high accuracy and very low rates of false positives. If defects are spotted, the AI systems reports them immediately.