Image Processing Helps Industrial Robots Process and Classify | Research & Technology

Automation technology developed at the Southwest Research Institute (SwRI) allows industrial robots to visually classify and perform tasks autonomously. The technology can be applied to grinding, painting, polishing, cleaning, welding, sealing and other industrial processes. According to SwRI’s R&D team, the solution increases process repeatability and decreases the need for rework, while reducing human exposure to hazardous environments.

The system uses machine learning algorithms developed at SwRI and classification software that works in conjunction with open source tools such as Scan-N-PlanMT and ROS 2, the latest version of the open-source robot operating system. Programming traditional robots can be slow and tedious, requiring an expert in the loop with knowledge of computer-aided design.


A robotic arm uses a 3D camera and machine vision software to dynamically scan and reconstruct contoured surfaces on aircraft parts. The robot uses 3D data to plan surface preparation paths with sanders, grinders and other power tools. SwRI leveraged Scan-N-Plan open sourceMT and ROS 2 software with custom algorithms, enabling intelligent image processing and classification with automated surface preparation. Courtesy of SwRI.


Scan-N-Plan, an ROS-Industrial technology, uses machine vision to digitize parts, creating 3D mesh data that robots use to plan tool paths and process paths while performing process monitoring in real time. The solution includes custom machine vision algorithms that allow robots to apply various media with varying pressure depending on the amount of surface work required. Feature-based processing is also enabled through additions that use semantic segmentation approaches to apply the right tool to the right feature – cutting versus sanding, for example.

The solution also addresses the need to perform different tasks at different stages of a process, or during different processes. The solution intelligently classifies regions and surface textures of parts at different stages of work, said Matt Robinson, robotics R&D manager at SwRI.

“These are breakthroughs that will help prevent robots from over-sanding or over-grinding metal surfaces,” added Paul Evans, director of SwRI’s manufacturing technology department.

SwRI showcased the technology in June at the Automate 2022 show in Detroit.

Mavis R. Bernier