Synthetic-to-real Composite Semantic Segmentation in Additive Manufacturing

The application of computer vision and machine learning methods for semantic segmentation of the structural elements of 3D-printed products in the field of additive manufacturing (AM) can improve real-time failure analysis systems and potentially reduce the number of defects by providing additional tools for in situ corrections. This work demonstrates the possibilities of using physics-based rendering for labeled image dataset generation, as well as image-to-image style transfer capabilities to improve the accuracy of real image segmentation for AM systems. Multi-class semantic segmentation experiments were carried out based on the U-Net model and the cycle generative adversarial network. The test results demonstrated the capacity of this method to detect such structural elements of 3D-printed parts as a top (last printed) layer, infill, shell, and support. A basis for further segmentation system enhancement by utilizing image-to-image style transfer and domain adaptation technologies was also considered. The results indicate that using style transfer as a precursor to domain adaptation can improve real 3D printing image segmentation in situations where a model trained on synthetic data is the only tool available. The mean intersection over union (mIoU) scores for synthetic test datasets included 94.90% for the entire 3D-printed part, 73.33% for the top layer, 78.93% for the infill, 55.31% for the shell, and 69.45% for supports.

• Free and open source code: https://osf.io/h8r45
• Blender

Keywords

3-D printing, additive manufacturing, g-code segmentation, sim-to-real, semantic segmentation, synthetic data, machine learning, open source software, open-source hardware, RepRap, computer vision, quality assurance, real-time monitoring, anomaly detection; Blender, synthetic images

See also

• OS Computer Vision for Distributed Recycling and Additive Manufacturing
• 
  Synthetic-to-real Composite Semantic Segmentation in Additive Manufacturing
• 
  Towards Smart Monitored AM: Open Source in-Situ Layer-wise 3D Printing Image Anomaly Detection Using Histograms of Oriented Gradients and a Physics-Based Rendering Engine
• 
  Open Source Computer Vision-based Layer-wise 3D Printing Analysis
• 
  Three Hundred and Sixty Degree Real-Time Monitoring of 3-D Printing Using Computer Analysis of Two Camera Views
• 
  Factors effecting real-time optical monitoring of fused filament 3D printing
• 
  Open Source 3-D Filament Diameter Sensor for Recycling, Winding and Additive Manufacturing Machines

• Additional OS Computer Vision Applications
• 
  A novel camera-based sensor for real-time wastewater quality monitoring
• 
  From blender to farm: Transforming controlled environment agriculture with synthetic data and SwinUNet for precision crop monitoring
• 
  Optimizing Strawberry Disease and Quality Detection with Vision Transformers and Attention-Based Convolutional Neural Networks
• 
  Real-Time Eye State Detection System for Driver Drowsiness Using Convolutional Neural Network
• 
  Open source disease analysis system of cactus by artificial intelligence and image processing
• 
  Open Source Completely 3-D Printable Centrifuge
• 
  Monofacial vs bifacial solar photovoltaic systems in snowy environments
• 
  Image Analysis Method for Quantifying Snow Losses on PV Systems
• 
  Snow Losses for Photovoltaic Systems: Validating the Marion and Townsend Models

• Integrated Voltage—Current Monitoring and Control of Gas Metal Arc Weld Magnetic Ball-Jointed Open Source 3-D Printer
• Low-cost Open-Source Voltage and Current Monitor for Gas Metal Arc Weld 3-D Printing
• Slicer and process improvements for open-source GMAW-based metal 3-D printing
• Open-source Lab
• Open source 3-D printing of OSAT