Developing an AI model to identify rust

Dushyant Puri (PM), SMART Centre, Mark Buchner (PI), Mahmud Ashrafizaadeh, PhD (PI), School of Applied Computer Science & IT

Phase two of this project built on previous work that developed a model to detect rust using the SELab iOS application. In phase one, the SELab app utilized various sensors on an iOS device to collect data and apply a machine learning model for real-time rust detection on metal surfaces, displaying labels on rust through augmented reality (AR). A key challenge was obtaining sufficient images and annotations for model training across different domains. 

In phase two, the project introduced the AnnoVision system to streamline image collection and labeling. This system comprised the SELab iOS app for capturing and uploading images and annotations, a web-based UI called Label Studio for editing annotations, and a backend that connected images and annotations to Label Studio. Users could take pictures, receive annotations from the pre-trained model, and modify them using Label Studio, enhancing the model's accuracy. ServiceEcho aimed to further develop the solution by integrating AR features to create a more immersive user experience, demonstrating the use of AR and AI to assist users in efficiently completing tasks. 

Funding for this project was provided by the Southern Ontario Network for Advanced Manufacturing Innovation (SONAMI).