Audio/Video AI Hand Washing Quality Analysis Tool
Revolutionizing Hand Washing Practices with AI-Powered Gesture Recognition
problem
We were approached by a client who is a medical service provider pioneering innovations in hand hygiene practices to enhance healthcare standards. Having developed a novel technological solution for hand hygiene, the client sought a way to efficiently bring their product to market. Their solution aimed to revolutionize handwashing by leveraging technology to monitor hand movements, providing immediate feedback on the quality of handwashing to ensure thorough hygiene. With time constraints looming, the client partnered with us to leverage data science expertise to expedite the project.
problem
We were approached by a client who is a medical service provider pioneering innovations in hand hygiene practices to enhance healthcare standards. Having developed a novel technological solution for hand hygiene, the client sought a way to efficiently bring their product to market. Their solution aimed to revolutionize handwashing by leveraging technology to monitor hand movements, providing immediate feedback on the quality of handwashing to ensure thorough hygiene. With time constraints looming, the client partnered with us to leverage data science expertise to expedite the project.
solution
Our goal was to develop a handwashing gesture recognition system and integrate it into the washing stations to offer users real-time feedback on their handwashing quality. The ultimate aim was to create a comprehensive platform capable of monitoring, coaching, and providing visual validation reports on hand hygiene. The project was divided into several stages to ensure optimal results: Data Labeling and Preparation, Modeling, and Deployment. During the data labeling stage - videos from the target platform were utilized for data labeling, with each handwashing step labeled according to its start and end times in the video. Around 100 videos were used for baseline development, with a 60-40 train/test split. Python served as the primary language for both modeling and infrastructure. PyTorch was employed for modeling neural networks, while OpenCV facilitated video processing. OpenVINO was utilized for communication with the Intel Compute Stick, with NumPy and Pandas handling data manipulation tasks, and Sklearn offering data science utilities. The system was deployed to AWS for cloud computing and Raspberry Pi 3b with an Intel Compute Stick 2 for local processing.
solution
Our goal was to develop a handwashing gesture recognition system and integrate it into the washing stations to offer users real-time feedback on their handwashing quality. The ultimate aim was to create a comprehensive platform capable of monitoring, coaching, and providing visual validation reports on hand hygiene. The project was divided into several stages to ensure optimal results: Data Labeling and Preparation, Modeling, and Deployment. During the data labeling stage - videos from the target platform were utilized for data labeling, with each handwashing step labeled according to its start and end times in the video. Around 100 videos were used for baseline development, with a 60-40 train/test split. Python served as the primary language for both modeling and infrastructure. PyTorch was employed for modeling neural networks, while OpenCV facilitated video processing. OpenVINO was utilized for communication with the Intel Compute Stick, with NumPy and Pandas handling data manipulation tasks, and Sklearn offering data science utilities. The system was deployed to AWS for cloud computing and Raspberry Pi 3b with an Intel Compute Stick 2 for local processing.
results
As a result of our collaborative effort, a handwashing event detector was built, categorizing different stages of the handwashing process. Leveraging a neural network, the system provided accurate event classification for each timestep, incorporating both spatial and temporal information. Predictions were further refined based on the distribution of handwashing event times for each event class. The developed algorithm was deployed in two environments: an AWS instance for cloud processing and a Raspberry Pi 3b with an Intel Compute Stick 2 module for real-time processing at the hand hygiene stations. RGB video from the Pi Camera v2 was utilized for monitoring hand hygiene practices. Our machine-learning model achieved an impressive 80% accuracy, enabling the client to move into production within a remarkably short timeframe of less than two months.
results
As a result of our collaborative effort, a handwashing event detector was built, categorizing different stages of the handwashing process. Leveraging a neural network, the system provided accurate event classification for each timestep, incorporating both spatial and temporal information. Predictions were further refined based on the distribution of handwashing event times for each event class. The developed algorithm was deployed in two environments: an AWS instance for cloud processing and a Raspberry Pi 3b with an Intel Compute Stick 2 module for real-time processing at the hand hygiene stations. RGB video from the Pi Camera v2 was utilized for monitoring hand hygiene practices. Our machine-learning model achieved an impressive 80% accuracy, enabling the client to move into production within a remarkably short timeframe of less than two months.
tech stack
tech stack
