AI-Powered Handwashing Assessment for Safer Food Production
Contec Co., Ltd., a member of the Daifuku Group, has developed an AI-based handwashing assessment system, designed for use in food production facilities and restaurants. The system guides users through proper handwashing procedures and records compliance. We spoke with Ryota Inoue and Takamichi Yuasa from Contec’s Technology Division about the development of this innovative solution.
Starting With Operational Needs
Handwashing is vital to public health and forms an essential part of daily operations in healthcare, nursing care, and food service. In the food industry, the June 2020 revision of Japan’s Food Sanitation Act made hygiene management under HACCP (Hazard Analysis and Critical Control Point), as defined by the Codex Alimentarius Commission, a legal requirement in addition to existing practices. To support compliance, the Ministry of Health, Labour and Welfare issued a manual outlining correct handwashing procedures and recommended motions. Yet in practice, rinsing is sometimes incomplete, wrists may be overlooked, and employees sometimes rely on their own improvised methods. As a result, many companies establish formal handwashing rules and have supervisors check compliance. However, this approach also brought challenges.
“One company told us that employees filled out paper checklists each time they washed their hands, and supervisors then entered the data into the system manually,” recalls Yuasa. “Because the process relied on self-reporting, there was no way to confirm if handwashing was done properly. Paper-based management also led to omissions and inefficiency. Supervisors also mentioned that the data-entry workload was a heavy burden. We wanted to improve the quality control of handwashing and reduce that burden—and that idea became the starting point for development.”
Contec’s IVA-HW100 AI Handwashing Assessment System uses a camera-equipped edge computer installed at the wash station to monitor handwashing and determine, through image recognition AI, whether it is being performed correctly. The system’s monitor also displays real-time footage of users washing their hands alongside reference images of the proper motions. When a step is completed correctly, the system prompts users on. Following this on-screen guidance helps employees internalize the proper method over time. Once the full sequence is completed, the results are recorded automatically, giving supervisors easy access to compliance data without the need for manual entry.
AI Handwashing Assessment System monitor and sample evaluation results
Ensuring Accurate AI Recognition Across Diverse Hands
At the heart of the handwashing assessment system is image recognition AI. The team initially considered extracting and modeling finger feature points similar to what is done in motion capture, using a 3D camera for recognition. However, the presence of soap bubbles and running water during washing made this method unreliable.
“Around the same time, AI research was gaining momentum within the company,” recalls Inoue. “We thought, if AI has advanced this far, perhaps a standard 2D camera with image recognition could be effective. That became the hint that drove development forward.”
Camera-based handwashing recognition
Running image recognition AI requires a high-performance GPU, but cost restrictions ruled out expensive components. The system also needed to be compact enough for varied wash station environments, where installation space is often limited. Considering performance, price, and size, the team chose to go with NVIDIA. Utilization of an NVIDIA GPU also allowed for a smooth transition from server-based AI training to execution on the edge computer.
“The next step was preparing a dataset for training,” explains Inoue. “At first, we used our own hands, varying the amount of soap and water and filming from different angles. This allowed us to build a base AI model, but because hand shapes can differ so widely, we had to expand the dataset even further.”
“Diversity goes beyond shape—skin tone varies as well,” adds Yuasa. “By adjusting brightness, contrast, saturation, and hue, we multiplied the dataset several times over. And to ensure the system worked with both cameras placed in front and to the side, we trained the AI using some 100,000 images for each position. The result was a robust model able to recognize handwashing motions across a wide range of people, without needing to rely on expensive 3D cameras.”
To ensure the AI ran smoothly, the team also fine-tuned the system itself. Early on, the heavy processing load of the real-time handwashing display caused delays and interruptions in the video feed.
“In a standard PC, the desktop environment launches at startup,” explains Inoue. “We adopted a command-line interface (CUI) instead, which significantly reduced resource use. As a result, the handwashing system operates much more smoothly.”
Tuning for Real-World Use
Beyond the core AI-based assessment and automatic recordkeeping functions, the system also includes customizable features. While the Ministry of Health, Labour and Welfare’s manual serves as the default reference for handwashing motions and steps, actual practices vary across industries. Many companies have their own guidelines, and the IVA-HW100 lets users adjust the sequence of steps, required poses, and duration of each stage. Environmental factors were also taken into account. Because lighting and wash station conditions vary, the system includes camera settings to minimize false positives and missed detections.
Zoom in for a closer look.
“While we can automatically correct white balance, brightness and sink reflections can require more precise tuning,” notes Yuasa. “We therefore developed a camera settings app for users to adjust brightness and color as needed. The live video feed also highlights the recognition area with a red frame, and the frame size can also be adjusted.”
New functions are also planned for future updates, one of which is user authentication.
“Many customers have asked for an accurate way to track who washed their hands and when,” says Yuasa. “Because the system’s cameras are angled toward the hands, facial recognition is not an option. We considered IC card authentication, but in some work sites, hygiene rules prohibit bringing cards in. Instead, we are developing a contactless authentication system that uses palm vein recognition. We plan to continue adding features and refining the system while applying what we’ve learned to future product development.”
So far, the IVA-HW100 has been developed primarily for frozen food plants, boxed meal factories, and restaurant chains. Looking ahead, Contec plans to expand into industries where hygiene management is equally critical, such as pharmaceuticals and cosmetics. Contec is also extending its reach overseas, with inquiries already arriving from regions where hygiene management presents a pressing challenge.
By applying rapidly advancing AI technology to something as routine as handwashing, this project has created new societal value in the form of improved hygiene management. Contec and the broader Daifuku Group remain committed to pursuing new frontiers and delivering such value widely across society.
Contec Co., Ltd.
Ryota Inoue, Technology Division, IoT Technology Department, Communication Development Group
Takamichi Yuasa, Technology Division, Applied Development Group