Real-time hazard detection that keeps workers safer on site

Client Industry

Industrial operations · multi-site worksites

Key Challenge

Recurring hazards despite CCTV & safety protocols

Core Tech Stack

Computer Vision · AWS SageMaker · MLOps · Terraform

Deployment Model

CCTV-integrated CV platform on AWS

SoftmaxAI developed Safety+, a production-ready computer-vision safety system which converts an industrial operator's current CCTV setup into a proactive risk-management solution. The platform takes in live feeds from the cameras, applies vision models to identify emerging hazards and monitor worker movement, assesses the level of risk in relation to predetermined hazard areas, and generates real-time alerts together with features for incident management and an analytics dashboard — everything being trained and deployed via an MLOps pipeline on AWS SageMaker using infrastructure defined by Terraform, thereby enabling safety teams to receive timely and actionable information without having to carry out extensive operational work.

• Incidents persisted despite protocols. Even with established safety procedures and extensive camera coverage, accidents and injuries still occurred since the footage was not continuously monitored or acted on in time.
• Hazard zones went unmonitored. Relying on human alertness to identify hazardous areas and to detect when workers had entered them meant that this approach could not be maintained reliably because there were many feeds involved.
• Worker movement was untracked. It was necessary to have an automated tracking system in order to follow the way in which people moved around a site and to anticipate potential risks, since the CCTV system then in place did not offer this.
• Adding systems could not add overhead. Any solution had to build on the cameras that were already there and bring risks to light without placing an excessive new operational burden on the teams.

Safety+ operates as a modular, left-to-right pipeline on AWS, taking the existing CCTV inputs, processing them through video ingestion and vision models operated by SageMaker for hazard detection and worker tracking, and then producing risk scores before issuing real-time alerts, carrying out incident management, and providing an analytics dashboard.

• Spec before code. Working forward-deployed alongside the operator, we prepared and approved a specification — covering the hazard zones, the tracking logic, and the incident and inspection workflows — before actually constructing anything.
• Diffusion-based model development. Senior engineers carried out an agent-accelerated build on the MLOps pipeline by using diffusion-based model development and fine-tuning, managed image annotation, and hyperparameter tuning in order to improve detection for a demanding environment.
• Infrastructure as code. The infrastructure needed for training and inference was defined as code using Terraform on AWS SageMaker, which made it possible to deploy the system repeatedly and on a large scale across different sites.
• Human-owned validation. The human reviewers were in charge of making the safety-critical decisions and checked the detections to ensure that the alerts remained reliable and trustworthy rather than becoming noisy.

• Turned existing CCTV into a safety sensor. The platform works with the cameras that are already in place, which allows the operator to obtain proactive monitoring without having to install new hardware or carry out a rip-and-replace operation.
• Continuous hazard detection. Vision models continuously monitor all the feeds in real time to detect any emerging hazards, replacing the occasional human inspection with constant awareness.
• Automated worker tracking. The movement of workers is automatically monitored throughout the site, providing the necessary context for anticipating risks in hazardous areas.
• Timely, dependable alerting. Events that have been assigned a risk score generate real-time alerts and are passed on to incident management as well as to customizable inspections, enabling teams to take action before the situations worsen.
• Intuitive analytics for proactive management. The analytics dashboard displays clear and actionable safety information, enabling the team to take steps to manage risks and keep the worksite safe.