Automated shelf intelligence that keeps stores stocked and compliant

Client Industry

Retail · multi-store chain

Key Challenge

Manual, error-prone shelf stock & planogram checks

Core Tech Stack

YOLO · Vision Transformers · OpenCV · MLOps

Deployment Model

Containerized computer-vision inference on GKE

SoftmaxAI has developed a production-quality computer-vision system which converts ordinary images of shelf displays into continuous and objective inventory data. The system takes in-store camera images, cleans them using OpenCV, and then uses a YOLO object detector together with a Vision Transformer classifier to count the products, check that the planogram requirements are met, and identify any empty spaces — sending the real-time shelf data back to the store teams. The system includes an automated MLOps training pipeline with experiment tracking that constantly improves the deployed models, so that accuracy increases over time without the need for manual tuning and without requiring staff to be taken off the floor to walk down the aisles.

• Counting was manual and slow. The staff counted the stock by hand along the long aisles — a job which was both repetitive and time-consuming, as it kept them away from the customers and yet still allowed for mistakes to occur.
• Planogram compliance drifted unseen. Since spot checks were used to verify that the products were in the positions that had been planned for, misplaced or mis-faced products might not be corrected between audits.
• Empty spaces went unnoticed. Gaps with no items on the shelf were not discovered unless someone happened to look, which meant that restocking was delayed and sales were thus quietly lost.
• Shelf state was a snapshot, not a signal. The manual audits provided only a snapshot of a particular point in time; the team did not have a continuous and objective overview of stock levels and compliance that they could act on as the conditions changed.

The system operates as a modular, left-to-right computer vision pipeline, starting with imagery from the in-store cameras, then going through OpenCV preprocessing and a cluster that combines YOLO with a Vision Transformer for inference, before reaching the shelf-compliance logic and the real-time metrics, the deployed models being continuously refined by an automated MLOps pipeline.

• Spec before code. We prepared a specification that had been approved by the client, which defined the detection tasks, the planogram-compliance rules, and the way in which shelf metrics would be made available for action, before proceeding with the writing of the production code.
• Detection plus classification. The YOLO object detector identifies all the products displayed on the shelf while the Vision Transformer classifies each one, thereby determining the counts and positions from the same clean frame, preprocessed with OpenCV.
• Compliance as explicit logic. The detection data is fed into a shelf-analytics layer which checks the items' placement against the planogram, counts the number of products, and identifies any empty spaces — using auditable rules rather than an opaque score.
• A self-improving, human-owned loop. Using an automated training pipeline together with experiment tracking, models are retrained and redeployed on actual imagery from shelves, while people are responsible for maintaining the accuracy standards and for interpreting the results.

• Automated product counting. The system obtains the stock figure directly from the images of the shelves, putting an end to manual counts by providing a constant, objective measure of the actual items on the shelves.
• Continuous planogram verification. Product placement is compared with the planogram on each individual frame, which means that compliance is constantly monitored rather than being checked only during periodic audits.
• Empty-space detection. As gaps and out-of-stock items appear, they are flagged, which gives the store teams an early indication that they should restock before the shelves are emptied.
• Real-time metrics for store teams. The shelf status — including the counts, compliance, and gaps — is sent back as live metrics, which turns what would otherwise be a periodic manual task into a continuous feed.
• Models that keep improving. By using an automated MLOps pipeline, the detection and classification models are retrained and re-deployed on new imagery, which causes their accuracy to improve without any need for manual re-engineering.