Real-Time Object Detection & Classification — Conveyor-Belt Plastic Sorting
We built a system to detect and classify plastic objects moving on a conveyor belt in real time, so each item could be sorted by type and handed off to the right downstream process. The hard part wasn't detection in the abstract — it was doing it on fast-moving objects, where the model has a very small window to see an item, classify it correctly, and act before it leaves the frame. That constraint drove nearly every technical decision.
Model selection & fine-tuning
As lead on this project, I directed the core work: fine-tuning off-the-shelf object detection models on our own data and then optimizing them hard for inference speed. We evaluated a range of established architectures — Faster R-CNN, Mask R-CNN, and RetinaNet — to find the right accuracy-versus-latency trade-off for a real-time setting rather than assuming a single model would win.
Optimization for real-time latency
Hitting real-time latency took a deliberate optimization pipeline. We quantized the model and converted it to a TensorRT-optimized runtime, moving through a TensorFlow → ONNX → TensorRT path, and inserted custom NMS ops directly in the ONNX graph to shave further time off the post-processing that usually bottlenecks detection models. Together these brought end-to-end detection time under 100 ms — fast enough to reliably catch and classify objects as they move past on the belt.