Trash Object Detection System
A fine-tuned RT-DETRv2 detector that recognizes trash, hands, and bins - with a gamified demo that awards a point when all three appear together.
About this project
Built by fine-tuning PekingU/rtdetr_v2_r50vd (RT-DETRv2) via HuggingFace's AutoModelForObjectDetection on the mrdbourke/trashify_manual_labelled_images dataset. The label set is 7 classes, not 3: alongside bin, hand, trash, and trash_arm, the dataset deliberately includes not_bin, not_hand, and not_trash as hard-negative classes - training the model to distinguish near-misses rather than only ever predicting a positive detection. A custom Trainer subclass applies discriminative learning rates (1e-5 for the pretrained backbone, 1e-4 for the detection head) over 10 epochs, with gradient clipping and a linear warmup schedule.
Technical details
Evaluated on a held-out test split with torchmetrics' MeanAveragePrecision: mAP 0.3806, mAP@50 0.5244, mAP@75 0.4268. Performance is not uniform across object scales - mAP for medium-sized objects sits noticeably lower at 0.1304 (mAP for small and large objects is stronger, at 0.30 and 0.40 respectively), a known limitation given the hand-labelled dataset's size rather than a bug. Non-maximum suppression is handled by the model's own post_process_object_detection call rather than a separate manual NMS pass.
The deployed Gradio app (Saint5/rt_detrv2_finetuned_trash_box_detector_v1 on the HuggingFace Hub) exposes the detection confidence threshold as a user-adjustable slider (default 0.3), draws colour-coded bounding boxes per class, and runs a simple set-membership check against {trash, hand, bin}: if all three are present in the detections, the app returns "+1 Point 🪙" and names the items found; otherwise it names precisely which of the three is still missing, prompting the user to retry rather than failing silently.
The Object_Detection_Notebook.ipynb also documents the full data pipeline from scratch: manual COCO-format annotation conversion, batch collation for variable-sized detection targets, and a companion notebook (Drawing_Bounding_Box.ipynb) covering conversions between XYXY, XYWH, and CXCYWH box formats used by PyTorch, general CV tooling, and YOLO respectively.