Camera-radar fusion and tracking
Camera-radar fusion and tracking on real roadside recordings
At Fraunhofer IVI I led a student team of four and the technical work: a camera detector fused with an automotive radar on recordings from a roadside site in Ingolstadt. In 2026 I extended it in the public repo with a tracker and a full evaluation.
- Organisation
- Fraunhofer IVI
- Period
- Mar 2024 to Jun 2024
- Role
- Sensor Fusion Engineer, Camera-Radar Traffic Perception (student research project)
The 2024 project
Trained several YOLOv8 sizes and compared accuracy against compute and latency for real-time use; analysed and balanced the dataset before training.
On real roadside recordings from Ingolstadt, the association handles every correspondence case (one to one, one to many, many to one, many to many).
Detection and association were scored separately, so an error could be traced to its stage.
Writing the fusion by hand showed me how much information is lost between stages.
The 2026 extension
An extended Kalman filter per road user takes the radar's range, azimuth and range rate, so a new track has its speed from the first radar frame.
With YOLOv8n detections over all 22 scenes (2,484 frames), association precision is 97.8%, recall 92.2% and IDF1 0.89; IDF1 measures how consistently a track keeps one road user. All scores come from one roadside site, whose scenes the detector also saw in training.
INFRA-3DRC by Fraunhofer IVI, CC BY-NC 4.0.
Fact sheet
- Data
- INFRA-3DRC, real roadside recordings from Ingolstadt: 22 scenes, 2,484 frames
- Stack
- Python, YOLOv8, Radar clustering, Extended Kalman filter