Skip to content
DK.
Perception & ML

Perception validation

A keypoint detector validated from existing segmentation masks, with no new labels

One cone three times: the camera image, the outline traced from its FSOCO mask, and the predicted keypoints scored by their distance to that outline
One cone's keypoints, scored against its mask outlineSourceCone-Pose-Validation, FSOCO images

I led the validation of the cone-keypoint detector that a driverless race car uses to find the track. FSOCO, the community cone dataset, has no keypoint labels, so I used the outlines of its segmentation masks as ground truth, and ran the model on images pooled from more than 40 teams' cars and cameras to test how it generalises.

Organisation
Schanzer Racing Electric, Formula Student Driverless
Period
Oct 2024 to Apr 2025
Role
Perception System Engineer
teams' cars and cameras in the validation data
40+
of manual keypoint labelling avoided
~100 h
Code on GitHub

Ground truth

A cone's mask outline is exactly where its keypoints should sit, so each predicted keypoint is scored by its distance to that outline.

Two cones with eight keypoints each: in the right order the layout check passes; with two keypoints swapped the outline error is still zero but the layout check fails
Outline distance cannot see a swap, so the layout is checked tooSourceCone-Pose-Validation

Testing generalisation

11,572 images with boxes and 1,517 with masks: a run over both sets shows how the detector holds up across camera setups, image quality and mounting positions.

Eight camera frames from eight Formula Student teams, each with its own car, camera, resolution and mounting
One image from each of eight FSOCO teamsSourceCone-Pose-Validation, FSOCO images

Range gate and exclusions

A range gate from each cone's known height keeps the 1 to 15 m band; cones cut off at the image edge or knocked over are ignored; occluded, dark and lens-distorted cones stay in as the realistic hard cases.

A track lined with cones: scored cones in green boxes; a knocked-over cone, a truncated cone and one beyond 15 m in grey, each with the reason
Which labelled cones are scored, and which are ignoredSourceCone-Pose-Validation, FSOCO images

What it found

In a 2026 re-run on a sample with the team's YOLOv8s-pose detector, recall fell with estimated distance: 99% below 5 m, 90% at 5 to 10 m, 54% at 10 to 15 m; median keypoint error stayed within 2 to 3% of cone height. Colour mix-ups were rare (29 of 4,522 cones found); the errors were cones not found.

In the 2025 season I carried pixel error through to angular error to show what a miss costs at each range, and the findings went back into the data: occluded cones named as a blind spot, range-balanced splits proposed for the next training round.

Bar chart: recall about 0.99 at 1 to 5 m, 0.90 at 5 to 10 m and 0.54 at 10 to 15 m, with the right colour almost as high in each band
Recall by estimated distanceSourceCone-Pose-Validation, example run
cameraZδu / fe1near conee2far conee ≈ Z · δu / fδu pixel error at the keypoint, f focal length in pixels, Z range:the same pixel error costs more position the farther the cone.● cone○ where the keypoint error places itcameraZδu / fe1near conee2far conee ≈ Z · δu / fδu pixel error at the keypoint,f focal length in pixels, Z range:the same pixel error costs moreposition the farther the cone.
The same pixel error costs more at rangeKindRelation, not dataSourceDrawn for this siteNoteThe angle is exaggerated.

The validation framework

Built the automated validation framework around the detector: one run gives detection, keypoint and per-class scores. The detector runs on the car on an NVIDIA Jetson; the validation around it is mine.

FSOCO: M. Vödisch et al., SAE International Journal of Connected and Automated Vehicles, 2022. Figures show FSOCO images from other teams' cars, not Schanzer's.

Fact sheet

Stack
Python, YOLOv8-pose, OpenCV
Next: camera-radar fusion and tracking