Roadsign Detection Using an Autonomous Vehicle on an Embedded System
A lightweight, real-time road sign detection and autonomous navigation system specifically optimized for the NVIDIA Jetson Nano and STM32 microcontroller.
01
Problem & Motivation
Autonomous vehicles require sub-second sign detection to make safe navigation decisions. On low-power embedded hardware, standard state-of-the-art CNNs produce unacceptable latency spikes, while lightweight single-stage detectors historically suffer from high miss rates on small, distant signs.
An autonomous vehicle’s reliability critically depends on the system's ability to read and interpret road signs in real-time within dynamic, safety-critical environments. Resource-constrained embedded platforms cannot sustain the latency and memory footprints of heavy two-stage object detectors (like Faster R-CNN), risking dangerous 'blind' intervals during autonomous navigation. This research addresses these embedded constraints by designing and evaluating a streamlined Single Shot MultiBox Detector (SSD) utilizing MobileNet V2 with depth-wise separable convolutions and Feature Pyramid Networks (FPN). To maximize GPU throughput, real-time computer vision inference is dedicated to the NVIDIA Jetson Nano P3450, while vehicle control and non-vision tasks are offloaded to an STM32 F446RE microcontroller. The pipeline is validated against the US LISA Traffic Sign Dataset.
02
System Architecture
Hardware-Isolated Vision & Control Architecture
Camera Stream Capture
Continuous video stream ingested directly by NVIDIA Jetson Nano P3450 at optimal frame rates.
SSD + MobileNet V2 Backbone
Replaced standard convolutions with depth-wise separable convolutions to drastically minimize FLOPs and parameter memory footprint.
Feature Pyramid Network (FPN)
Constructed multi-scale feature hierarchies to resolve the single-stage detector failure point of missing small, distant road signs.
Hardware Task Isolation (STM32)
Offloaded actuator control, motor drivers, and low-level safety routines to an STM32 F446RE microcontroller, preserving 100% of Jetson GPU compute for vision inference.
03
Technical Decisions & Trade-Offs
MobileNet V2 Depthwise Separable Convolutions
Reduces computational cost and memory footprint by over 70% compared to standard convolution backbones without catastrophic accuracy degradation.
Feature Pyramid Network (FPN) Integration
Combines low-resolution semantically strong features with high-resolution semantically weak features, enabling reliable detection of small signs early in the vehicle's approach.
STM32 F446RE Heterogeneous Microcontroller Separation
Ensures determinism in vehicle control routines and prevents CPU/GPU starvation on the Jetson Nano during peak vision inference loads.
04
Data & Experimental Evaluation
Dataset Specifications
- Dataset:LISA Traffic Sign Dataset
- Size:Real-world US road and traffic sign video sequences
- Split:Evaluated across variable lighting, occlusion, and distance conditions
- Protocol:
Validated against challenging edge cases including faded signs, glare, and partial occlusions.
Evaluated Baselines & Ablations
- Standard Two-Stage Faster R-CNN (Baseline)
- Standard SSD without FPN (Baseline)
- Proposed SSD-MobileNetV2 + FPN (Jetson Nano Optimized)
06
Technologies & Tools
07
Key Takeaways & Findings
Achieved stable real-time frame rates suitable for closed-loop navigation on low-power hardware.
Hardware-level isolation between Jetson Nano and STM32 eliminated frame drops and navigation stutter.