Real-Time Object Detection for Autonomous Driving with YOLOv8 & SS3D

Introduction

Self-driving cars rely on vision systems that can spot and place road users and obstacles in real time. Modern detectors are fast enough to keep up with busy, fast-changing traffic scenes, which is why they sit at the heart of this kind of work.

This project builds a real-time detection system that not only finds objects in a camera image but also places them in a three-dimensional view of the scene. The aim is to give a vehicle a clearer sense of what is around it and where, while staying fast enough for live use.

Problem Statement

Driving safely means detecting many kinds of objects, other cars, pedestrians, cyclists, and roadside features, in all sorts of conditions:

1. Small or far-away objects are easy to miss at speed.
2. Rain: fog: glare, and low light make scenes harder to read.
3. Decisions have to happen almost instantly.
4. It is not enough to know what is there; the system needs to know where it sits in space.

Objectives

1. Combine fast 2D detection with a depth-aware 3D view in one pipeline.
2. Train and test the system on driving-relevant scenes.
3. Improve detection of small: distant, or partly hidden objects.
4. Present results in an easy-to-read 3D scene view that supports navigation.
5. Lay a foundation for richer sensing (camera: LiDAR, radar) later on.

Dataset

The work is built around a dataset that reflects real driving scenes. Depending on the client, this can be a public benchmark, licensed or private footage, data from other trusted sources, or a custom set captured for the project. Images are organised and prepared, then lightly varied (for scale, viewpoint, and lighting) so the system copes with real-world conditions.

Methodology

The pipeline pairs two models: YOLOv8 finds objects with 2D boxes in each camera frame, then SS3D (Single Shot 3D) lifts those detections into a 3D view of the scene so their position in space becomes clear.

The novelty is in the pairing. Instead of choosing between speed and spatial awareness, the design keeps real-time detection up front and adds depth understanding on top, including a lightweight way to carry information across consecutive frames so predictions stay steady as objects move quickly through view. The result is a setup that aims for safe, spatially aware perception without the heavy cost of a full 3D-only system.