Pedestrian Detection

My long-term research theme at Daimler R&D, spanning 1997-2013, has been active safety for vulnerable road users. With my team, we have pursued novel computer vision methods for detecting pedestrians and bicyclists from a moving vehicle. Research was originally conducted within EU projects PROTECTOR (2000-2003), SAVE-U (2002-2005) and WATCH-OVER (2005-2008), but this later shifted to a german context by means of the past AKTIV-SFR (2006-2010) project and the current URBAN-SVT (2012-2016) project. Several publications resulted.

        Home Gavrila
        Chamfer System
        Facial Features
        3D Head Model
        3D Body Tracking


PROTECTOR web site       SAVE-U web site                  



Our longstanding Daimler research on video-based pedestrian detection has paid off: active pedestrian safety, based on stereo vision, is now part of both the new 2013 Mercedes-Benz E-Class and S-Class models.  More ...



© Daimler


1. Motivation

Pedestrian Data Set Datasets

© Daimler


More than 150.000
 injured yearly in the EU.

More than
6000 are killed.

Pedestrians are arguably the most vulnerable traffic participants.

Children are
         especially at

Source: Bosch Accident Research

The last decade has seen increased awareness of the plight of vulnerable road users at the national and EU level. In 2003, the EU passed Phase 1 of Directive 2003/102/E on pedestrian protection, focussing on passive safety, i.e. meaning to reduce injury levels upon impact, by specifying various maximum impact criteria (e.g. head, leg). More recently, June 2008, the EU Parliament approved the Phase 2 draft legislation, which specifies a combination of passive and active safety measures. In particular, Phase 2 requires new passenger cars to be fitted with Brake Assist Systems (BAS) as early as 2009. Pedestrian protection is meanwhile also a major theme for consumer rating groups like Euro NCAP.

Passive pedestrian safety measures involve vehicle structures (e.g. bonnet, bumper) that expand during collision in order to minimize impact of the pedestrian leg or head hitting the vehicle.

mercedes pedestrian protection

© Daimler

For example, Mercedes-Benz introduced the active bonnet as standard for its 2009 new E-Class. The system includes three impact sensors in the front section as well as special bonnet hinges pretensioned by powerful springs. Upon impact with a pedestrian, the rear section of the bonnet is pushed upwards by 50 millimetres in a fraction of a second, thus enlarging the deformation zone. The system is reversible and can be reset manually by the driver.

2. Aim & Challenges

Although important, passive pedestrian safety measures are constrained by the laws of physics in terms of ability to reduce collision energy and thus injury level. Moreover, passive measures cannot account for injuries sustained in the secondary impact of the pedestrian hitting the road.

Pedestrian Data Set Datasets


The aim is to develop active (video-based) driver assistance systems which detect dangerous situations involving vulnerable road users (pedestrians, bicycliss) ahead of time, allowing the possibility to warn the driver or to automatically control the vehicle (e.g. braking). Such systems are particularly valuable when the driver is distracted or visibility is poor.

Yet vision-based pedestrian detection is a difficult problem for a number of reasons. The objects of interest appear in highly cluttered backgrounds and have a wide range of appearances, due to body size and pose, clothing and outdoor lighting conditions. Because of the moving vehicle, one does not have the possibility to use simple background subtraction methods (such as those used in surveillance applications) to obtain a foreground region containing the human. Furthermore, pedestrians can exhibit highly irregular motion, making prediction and situation analysis difficult. Finally, there are hard real-time requirements and stringent performance criteria.

3. Our Vision-based Pedestrian Detection Research

An overview of our long-term research on video-based pedestrian detection, incl. videoclips, is given in my keynote Smart Cars for Safe Pedestrians at the IEEE Intelligent Vehicles conference 2012, June 5, 2012. 

More technical details can be found in our publications, see in particular our IJCV’07, TPAMI’07, TITS’08, TPAMI’09, TITS’11 and TIP’11 papers.  

Video Gallery


Pedestrian detection in downtown Ulm (note the beautiful “Ulmer Münster” cathedral in backdrop).

Middle panel: obstacle detection by dense stereo (distances color-coded). Left panel: sensor coverage area and detected pedestrian. Right panel: top view of sensor coverage area and pedestrian track.


Automatic braking and evasion on (dummy) pedestrians


Pre-crash tests in the EU SAVE-U project with real pedestrians (2005)

Pedestrian manifold visualisation


Virtual pedestrian sample generation.




4. Benchmarking

Pedestrian detection has meanwhile attracted an extensive amount of interest from the computer vision community. Many techniques have been proposed in terms of features, models and general architectures (see recent survey on pedestrian detection). The picture is increasingly blurred on the experimental side. Reported performances differ by up to several orders of magnitude (e.g. within the same study). This stems from the different types of image data used (degree of background change), the limited size of the test datasets, and the different (often, not fully specified) evaluation criteria such as localization tolerance, coverage area, etc.

In order to increase visibility by providing a common point of reference from experimental perspective, we published a number of pedestrian benchmarks: data sets and associated performance metrics.


5. The Road Ahead



It has been gratifying to see our long-term pedestrian detection research be incorporated in the 2013 Mercedes-Benz E- and S-Class. This is especially the case since evaluations of the German GIDAS accident data carried out by Mercedes-Benz indicate that this new technology could avoid 6 percent of pedestrian accidents and reduce the severity of a further 41 percent. This translates to less injuries and more lives saved.

We will further improve our pedestrian system by both increasing its scenario coverage and recognition performance - continuing on Daimler’s
Road to Accident-free Driving.

[Home Gavrila] [Resume] [Research] [People] [Publications] [Datasets] [Media Coverage] [Open Positions] [Search]

Copyright © 2001-2013 Gavrila. All rights reserved. Disclaimer.