Main characteristics:

1. •10 sequences of drivers, recorded outdoor (7) and indoor (3)
   

2. •Cameras are placed on the dashboard
   

3. •Drivers perform typical actions: talking, moving, operating the audio system, etc.
   

4. •Video sequences are recorded in grayscale, in high-resolution and 30 fps.
   

5. •Faces have been marked with 20 points, in the same locations as in the BioID database.
   

Overview

The RS-DMV dataset is a set of video sequences of drivers, recorded with cameras installed over the dashboard. The dataset currently contains 10 video sequences. The drivers were fully awake, talked frequently and were asked to look regularly to rear-view mirrors and operate the car sound system. Sequences contain occlusions, illumination changes and other elements that are problematic to face tracking and driver monitoring systems using computer vision.

Frames are recorded in gray-scale, at 30 frames per second, and stored as RAW video. Frame size of outdoor videos is 960x480 pixels, and 1390x480 for indoor videos. Faces in the videos have been marked with 20 points.

The RS-DMV was created by the RobeSafe Research Group of the Dpt. of Electronics of the UAH.  Videos recorded in a simulator were recorded under the CABINTEC project.

Distribution

This face database is publicly available and can be obtained from the authors for free for purely research purposes. Commercial distribution or any act related to commercial use of this database is strictly prohibited. Permission is NOT granted to reproduce, post or upload the database as a whole or in part to any publicly available server. You must not rent or sale the data, or obtain any kind of economical profit from the dataset.

The size of the full database is over 40GB, so it can not be distributed via web. (A compressed sample is available below). If you are interested in obtaining the database, please:

1. 1.Send us an e-mail to let us know about your interest in obtaining the data. (Preferably using your e-mail at your research institution).
   

2. 2.We will ask you for a laptop sized hard-drive, or some USB pen-drives where we can copy the data.
   

3. 3.We will mail you the media with the database.
   

Each frame has a header, indicating frame #, capture time, size and other information, written in the first row of the image. Source code is provided to read the video files.

Sequences in the dataset

Outdor sequences were recorded in the University campus, with 6 subjects driving a path of 1.1km (0.68 miles) in length. These sequences are around two minutes long (~3500 frames, 1.8 GB).

The vehicle used to record the sequences

Indoor sequences were recorded in a realistic truck simulator.  Illumination is very low, and in some sequences near-IR LEDs were used to increase available light. The diaphragm of camera was wide open and exposure time was high, so blur from defocus and motion is noticiable in some moments when the driver moves. This sequences are ~9000 frames in length (10GB).

Samples from indoor sequences

Sample

You can download a 30 second sample (AVI file, 5MB) to get a feel of the video sequences.

Also, take a look at the RSMAT sample video on the main page. It contains parts from most of the sequences in the dataset.

Face mark-up

Faces in the dataset have been manually marked with the same pattern as the BioID database.

Not all images are marked (there are too many of them). In each video, the following frames have marks:

1. 1.The first 10 frames.
   

2. 2.From frame 11 to frame 300, one frame in each 10.
   

3. 3.For the rest of the video, one frame in each 30 (one per second).
   

We wrote a software to help marking the videos. It is available in the “Downloads” page. The marks are saved using OpenCV's embedded XML writer.

Because not all marks are visible when the face is turned or an occlusion takes place, some are tagged as "occluded". In this case, the mark was placed tentatively, and the coordinates multiplied by -1 in the file. These appear as a white circle in the figure. We were mostly conservative, so in doubt we tagged the marks as occluded.

The marks can be downloaded [here].

Remarks

Please cite us in your publications using this dataset.  “J. Nuevo, L.M. Bergasa, P. Jimenez. RSMAT: Robust simultaneous modeling and tracking. Pattern Recognition Letters. December 2010”. [BIBTEX]

Acknowledgements

We would like to thank the drivers that volunteered to participate in the outdoor recordings, as well as the members of the CABINTEC project. Special thanks to all members of the RobeSafe Research Group who developed the hardware and software that was used to capture the sequences.

Users of RS-DMV