Event based projects data
Event cameras are bio-inspired silicon retinas that independently detect the change of luminance in each pixel and produce an asynchronous feed of pixel coordinates where there has been a change, called events. Since no frame or image is produced, events can be detected and transmitted in the order of microseconds. The operation principle of event cameras make them suitable to perform in challenging lighting conditions and fast-motion scenarios where conventional cameras are limited. But, the asynchronous nature of events lead to the difficulty of using directly computer vision algorithms that are intended for frame-based cameras.
In this project, we present an event based vision dataset created to investigate methods for fast, robust and accurate motion estimation and environment structure reconstruction. The considerable presence of straight edges in this sequences opens the door for investigating the use of line features for SLAM.
Most sequences were developed in a human-made environment like offices, desktops, and artificial scenarios with straight-shaped patterns for experimental purposes. Additionally, the camera perform hand-movements starting at a regular speed which is later increased. The challenging lighting conditions introduced here result from turning on and off the lights of the laboratory.
This project was developed at the Institut de Robòtica i Informàtica Industrial in Barcelona-Spain, as part of the Mobile Robotics research group, where we are actively working on several research areas like SLAM, computer vision, social robotics, among others. Visit us! here
Projects
EB-SLAM: Event-based simultaneous localization and mapping
At IRI we are working with high-speed and high-dynamic-range localization and mapping devices that fuze inertial measurements with those of a dynamic vision system, these devices are commonly known as event-based cameras. Such devices can be used for estimating the movement of an autonomous vehicle or UAVs in environments without GPS readings, undergoing high dynamics, and under conditions of poor illumination or severe illumination changes. Further details on the project can be found here
Localization and mapping dataset
Each dataset consists of random camera trajectories at different speeds and lightinig conditions inside a 3D
scenario with straight-shaped landmarks.
All datasets are rosbag files for use with the Robot Operative System (ROS) and their format is similar to the one used by the RPG_DVS_ROS
driver. The datasets contain:
- Event streams at 30Hz in the topic /dvs/events using the /dvs_msgs/EventArray
message type
- Gray scale images at 30Hz in the topic /dvs/img_raw
- Camera ground truth provided by a motion capture system Optitrack in
/vrpn_client_node/camera/pose using the standard /geometry_msgs/pose_stamped
- Camera pose estimation with our event based tracking system in
/LieEKF/pose using the standard /geometry_msgs/pose_stamped
- Camera calibration information in /dvs/camera_info using the standard /sensor_msgs/CameraInfo
message type. The camera-imu calibration for Davis240C and Davis346 used for all datasets is attached HERE in a txt format
The datasets were generated with the DAVIS240-C and DAVIS346 camera model from Inivation.
These cameras provides gray scale images, event streams and IMU measurements. The datasets are classified per categories and available for download.
Some datasets include a line based markers map with 3D line endpoints summarized in YALM file. These markers are also available for printing
in 1:1 scale.
Publications
If you use this work in an academic context, please cite the following publications:
Chamorro Hernández, W. O., Andrade-Cetto, J., & Solà Ortega, J. (2020).
High-speed event camera tracking.
In Proceedings of the The 31st British Machine Vision Virtual Conference (pp. 1-12).
Chamorro, W., Solà, J., & Andrade-Cetto, J. (2022).
Event-based Line SLAM in Real-time . IEEE Robotics and Automation Letters, 7(3), 8146-8153.
Name
Description
Camera
Rosbag
Marker
Img
cube_regular_IMU
Soft dispalcement of about 0.5 m/s around a 3D cube with square patterns. Include IMU readings.
Davis240C
rosbag
Cube_marker
Name: cube_scenario
cube_fast_IMU
Fast displacement and rapid rotation changes of about speed 0.7 m/s and 14 rad/s
around a 3D cube with square patterns. Include IMU readings.
Davis240C
rosbag
trihedron_regular
Soft motion (approx. 0.5 m/s) around a wide scenario made of straight patterns.
Each face is a square with a 0.5m side. No IMU readings.
Davis240C
rosbag
Trihedron_marker
Name: trihedron_scenario
trihedron_fast_light
Strong hand-shake movement (>1m/s) and quick rotation changes, the illumination changes
during the rosbag execution by turning the lights ON and OFF. No IMU readings
Davis240C
rosbag
plane_regular_IMU
Regular displacements (approx. 0.4m/s) and rotation changes around a planar marker. Include IMU readings
Davis346
rosbag
Plane_marker
Name: line_scenario
plane_fast_IMU
Strong camera hand-shake (approx. 1m/s) and rotation changes around a planar marker. Include IMU readings
Davis346
rosbag
4-bar_test1
Very fast motion of a marker placed in a four-bar mechanism in which a DC motor powers
the device and increases the speed gradually from 360 rpm to 1000 rpm.
The camera is static and the marker is moving. No IMU readings
Davis240C
rosbag
Marker_4bar
Name: moving_scenario
4-bar_test2
Very fast motion of a marker placed in a four-bar mechanism in which a DC motor powers
the device and increases the speed gradually from 100 rpm to 1000 rpm.
The camera is moving and the object static. Include IMU readings
Davis346
rosbag
Marker_4bar
Name: plane_scenario
office_L_shape
Regular motion around a desktop with an L shape. It contains straight objects, and an initial
plane marker to start (if needed). Include IMU readings.
Davis346
rosbag
Plane_marker
Name: office_L_shape
office_large
Displacements around a large office. After an exploration phase the sequence includes
strong motion (>1m/s) and challenging lightinig conditions (lights on and off).
The sequence also contains a planar marker to initialize (if needed).
Include IMU readings.
Davis346
rosbag
Plane_marker
Name: office_large
office_far
Office sequence with several distant objects and weak edges.
The sequence also contains a planar marker to initialize (if needed).
No IMU readings.
Davis240
rosbag
floor_marker
Name: office far
desktop_straight_shapes
High concentration of straight-shaped objects placed on a desktop observed
at a regular speed.
No IMU readings.
Davis240
rosbag
plane_marker
Each dataset consists of random camera trajectories at different speeds and lightinig conditions inside a 3D scenario with straight-shaped landmarks. All datasets are rosbag files for use with the Robot Operative System (ROS) and their format is similar to the one used by the RPG_DVS_ROS driver. The datasets contain:
- Event streams at 30Hz in the topic /dvs/events using the /dvs_msgs/EventArray message type
- Gray scale images at 30Hz in the topic /dvs/img_raw
- Camera ground truth provided by a motion capture system Optitrack in /vrpn_client_node/camera/pose using the standard /geometry_msgs/pose_stamped
- Camera pose estimation with our event based tracking system in /LieEKF/pose using the standard /geometry_msgs/pose_stamped
- Camera calibration information in /dvs/camera_info using the standard /sensor_msgs/CameraInfo message type. The camera-imu calibration for Davis240C and Davis346 used for all datasets is attached HERE in a txt format
Publications
If you use this work in an academic context, please cite the following publications:
| Name | Description | Camera | Rosbag | Marker | Img |
|---|---|---|---|---|---|
| cube_regular_IMU | Soft dispalcement of about 0.5 m/s around a 3D cube with square patterns. Include IMU readings. | Davis240C | rosbag | Cube_marker |
Name: cube_scenario |
| cube_fast_IMU | Fast displacement and rapid rotation changes of about speed 0.7 m/s and 14 rad/s around a 3D cube with square patterns. Include IMU readings. | Davis240C | rosbag | ||
| trihedron_regular | Soft motion (approx. 0.5 m/s) around a wide scenario made of straight patterns. Each face is a square with a 0.5m side. No IMU readings. | Davis240C | rosbag | Trihedron_marker |
Name: trihedron_scenario |
| trihedron_fast_light | Strong hand-shake movement (>1m/s) and quick rotation changes, the illumination changes during the rosbag execution by turning the lights ON and OFF. No IMU readings | Davis240C | rosbag | ||
| plane_regular_IMU | Regular displacements (approx. 0.4m/s) and rotation changes around a planar marker. Include IMU readings | Davis346 | rosbag | Plane_marker |
Name: line_scenario |
| plane_fast_IMU | Strong camera hand-shake (approx. 1m/s) and rotation changes around a planar marker. Include IMU readings | Davis346 | rosbag | ||
| 4-bar_test1 | Very fast motion of a marker placed in a four-bar mechanism in which a DC motor powers the device and increases the speed gradually from 360 rpm to 1000 rpm. The camera is static and the marker is moving. No IMU readings | Davis240C | rosbag | Marker_4bar |
Name: moving_scenario |
| 4-bar_test2 | Very fast motion of a marker placed in a four-bar mechanism in which a DC motor powers the device and increases the speed gradually from 100 rpm to 1000 rpm. The camera is moving and the object static. Include IMU readings | Davis346 | rosbag | Marker_4bar |
Name: plane_scenario |
| office_L_shape | Regular motion around a desktop with an L shape. It contains straight objects, and an initial plane marker to start (if needed). Include IMU readings. | Davis346 | rosbag | Plane_marker |
Name: office_L_shape |
office_large | Displacements around a large office. After an exploration phase the sequence includes strong motion (>1m/s) and challenging lightinig conditions (lights on and off). The sequence also contains a planar marker to initialize (if needed). Include IMU readings. | Davis346 | rosbag | Plane_marker |
Name: office_large |
office_far | Office sequence with several distant objects and weak edges. The sequence also contains a planar marker to initialize (if needed). No IMU readings. | Davis240 | rosbag | floor_marker |
Name: office far |
desktop_straight_shapes | High concentration of straight-shaped objects placed on a desktop observed at a regular speed. No IMU readings. | Davis240 | rosbag | plane_marker |
|