Real2Sim - Real-data-based simulation and field tests for validating autonomous vehicles
The aim is to advance the automation of road traffic in Germany and to make road traffic safer, more efficient, and more accessible through Advanced Driver Assistance Systems (ADAS) and, ultimately, Autonomous Driving (AD). However, these systems must first undergo extensive testing phases—initially in virtual environments and then in real-world field tests on the road. In the context of NCAP vehicle testing (The European New Car Assessment Programme), there is an innovative approach that involves introducing digital road users, traffic signs, and other objects into vehicle systems in real time—instead of using conventional crash test dummies—so that the sensor systems perceive them as seemingly realistic objects and vehicles. This methodology is suitable for validating the safety of autonomous driving through closed-loop tests in driving simulators as well as in real mixed traffic. The advantages of this method—in contrast to pure field tests—include reproducibility and the ability to simulate critical traffic scenarios. Automotive manufacturers and suppliers can benefit from this testing capability, particularly when scenarios can be tested consistently—first as simulations and then in the field using augmented reality. Translated with DeepL.com (free version)
Real2Sim is developing a synchronous validation approach that combines real-data-based simulation with real-world field tests using augmented reality (AR)
The Real2Sim project is being implemented by EVOMOTIV GmbH and Karlsruhe University of Applied Sciences (IEEM) and is funded by the Invest BW grant program of the Baden-Württemberg Ministry of Economic Affairs, Labor, and Tourism. The research objective is to develop real-data-based simulations featuring high-fidelity environmental settings for sensor fusion systems, and to employ augmented reality approaches — both in field tests and simulations — to overlay artificial objects. In doing so, the study examines the extent to which such simulation-based tests align with real-world testing environments.
Objectives
The unlimited variety of test scenarios and combinations of different test situations enables a greater depth of testing at an early stage of ADAS/AD development. In Real2Sim, EVOMOTIV GmbH focuses on practical implementation on the vehicle as well as on the operation and commercialization of the AR-based test methodology. Karlsruhe University of Applied Sciences is developing methods for realistic simulation and validation of sensor fusion. Real-data-based simulation can serve as a virtual test platform for AD in the longer term. It can also support the development of new objects and their representations for cameras, radar, and lidar in a digital environment. The following figure illustrates the basic principle of the field test methodology. The challenge lies in augmenting objects in real time and synchronously for different sensor types.
Goals and Benefits of Augmented Reality Field Tests in the Area of Sustainability
- Safe Testing of High-Risk Situations Resource Efficiency, Reduced Effort, and Faster Testing in the NCAP Context
- Reproducibility, scalability, and depth of testing through the evaluation of risk scenarios Realistic testing and representation of objects in the perceptual environments of various sensors
- Testing of scenarios that cannot be implemented locally on-site, such as the recognition of international traffic signs
Research questions to be addressed
- What approach is suitable for creating a seamless virtual reconstruction of the environment from video environmental data for real-data-based simulations, as well as a virtual environment for virtual objects in AR field tests?
- How can the interface between vehicle sensors and augmentation be designed for sensor fusion?
- How can objects be augmented in scenes using data from different types of sensors? How can the intelligence and interactivity of the objects be designed?
- How can the user interface be implemented to create test scenarios?
Key data
Karlsruhe
Institute of Energy Efficient Mobility (IEEM)
Moltkestr. 30
76133 Karlsruhe
Post >
Institute of Energy Efficient Mobility (IEEM)
Postfach 2440
76012 Karlsruhe