In the event that you are here and understand Scenario-Based Testing, odds are you’re keen on getting in touch with OSC/ADAS V&V space specialists here at Vayavya Labs.
There has been expanding interest in building more secure innovation in the automobile business. When technology can help overcome human error and save a lot of lives, why not? Autonomous Vehicles (AV) and Advanced driver assistance systems (ADAS) are here to assist the human driver by reducing the effort in driving and trying to avoid situations that a typical driver would most likely not be able to do. A key question to answer here is – when an AV is deployed, how safe is the vehicle to drive on the road? Has it been validated enough and if so what cases did it cover during the validation? This is where the role of verification and validation comes into the picture. The most commonly used methodology of validation is calculating the number of miles driven by the vehicle wherein the vehicle drives in real-time in a test region. “There’s an estimate that to get fully autonomous driving you will have to drive 8 billion to 10 billion miles”, says Tony Hemmelgarn, CEO of Mentor, a Siemens business, and president and CEO of Siemens PLM Software. Driving these many miles won’t be a reliable engineering solution, as the probability of the vehicle coming across edge cases is very low and it involves safety risks and consumes enormous resources. Scenario-based testing & verification enables the detection of possible edge cases using minimal resources and ensuring maximum safety within a short time is thus being widely used in the industry.
The Scenario-based simulation addresses this issue of known and unknown edge cases. A scenario is typically a test case for a particular functionality and in autonomous vehicles, it is a typical driving situation on the road. Let’s take a simple example of an ego (System under test for AD/ADAS functionality) driving on a road with no other vehicle around it (Figure 1). Now, this is called a scenario and this can be built in different ways. While we can build this on a simulator like Carla, IPG Carmaker, there is this new way of achieving this using “ASAM Openscenario 2.0”. Openscenario 2.0 is an ASAM standard with a language and domain model. The aim of Openscenario 2.0 is to develop, verify and validate automated driving systems. For a higher level of autonomy, scenario-based testing becomes more critical as the system requires more situations to be experienced and decisions to be computed. The goal is to detect unknown edge cases and make them known to the function in order to make the system to be a safer one.
Figure 1
The scenario design can be a concrete design or an abstract one. While a concrete scenario helps in taking the system to the extremes of its design, it is typically an abstract scenario that would probably find the unknown edge cases. Typically an ADAS/AV function designer should be able to assess where his/her system might fail or at what configuration it should be tested so that it is difficult for the system to take the decisions while maintaining SOTIF(Safety of the Intended Functionality).
Let’s take a look at one of the very common road scenarios to bring more clarity to concrete and abstract scenario design. Figure 2 shows a typical case of Ego driving and a Pedestrian crossing their driving path from the right side of the road. Now an ADAS/AV function like Automatic Emergency Braking (AEB) would help in ensuring that the ego does not hit the pedestrian while maintaining the safety of the person inside the car. So if we have to design a scenario for this situation, again we have the option of writing a concrete or an abstract scenario. Figure 2 is an example of a concrete scenario where the drive parameters are predefined to be at 80kph for our ego and 8kph for the pedestrian with pedestrians approaching the ego from the right. Figure 3 is an example of abstract scenario design where the intended scenario happens but you are not necessarily controlling any parameters of the ego’s drive or the pedestrian’s movement. Imagine you were to brew yourself a cup of coffee, and you would be needing to provide water and coffee beans to the coffee machine. You would need not worry about how the coffee machine is actually doing it internally. Similarly, in the scenario, you would provide a pedestrian but you would not be controlling how it approaches the ego and thus creating a huge number of drive combinations of the same scenario.
Figure 2 – Concrete scenario
Figure 3 – Abstract scenario
The VE (Verification environment) has to be scalable, reusable, extensible, and portable which is the key to ADAS/AV verification and validation. And thus simulation-based verification is becoming mainstream in the automotive industry.
In our next blog, we will dive deeper into the world of concrete and abstract scenarios. We will use Open Scenario 2.0 code snippets to drive home the relevant points. Stay tuned!!
