Institute for Communication Technologies and Embedded Systems

Multi-Scale Multi-Domain Co-Simulation for Rapid ADAS Prototyping

Problem Description

Over the last several decades, vehicles have evolved from mainly mechanical machines to sophisticated hardware and software dominated systems. The origin of this paradigm shift towards vast digitalization lies in the integration of novel vehicular functionalities, such as safety and comfort features. Such applications are manifested through conventional Advanced Driver Assistance Systems (ADAS), i.e., vehicle-based intelligent safety utilities. However, in recent years another technological paradigm shift has been taking place: the pursuit of highly automated and autonomous driving. The primary goal of such approaches is to enable a step-wise transitioning of automotive control functions from humans to an in-vehicle automation system. Yet, these trends impose a major complexity explosion on the developed ADAS-related in-vehicle HW/SW infrastructure. Herein the quantity of integrated SW exploded in recent years nearly exponentially, overgrowing multiple vast application areas, ranging from infotainment, over safety-critical ADAS, heading all together towards highly automated and autonomous driving. With sophisticated vision-based and machine learning applications on the rise, a high-end vehicle of the upcoming decade might integrate up to 200-300 million Lines of Code (LoCs). To support the performance demands of this vast amount of application SW, the number of HW controllers also increased. The trends herein show a strong initial increase in the utilized Electronic Control Units (ECUs) over the past, alongside a reduction of devices in upcoming years. This anticipated decline can be explained with  the increase in multi-processor architectures employed in cars, implying a trend towards more integrated architectures, accommodating powerful multi and many-core devices. In summary, with tomorrow's high-end cars running hundreds of million LoCs on over 70 multi-core ECUs distributed over 10 buses and 8000 signals, the design and validation of the in-vehicle HW/SW infrastructure has become immensely complex.

Possible Solutions



a) System Engineering via the V-Model

A rather classic approach to cope with the resulting design complexity is to follow standardized system development processes, such as the reputable V-Model. The depicted figure shows a specific extension to the original model, which is adapted explicitly for safety-critical ADAS. The V-model consists principally of a top-down Design phase, a bottom-up Validation phase and an iterative Refinement phase. As can be seen in the figure, the V-model further subdivides different stages of the system development process, e.g., to capture functional and safety requirements, set specifications, design and implement subsystems, and finally to validate these and integrate them into a target system. Although the model helps greatly reducing late nonconformities, its iterative design-validation-refinement cycle lengthens the development process and thus increases its cost. To bridge this gap, dedicated simulation-driven approaches emerged which enable early system exploration prior to implementation.

b) Model-Based Design

Model-Based Design (MBD) is a divide-and-conquer approach for describing complex composite systems of systems in a more structured, hierarchical manner. The MBD methodology is manifested via graphical modeling and simulation frameworks, commonly referred to as MBD tools. Such tools frequently offer facilities for simulation-driven algorithmic prototyping, in tool testing and certified code generation. Thus, MBD tools have become popular in the development of vehicular embedded SW. The importance of model-based design is highlighted by industry studies where MBD tools helped to reduce ADAS turnaround times by shortening the aforementioned design-validation-refinement cycle by up to 50% reportedly. This quantification indicates the importance and future potential of model-based ADAS design. However, MBD tools have certain general drawbacks that limit their utilization:

  1. Usually such frameworks provide only artificial Inputs/Outputs (I/O) for simulation purposes, limiting the integrated in-tool testing capabilities.
  2. MBD tools support generally functional aspects of ADAS design only. This restricts representing the temporal behavior of model implementations.
  3. Frequently such tools are bound to their modeling environment, i.e., there is only limited external connectivity to further simulation ecosystems.

c) Traditional XiL Validation Techniques

Addressing the first limitation of MBD tools, so-called X-In-the-Loop (XiL) methods are provided to extend testing capabilities. Particularly, Model-, Software-, Hardware and Vehicle-In-the-Loop (MiL, SiL, HiL, ViL) techniques are classic approaches that help reducing late nonconformities by refining validation steps. The following XiL procedures are presented (in the order of increasing effort and cost):

  • MiL: Refers to the method of algorithmically prototyping ADAS as an executable specification which functionally matches the development objective. Recalling the previous figure, this approach can be used in the early design phase of the V-model.
  • SiL: Describes the approach which validates the ADAS model’s SW implementation in a simulation system which is similar to the final HW target in terms of computing power and real-time operation. As shown in the previous figure, the SiL approach can be used in the late design and early validation phases of the V-model.
  • HiL: Herein the created SW implementation of the ADAS is embedded on a target HW device, where a combination of simulation and hardware-based component tests validate the functional and temporal behavior of the model. As indicated in the previous figure, HiL testing can be used in several later validation steps of the V-model.
  • ViL: Refers to the road test based evaluation of the final ADAS HW/SW implementation integrated in a specially augmented experimentation vehicle. As shown in the figure, ViL can be used in the latest validation steps of the V-model.

Even though XiL methods vastly enhance ADAS design and validation, HiL and especially ViL requires a greater effort and cost. Furthermore, field tests make it difficult to reproduce scenarios, while also increasing the risks associated with testing in open environments. However, this project proposes several innovative simulation-driven approaches to fully virtualize these processes.

d) Utilization of Driving Simulators

The conduction of road tests is unavoidable to finalize ADAS integration. Yet, virtual
Vehicle-in-the-Loop (vViL) validation, i.e., the use of driving simulators can overcome
the aforementioned limitations of field testing. Furthermore, vViL can augment and
improve multiple steps in the V-model, from early design to late validation phases, as
shown in Figure 1.7. Moreover, recalling Section 1.2.2, such frameworks address the
main constraints of MBD tools by providing realistic I/O via their virtual driving en-
vironment. This enables ADAS evaluation/refinement using virtual test drives, while
having the freedom to experiment with various traffic and environment conditions, as
well as vehicle types (e.g., cars, trucks). Moreover, in contrast to road testing, driving
simulation implicitly ensures the deterministic repeatability of test scenarios, which
is especially crucial if spurious errors occur (e.g., when integrating multiple ADAS).
Lastly, such tools can also interactively involve the developer in the ADAS virtual test
driving loop, thus capturing human reactions instead of using artificial control only.

However, driving simulators are generally limited to integrate only functional
manifestations of ADAS models only. In addition, they are restricted to their confined
simulation environment, making a joint usage with further tools troublesome.


e) System Exploration via Virtual Prototyping

A promising approach to extend the simulation scope of ADAS to the embedded HW/SW implementation is virtual prototyping. This methodology builds upon so-called System Description Languages (SDLs) which provide a highly specific formalism to create models of real HW modules. Among these languages, SystemC has become the most prominent one for Electronic System Level (ESL) design. SystemC is a regular C/C++ library that supports functional, temporal and concurrency modeling of system HW components (e.g., CPUs, buses, memories, peripherals). By using SystemC, so-called Virtual Platforms (VPs) can be assembled, which are the system-level hierarchical ensemble of such HW models and their interconnects.

VPs enable HW/SW co-design by also simulating the complete ADAS SW layer as executed by the virtual HW system. Thus, VPs assist and accelerate system proto- typing by maintaining full HW/SW visibility, debuggability and non-intrusive monitoring, while ensuring simulation determinism. In contrast to MBD tools and driving simulators, VPs enable exploring beyond-functional system attributes, e.g., HW/SW partitioning, task mapping, schedulability and worst-case execution time analysis.

Thus, VPs provide a promising way to decrease ADAS design complexity and to bridge the verification and functional safety gap of the digital in-vehicle infrastructure. Recalling the V-model, VPs are envisioned to augment or even replace HiL in multiple validation steps of the V-model via virtual Hardware-in-the-Loop (vHiL) testing.

However, similarly to driving simulators, VP frameworks are also bound to their closed environment, i.e., external connectivity to further modeling ecosystems is highly limited. Moreover, VPs are vast simulation systems that are usually constrained by either their execution speed or modeling accuracy which issue is addressed in the following.

f) Multi-Scale Simulation

The first methodological pillar of this work is manifested via the multi-scale simulation approach. The general idea behind multi-scale simulation is to provide a generic mechanism for large-scale simulation ecosystems to integrate their models on different design abstractions. The main benefit herein is providing a tunable resolution for various models of a top-level system, thus enabling the possibility to find an optimal trade-off between the execution performance and accuracy of the simulation.

There are multiple possibilities to implement the multi-scale approach, most of which focus on reducing model complexity. One alternative is to simplify the description of a model by reducing its state space, which approach is often applied in continuous-time systems. A further, more generic option is to abstract away certain modeling attributes which are selected via an expert insight in the top-level system. Especially the latter idea is utilized in this project on simulation subsystems which were identified as the most performance-critical: the vehicle dynamics within a driving simulator and the VP-based HW/SW simulation.

g) Multi-Domain Co-Simulation

As can be seen, the proposed complete virtualization of XiL methods requires the simultaneous utilization of multiple simulation ecosystems. This is explained with the multi-disciplinary nature of vehicular systems, requiring specialized simulators that are best suited to implement a formalism representing an individual domain. However, since strong inter-dependencies exist between such subsystems, all aforementioned tools/models need to be interconnected and synchronized to capture essential interactions. Although this functionality could by achieved by point-to-point tool connections, custom inter-linkage does not scale sufficiently with an increasing number of simulation systems to be integrated.

To cope with this problem, multi-disciplinary cooperative simulation, i.e., multi-domain co-simulation standards emerged. Such formalism-agnostic specifications extend the boundaries of application-specific simulation tools/models by providing standard means for cross-domain interconnection and joint control. The multi-domain approach enables whole-vehicle simulation for providing early safety guarantees for the ADAS design cycle. Thus, as a second methodological pillar of this project, a collection of multi-domain techniques will be explored to join the aforementioned vast simulation environments of MBD tools, driving simulators and virtual platforms, among others.

Overall Project Overview

The main contributions of this project revolve around incorporating all previously introduced tools, techniques and standards into one comprehensive framework system with the aim of virtualizing all XiL processes of the V-model. A simplified overview of the envisioned fully virtual ADAS prototyping ecosystem and the corresponding design flow is shown in the provided figure. The proposed infrastructure virtualizes and thus vastly accelerates the aforementioned iterative ADAS design-validation-refinement cycle, which in the new simulation-driven form consist of the following steps:

1.) Model-Based ADAS and Vehicle Dynamics Design: Following the provided figure, the developer may first algorithmically prototype an idea of a novel driver assistance application using an MBD tool of choice. In the envisioned flow, the modeling of the controlled system, i.e., the vehicle dynamics, was herein defined an indispensable step, since control actions of the target ADAS rely primarily on the underlying system.

2.) Virtual System Integration: Next, the designed ADAS prototype needs to be integrated onto a target VP for vHiL validation. In this project, multiple SW integration methods are thus explored, ranging from simple bare-metal run-time environments to dedicated automotive real-time operating system. Alongside the algorithmic implementation, the designed vehicle dynamics need to be provided for the driving simulator, targeting its utilization for vViL testing. With both of these steps, the emphasis is placed on design automation, i.e., generating all possible models from within an MBD tool to achieve system integration most reliably and rapidly.

3.) Virtual System Testing and Verification: As shown in the figure, the next step consists of applying multi-domain co-simulation to enable the fully virtual XiL testing of the system. Herein, driving scenarios may be conducted that involve the target VP, which by executing the full ADAS SW layer, controls the instantiated virtual vehicle within the driving simulator. The goal of this step is to behaviorally and temporally evaluate the overall ADAS HW/SW infrastructure in a realistic whole-vehicle simulation.

4.) Refinement: Lastly, based on the selected evaluation criteria, the developer can assess the algorithmic and/or execution performance of the designed ADAS prototype (or the virtual vehicle for that matter). Thus, necessary refinement steps may be performed and the iteration can recommence until the system’s design goals are met.

The implementation of the aforementioned design automation steps are manifested via several research works, conducted within the scope of this project. Each of these endeavors are given thought sub-frameworks (also depicted in the figure) as follows:

A) Multi-Scale ADAS Code Generator: Illustrated in part A), a retargetable “one-click” ADAS model generation facility is integrated in form of a plugin of an MBD tool. Following the suggested multi-scale approach, the proposed framework enables ADAS refinement over multiple levels of abstraction, implementing incremental targets for algorithmic exploration, task encapsulation and embedded real-time code generation. Thus, this methodology simultaneously addresses performance limitations of VP-based simulations and continuous ADAS refinement and integration.

B) Multi-Scale Vehicle Dynamics Generator: Alongside ADAS generation, a second MBD plugin is used in this project, focusing specifically on the design of vehicle dynamics. Depicted in part B), this code generation facility enables a hand-in-hand controller and controlled system co-design in a fully virtual development environment. Moreover, this framework also follows the proposed multi-scale approach, enabling a tunable resolution for the designed mechatronic models, thus addressing possible performance issues of vehicle dynamics simulation.

C) Virtual Platform Integration: Highlighted in part C), a significant research effort aims at the integration of VPs into the given multi-domain co-simulation ecosystem for vHiL testing. Moreover, to address safety regulations, system-level fault injection is herein implemented for functional safety testing and for increasing the overall robustness of the HW/SW infrastructure.

D) A Joined Multi-Domain Co-Simulation Ecosystem: Herein an underlying joint framework system is referred to, interlinking all aforementioned tools and models, as depicted in parts D1-D2. This research effort corresponds to the infrastructural assembly and optimization of the fully virtual ADAS prototyping ecosystem. As indicated in the figure, a generic compliance method is deployed here for all involved frameworks (e.g., virtual platform, driving simulator, among others) to a selected multi-domain co-simulation standard. Moreover, since the pursuit of near real-time whole-system simulation execution is of utmost importance, this infrastructure utilizes multi-scale techniques to overcome the critical performance bottlenecks of the whole system. The resulting infrastructure provides a research vehicle for ADAS prototyping by incorporating and virtualizing all XiL testing and validation methods.