Integration. A word to scare children with? Maybe not, but it definitely is one of the hardest parts of system engineering and building. When different pieces of hardware, firmware, and software are combined to build a complete system, all kinds of “fun” issues can arise. The classic way to build computer systems and software systems often had a “big bang integration” towards the end, where somehow magically all the pieces would come together and work. We all know that was a bad idea, and hence the move towards continuous integration and doing integration in small steps from the bottom up.
A Tangled Problem
For computer chip and system-on-chip design, integration has to be done pre-silicon in order to find integration issues early so that designs can be updated without expensive silicon re-spins. Such integration involves a lot of pieces and many cross-connections. Even if we only include a couple of IP blocks the picture becomes rather tangled:
For each IP block, there is typically firmware to integrate onto the block. The combined IP block will have an interface to its software driver, which in turn integrates to the operating system. In some cases, accesses from the driver and OS all go to the firmware, while in other cases there might be direct hardware accesses in addition to the firmware-mediated accesses. Each IP block will communicate with other blocks, via direct lines or Networks-on-Chip (NoC) or buses. Power management hardware and firmware will be controlling the activity of all blocks in the chip. The UEFI (Unified Extensible Firmware Interface) or other BIOS and boot code that brings up the system will need to access hardware to take inventory, bring it up, and enable some of it. The software drivers for IP blocks sometimes load the actual firmware onto the IP blocks, and are thus responsible for booting them.
In short, there are many scenarios to consider and test, across quite a disparate set of components and types of software, firmware, and hardware.
Simulating the Tangle
In order to do integration in pre-silicon, we need to build virtual platforms that provide a complete system setup – from the “obvious” main cores running the main software stacks, to the obscure processor cores inside the IP blocks that run firmware.
Such models are built as part of the overall virtual platform development task, but just as often there are existing models from IP block teams and IP vendors that can be used to quickly get a model in place. Such models come from many different sources, and are written using a wide variety of frameworks. A particularly common case are accelerators for functions like graphics, media, and networking – in such cases, the hardware designers tend to use simulators to design the system. Looking beyond the boundaries of the computer system, we also often need to include simulators for physics and mechanics to build a truly complete system.
The overall picture comes to:
We have many different groups, each contributing their models built using their own favored modeling technologies, and need to pull it all together into a single platform that can run real software and looks like the real thing to software.
One way to do this is to build ad-hoc point integrations between different simulators in order to tackle particular problems. This has the potential to create quite a few separate integrations, such as:
This works well for a few models – but when the number of models starts to increase and the number of combinations goes up, it quickly becomes an exercise in futility (in theory, for each model you add, the number of possible combinations more than doubles – for example, at 4 models we have 11 combinations, and for 5 models, there are 26 possible combinations).
Building Integrations with Simics* as the Base
A more practical solution is to build adapters or integrations from each model into a common base such as Simics:
In this case, we only need to build a single integration for each model, and then any combination of models can be produced by fitting different models into the common base simulator, Simics. This approach makes it feasible to deploy arbitrary combinations of models, facilitating integration testing across blocks.
The Simics framework has proven to be pretty good at this over time, thanks to a few core technology choices that actually date back to the earliest days of Simics.
First of all, Simics was designed to support multiple languages for modeling. C, C++, Python, SystemC are all supported as standard languages – but we have also seen users integrate code in other languages like Matlab, CUDA, and Java into Simics modules over the years.
Second, Simics uses the host platform C-level ABI for linking models and for the interfaces between models. This ABI is the same regardless of the compiler used, avoiding the need to specify a specific compiler version for model builds. It is normal for system models to be built with models compiled using a range of compilers and compiler versions.
Third, Simics packages models into binary modules. This separates the model build from the user’s system configuration. A user of a model does not need access to the source code, nor do they need to care about how a model is built.
Fourth, communication between models is performed using a set of Simics interfaces. This provides interoperability via a common standard, and ensures that models can connect to Simics platforms and Simics features like PCIe and Ethernet network simulation. It does not mean that the interfaces are static over time – rather, they have evolved as needed to support new use cases.
Fifth, the Simics API is deep and open to all users, making it possible to write powerful integrations without relying on the Simics base product to add specific support.
Heterogeneous Virtual Platforms Built with Simics
The result is that Simics platforms are made up of a heterogeneous set of components. Normally, there is a base platform modeled using Simics directly – providing the main processor cores and chips of an Intel platform, for example. The Simics platform is fast and can boot and run operating systems and complete software stacks.
Then, additional components are added to the base, or components from the base platform are replaced by more detailed models. In many cases, we have both a base Simics model of a device or subsystem, and a more detailed integrated simulator.
For example, an audio subsystem can either be simulated as a fast functional model at its interface to the rest of the system, or we can use a full “white box” model that contains models of the processor cores and devices found internally in it, and that runs firmware to enable integration testing between drivers and firmware.
Another example shown in the picture above is replacing parts of the platform with actual RTL (Register-Transfer Level) running on emulators, simulators, or FPGA prototypes. In this case, transactors are used to connect RTL to the transaction-level simulator, usually running the RTL on some form of external hardware box in order to make it fast enough to be useful.
On the left in the above picture, we also see the example of Simics being integrated with environment and world models from the physical domain. I have a longer discussion on this particular case at http://jakob.engbloms.se/archives/2116.
What about Performance?
Building virtual platform by integrating many disparate parts can have a performance impact. However, that is usually not caused by the integration per se. From experience, the effect of translations between interfaces has a very small impact on overall simulation performance. What tends to cause performance issues are rather three different effects.
The first is that when you put more simulators together, the overall work needed to run the simulator goes up. Which means that the combined system runs slower even if all parts are nicely optimized. There is an unavoidable difference between simulating one processor and simulating a hundred. The way to mitigate this issue is to make sure the setups used are appropriate for each test – to not always run with the most complete and complex possible platform.
Second, some models work well on their own but do not work ideally when integrated in a bigger context. For example, driving a model forward with lots of simulator events tend not to be all that bad when running a model on its own – but it really hurts when integrating a model in a bigger context. This is usually pretty easy to fix once the problem has been identified.
Third, the integrated models might be slow by design, which will slow down the overall integrated platform. A cycle-level model that faithfully models the microarchitecture of a subsystem is going to run a lot slower than a transaction-level model (say from 1000 to 100000 times slower), and that is going to impact the overall speed of the platform. The solution to this is to pair a detailed model with a fast model, minimizing the amount of time spent using the detailed model when its details are not really needed.
So overall, integrating many different simulators won’t in itself necessarily hurt performance compared to building a homogeneous platform from scratch.
Integrating the Virtual Platform into Other Systems
Another aspect of integration that might not be immediately obvious is that the virtual platform itself is something that is integrated into higher-level flows. In most cases (at least in terms of simulated hours and number of simulation runs), VP models are run from some kind of automatic test system or launcher system, rather than as interactive runs on user’s desktop.
For such cases, it is really helpful to have a common simulator platform that encapsulates all other models. The higher-level systems can be written to make use of a single tool, regardless of the internal make-up of the model that is being used. By providing a consistent automation and encapsulation interface, Simics makes it possible to build reusable test infrastructure that can work across different targets and different configurations of the targets. There does not really need to be any native Simics models in the target virtual platform at all – it is still beneficial to integrate with Simics just to fit into the infrastructure built around it.
Such integrations also last all the way from pre-silicon to post-silicon, deployment, and maintenance. Getting automation in place is a key piece of modern software development, and virtual platforms can really help with that discussed previously in this blog.
Supporting integration with virtual platforms is necessary in order to do system integration early. To build complete platforms that can run all required software loads (in particular, firmware), the virtual platforms are often built as integrations of various pre-existing models and parts. Such integrations provide a way to quickly get virtual platforms in place that have sufficient detail to run all the software, while still providing uniform packaging towards other systems.
Read all of Jakob Engbloms's post.