Up until recently, the most important question when buying a new server for enterprise software was what server could provide the required performance. The SAP Sales and Distribution (SD) Benchmark is the de-facto industry standard for measuring the performance of enterprise software on a server and its results are used by SAP in their sizing process. With increasing energy costs, the focus when buying hardware is slowly shifting towards the power requirements in your data center. SAP is supporting this trend and has joined forces with hardware and software partners to develop a “power benchmark”. On behalf of Intel, I was part of this workgroup and lived through the ups and downs of creating a new benchmark that is suited to determine the power efficiency of a server. There is even the vision that the results of the benchmark can, one day, be used to do something like “power sizing”, or at least to provide power estimates that are specific for hard- and software. However, this is a still a long way to go, and the definition of the benchmark is the first, but significant step towards this goal.
As there are already other power benchmarks available, most prominently SPEC power, one might ask, what the purpose of the SAP power benchmark is. What a lot of people don’t realize is the fact that different software performs different on hardware, performance- and power-wise. A fairly obvious example are the memory requirements for a workload--and the amount of memory has a measurable impact on the power consumption. A less obvious difference is the possibility of today’s processors that can turn off functional units like floating-point arithmetic if they are not needed by the software. These are just examples of current hardware. With an increasing number of power-saving features, the differences will become even bigger in future. Even the SPEC power committee didn’t think that they would create the one and only power benchmark. Instead, their goal was to create the framework for other power benchmarks to follow their approach.
It is therefore not surprising that the SAP power benchmark share some characteristics with SPEC power. One similarity are the different load levels. Servers are typically not used 100% all of the time. A certain headroom is needed for peak times and, especially if a server is used only in one geography, is might be mostly idle at night. In order to capture these different load levels, the SAP power benchmark runs through several phases, each using only a certain fraction of the maximal number of users. The power and throughput for this load level is the measured and the average of these ratios is used as the key performance indicator in Watts/kSAPS. Please note that this is different than simply measuring the throughput and power consumption at the average load level only. Similarly, you have a different fuel consumption when you drive 200 km/h for 30 minutes and wait 30 minutes or if you drive 60 minutes at 100 km/h.
The load levels follow a fixed pattern that includes ramping up and down the number of users. This way, power saving features must not only be able to power off certain parts of the server, but also to quickly resume to full operation. Furthermore, no manual interaction is permitted like manually shutting down servers in a three-tier setup during phases of low usage. The detailed description of the load levels and further restrictions can be found in the current draft of the benchmark specification on the SAP website.
What I have described so far covers only the SAP server power benchmark. However, there is also a benchmark in preparation that includes the complete system consisting of server and storage. Ultimately, you need both in order to run SAP software. For this reason, the workgroup is actively working to define such a holistic benchmark. Even though the benchmark itself will be different SAP server power benchmark, the power consumption of server and storage will still be measured separately, if this is technically possible. Nevertheless, the server-only benchmark has its own merits and was therefore implemented as stage 1 on the way to the complete benchmark. So, stay tuned for the first SAP server benchmark results and the full definition of the SAP system power benchmark in 2010.