There are many resources about how to develop applications to work with the Intel AMT platform, but there is no such as availability when it’s about executing a testing process; this article defines some test cases based on the test scenarios described in the “Validation Design Guide” included in the Intel AMT SDK.
The first sample scenario We are going to work with is called: “A single Management Console running against the system containing Intel AMT”; this scenario describes how a management console can access to a client Intel AMT machine (thought the C static libraries and the SOAP Web Services) to use its functionality (non volatile storage, SOL/IDER, setup, asset inventory and so on). Picture 1 shows this scenario.
Picture 1. Test Scenario 1.
We can add these constraints to the Test Scenario 1: LAN connection can be secured (HTTPS) or non-secured (HTTP), machines can run any Operative System (Windows or Linux) and the managed machine expose the same functionality at any ACPI Power State.
The one million dollar question is: ¿how would you test a management console? Answers can vary, but personally I try to categorize and organize the test cases according to: ¿what is it going to be tested? ¿what’s the point of view? ¿what’s the level? And ¿what the techniques are?
To clarify the first question, we can group the testing features by: the functional requirements (derived from use cases), the non functional requirements (derived from capabilities of software) the application’s code itself (derived from the Architecture and the developer’s style).
If we check the Test Scenario 1 twice we can find some functional requirements: non volatile storage, SOL/IDER, setup and asset inventory. As for the non functional requirements, the test Scenario 1 enumerates the secured communications, the operative system configuration and the available power states; but we need to reduce the ambiguity and ask for the capabilities that software have to meet. Picture 2 shows some useful metrics that we can use to compare with.
Picture 2. Some useful metrics.
*Storage Partner capacity depends on some variables like the number of registered applications or if the application is from a partner; so this value is a relative average.
The information sources from Picture 2 are: the Storage Design Guide, the Network Interface Guide and the Validation Design Guide; although the above information is not complete or updated, it gives us some useful metrics. By other hand, we don’t have into account the code testing because the management console is a client of the C static library and the WSDL files (and we don’t have their source code).
Until this point we have defined the scope and we have found many features, so for simplicity we choose only the Partner Storage feature to test; also we define this configuration: Windows XP Professional (32 bits), no secured communications (HTTP), no LAN (direct connection, static IP addresses, no DNS/DHCP server) and Intel AMT Release 2.0.
Now we answer the question: ¿what’s the point of view? We can choose black box, White box and/or gray box; we choose black box because is better for functional and non functional testing. Finally, we answer the question ¿what’s the testing level? We can choose system, unit and/or integration; we choose system for keeping things simple.
With all that information gathered we can derived these two samples from the Test Scenarios (TS):
• TS 1.1: The management console writes in the Storage Partner a 96 Kb character string containing a virus signature.
• TS 1.1: The management console reads in the Storage Partner a 96 Kb character string containing a virus signature.
With the Scenarios 1.1 and 1.2 we can design the following Test Cases (TC) based on the functional techniques (to keep things simple we don’t cover partner registration or account support):
Note: Due to ACPI state transitions –shown in Picture 3- we can obtain many sub test cases, so we perform an all-pairs analysis to obtain these test vectors: Working (S0) –for reading- and Turned-Off (S2) –for writing-.
Picture 3. ACPI state combinations.
• TC1 – Random input for Reading/writing: we can search a random string generator to create some data to write them to the Partner Storage (by using another tool, not the console). Then we can verify how the management console handles that bad data.
• TC2 – Heuristic Read/Write time: we can set an oracle metric (1 second per Kb for example) and then verify if our console meets that criteria.
• TC3 – Storage capacity: we can verify if the management console can write and read up to 96 Kb by testing 96 times and incrementing 1 Kb until complete the total capacity, or we can better define an equivalence partition so we can reduce the number of tests to one (only 96 Kb). We can also complement this test case with a boundary analysis of the 95 Kb and the 97 Kb to check the application’s behavior.
Based on the structural techniques we can obtain the following test cases:
• TC4 – Exception Management: we can imagine all the possible failure scenarios and then execute them to exercise the error handling code; some of the most common errors are: lose of session, lose of connection, bad parameters and so on.
• TC5 – Web Services calls analysis: we can dynamically analyze the console code to find statements which ones are never used and to profile the functions performance; also we can statically analyze the code to verify that Web Services calls meet the 18 limit.
Based on the capabilities of the software we can obtain the following test cases:
• TC6 – Read/write operations under stress: we can break the 96 Kb capacity limit (with 200 Kb for example) and the connections concurrence limit (with 30 SOAP calls for example ) to check the console behavior.
• TC7 – Security of the write/Read operations: we can verify the integrity of the virus signature when it’s read/written/updated/deleted (CRUD design pattern).
• TC8 – Globalization of data being read/written: we can perform some internationalization testing to verify that only supported character sets are being read/written by the management console.
Conclusion
This article is generic, conceptual (without tools) and an introduction to provide an example of how to initiate a test process to a management console for an Intel AMT machine. More test cases can be run, but this is a good start point.
Javier Andrés Cáceres Alvis
http://speechflow.spaces.live.com/
http://software.intel.com/en-us/blogs/author/javierandrescaceres/
