While the WCET bounds computed by aiT are safe, they are also very tight. Overestimation of the actual WCET is kept extremely low. This has been repeatedly proven by our own tests, customer feedback, and independent reviews alike.
| Experiment | Target processor | Average WCET overestimation | |
|---|---|---|---|
| Other methods | aiT | ||
| Review by Airbus | MPC755 | “useless” | <25% |
| Review on Volvo code | C16x | 212% | 16% |
| Tests with ASCET | ST10 | ? | 3% |
| aiT vs. ARMulator | ARM7 | ? | 5% |
| WCET Tool Challenge 2006 | C16x, ARM7, MPC565 | >81% | 7–8% |
| Study by University of Stuttgart | ARM7 | overestimation not explicitly tested | |
| Study by University of Erlangen–Nuremberg | ARM Cortex-M4 | 96% | 23% |
| Study by McGill University | XMC4500 (Cortex-M4) | 33–59% | 4–15% |
Review by Airbus France
(aiT for PowerPC 755)
“The comparison shows that the WCET [computed by aiT] typically is about 25% higher than the measured time for the same task, the real but non-calculable WCET being in between. Another comparison is worth to mention: the one between aiT’s results and those of Airbus’ traditional method. As predicted when the decision for using aiT was made, the figures obtained by the traditional approach are a lot higher than those produced by the aiT-based method. Actually the overestimation is such that the traditional figures are useless.”
The review was conducted with an early prototype of aiT in 2003.
The accuracy has only kept improving with every stable release ever since.
Review by Mälardalen University on Volvo code
(aiT for C16x)
In his master’s thesis titled “Static WCET Analysis of Task-Oriented Code for Construction Vehicles”, Daniel Sehlberg found that aiT overestimated the measured WCET by an average of only 16%. Volvo’s traditional approach overestimated the WCET of the same tasks by an average of 212%.
Tests with ETAS’ ASCET
(aiT for ST10)
aiT supports tight integration with the ASCET modeling tools from ETAS. The user can start aiT directly from the ASCET project editor menu to perform a WCET analysis of the automatically generated ECU code.
The following table shows a comparison of maximal measured run times and WCETs predicted by aiT for a sample engine throttle control module. The module was specified in ASCET and compiled with the TASKING C compiler v7.5 for an ST10F269 microcontroller board. Run times were extracted from bus traces (iSYSTEMS ILA 128 logic analyzer). The experiment showed an average overestimation of only 3%.
| Procedure | Measured WCET | Computed by aiT | Overestimation |
|---|---|---|---|
| 1 | 291 | 291 | 0.0% |
| 2 | 6 | 6 | 0.0% |
| 3 | 26 | 26 | 0.0% |
| 4 | 263 | 283 | 7.6% |
| 5 | 263 | 283 | 7.6% |
| 6 | 263 | 283 | 7.6% |
| 7 | 2980 | 3138 | 5.3% |
| 8 | 133 | 133 | 0.0% |
| 9 | 109 | 110 | 0.9% |
| 10 | 116 | 117 | 0.9% |
- Download full report at www.absint.com/aiT_ASCET.pdf (PDF, 210kB)
aiT for ARM7 vs. ARMulator
The ARMulator is a processor simulator provided by ARM as part of the ARM development toolkit. It enables the developer to evaluate the behavior of a program for a certain ARM processor without using the actual hardware.
In an experiment performed by Daniel Sandell at the Mälardalen University, the WCET estimates provided by aiT were on average only 5% higher than the simulated results provided by ARMulator.
| Program | ARMulator | aiT | Ratio aiT/ARMulator |
|---|---|---|---|
| Array_complex_updates | 2399 | 2539 | 1.06 |
| Array_real_updates | 1263 | 1307 | 1.03 |
| Biquad_one_section | 150 | 167 | 1.11 |
| Convolution_filter | 660 | 703 | 1.07 |
| Complex_multiply | 172 | 190 | 1.10 |
| Complex_update | 104 | 121 | 1.16 |
| Digital_filter | 978 | 1022 | 1.04 |
| Dot_product | 95 | 108 | 1.14 |
| Fibonacci | 522 | 532 | 1.02 |
| Impulse_response | 1520 | 1598 | 1.05 |
| Matrix_convolution | 7096 | 7401 | 1.04 |
| Matrix_product_1 | 37887 | 39900 | 1.05 |
| Matrix_product_2 | 34787 | 36800 | 1.06 |
| Matrix_product_3 | 317 | 345 | 1.09 |
| Matrix_product_4 | 5185 | 5442 | 1.05 |
| Real_update | 55 | 66 | 1.20 |
“The aiT tool produced in all cases estimates that were higher than the simulated results. We think this is positive, because it gives an indication that the WCET estimates are safe. It should finally be noted that the WCET estimates have been compared against another hardware model and not against the real hardware.”
WCET Tool Challenge
(aiT for C16x, ARM7, and PowerPC 565)
In 2006, aiT participated in the first WCET Tool Challenge, organized by the University of Mälardalen and sponsored by the ARTIST2 Network of Excellence in Embedded Systems Design. The aim of the challenge was to inspect and compare different approaches in analyzing the worst-case execution time. All available tools and prototypes able to determine safe upper bounds for the WCET of tasks have participated.
aiT was the only tool that started for three different real processor architectures. It was the only tool that was able to produce results for all test programs. The results of analyses by aiT were in most cases exceptionally precise. One finding in the challenge was, amongst others, that aiT allows for user-friendly WCET analysis, advises against unrealistic annotations, and provides tight WCET values.
| Tool | Overestimation | Benchmarks | Automation | Processor complexity |
|---|---|---|---|---|
| aiT | 7–8% | 17/17 (100%) | 54% | simple, medium, very complex |
| Bound-T | n/a | 13/17 (76%) | 26% | simple, medium |
| SWEET | n/a | 15/17 (88%) | 88% | medium |
| Chronos | 81–89% | 13/17 (76%) | 24% | simulated processor |
“aiT was able to handle every kind of benchmark and every test program that was tested in the Challenge. aiT is able to support WCET analysis even for complex processors. […] aiT demonstrates its leading position through all its features, which contribute to its position as an industry-strength tool, satisfying the requirements from industry as posed by EADS Airbus and proven by the accomplishment in various projects.”
Study by University of Stuttgart
(aiT for ARM7)
In 2011, an extensive study at the University of Stuttgart compared notable tools for static WCET analysis, including commercial and academic contenders and using a wide variety of benchmarks and test criteria.
| Benchmark | aiT | Bound-T | WCA | OTAWA | METAMOC | |
|---|---|---|---|---|---|---|
| PapaBench with various entry points |
AA | 399 | 383 | 29054 | 405 | ✗ |
| AC | 1694 | 1637 | 126515 | 1797 | ✗ | |
| AL | 58 | 56 | (21) | 56 | ✗ | |
| AS | 1611 | 1570 | 156974 | 1693 | ✗ | |
| AV | 99 | 97 | ✗ | 97 | ✗ | |
| modified AV | 165 | 163 | ✗ | ✗ | (125) | |
| FCF | 1080 | 1597 | ✗ | 2066 | ✗ | |
| FCM | 1659 | 1640 | 9710 | 2112 | ✗ | |
| FSD | 729 | 705 | 11574 | 895 | ✗ | |
| FST | 873 | 801 | ✗ | 871 | ✗ | |
| FT | 3745 | 3691 | 4629 | 4239 | ✗ | |
| CLOOP1 | 221 | 219 | 222 | ✗ | ✗ | |
| CLOOP2 | 88 | 86 | 89 | ✗ | ✗ | |
| INFINITE1 | ∞ | ∞ | (99) | ✗ | ✗ | |
| INFINITE2 | ∞ | ∞ | (99) | ✗ | ✗ | |
| INFINITE3 | ∞ | ∞ | (2174129) | ✗ | ✗ | |
| BRANCH1 | 41 | 39 | 129 | ✗ | ✗ | |
| BRANCH2 | 51 | 49 | 154 | ✗ | ✗ | |
| BRANCH3 | 60 | 875 | 253 | ✗ | ✗ | |
| BRANCH4 | 48 | 46 | 17439 | ✗ | ✗ | |
| BRANCH5 | 114 | 2028 | 35189 | ✗ | ✗ | |
| MLOOP1 | 686 | 684 | 61748 | ✗ | ✗ | |
| MLOOP2 | 695 | 693 | 61773 | ✗ | ✗ | |
| JUMP1 | 5874 | 5840 | ✗ | ✗ | ✗ | |
| JUMP2 | 9591 | 9652 | ✗ | ✗ | ✗ | |
“aiT is very easy and straightforward to use. While it does provide a lot of configuration options, one can get to a first WCET computation result very quickly. Advanced configuration through annotations can be added later.”
“A clear winner in terms of additional functionality is AbsInt, which provides tools like a syntax-aware editor for annotations including a wizard for easily adding annotation lines, and an interactive callgraph viewer. The other tools have static output which may be viewed, but cannot be interacted upon.”
“AbsInt has by a wide margin the best capabilities to resolve dynamic calls automatically.”
“Recursion is a mostly ignored language feature in the area of real-time programming, so the basic support provided by AbsInt rounded off its image as the most feature-complete tool in this contest.”
Study by University of Erlangen–Nuremberg
(aiT for Cortex-M4)
A 2017 study by Christian Eichler, Peter Wägemann, Tobias Distler, and Wolfgang Schröder-Preikschat reported that on average, aiT overestimated the measured execution times by 23%. A competing tool used for reference showed an average overestimation of 96%.
Study by McGill University
(aiT for Cortex-M4)
In their 2018 study, Márton Búr, Kristóf Marussy, Brett H. Meyer, and Dániel Varró tested aiT on four benchmark programs running on an Infineon Relax Lite Kit-V1 board with the XMC4500 F100-K1024 microcontroller.
The researchers reported that on average, aiT overestimated the measured execution times by 28% in low-precision mode and by 9.75% in high-precision mode. A competing tool used for reference showed an average overestimation of 48.58%.
 |
Benchmark | Measured WCET | aiT | OTAWA |
|---|---|---|---|---|
| Close trains | 2652 | 3038 | 4210 | |
| End of siding | 1395 | 1477 | 1860 | |
| Misaligned turnout | 939 | 987 | 1370 | |
| Train locations | 489 | 507 | 695 |
