Westermo test system performance data set

There is a growing body of knowledge in the computer science, software engineering, software testing and software test automation disciplines. However, a challenge for researchers is to evaluate their research findings, ideas and tools due to lack of realistic data. This paper presents the Westermo test system performance data set. More than twenty performance metrics such as CPU and memory usage sampled twice per minute for a month on nineteen test systems driving nightly testing of cyber-physical systems has been anonymized and released. The industrial motivation is to spur work on anomaly detection in seasonal data such that one may increase trust in nightly testing. One could ask: If the test system is in an abnormal state - can we trust the test results? How could one automate the detection of abnormal states? The data set has previously been used by students and in hackathons. By releasing it we hope to simplify experiments on anomaly detection based on rules, thresholds, statistics, machine learning or artificial intelligence, perhaps while incorporating seasonality. We also hope that the data set could lead to findings in sustainable software engineering.

Keywords: software engineering, anomaly detection, open data, nightly testing, data visualization, seasonal data, embedded systems, cyber-physical systems, sustainable software engineering.


Subject: Computer Science – Embedded Systems, and Software Engineering

Specific subject area: Performance metrics from servers in test systems.

Type and format of data: Tabular data in 19 CSV files, each containing 23 or 24 time series sampled about 86 thousand times. Total size is about 360 MB.

Data collection process: Performance data were acquired from servers using node exporter, stored with grafana and then exported to CSV.

Data accessibility: Available at GitHub: https://github.com/westermo/test-system-performance-dataset External link.

Related research article: P E Strandberg. (2021). Automated System-Level Software Testing of Industrial Networked Embedded Systems, PhD Thesis, Mälardalen University. ISBN: 978-91-7485-529-6. [4]