Research Project Image

CITCOM: Causal Inference for Testing of Computational Models

Computational models can be hard to test. They have large input spaces (often hundreds of parameters). A single execution can take hours days. They can be inherently non-deterministic. Thus, running large numbers of carefully controlled tests is not feasible. Furthermore, given that they are often used to explore behaviours of unfamiliar phenomena, there may be a lack of certainty around what constitutes a "correct" output.

Causal Inference offers promising solutions to these problems. It provides a framework to enable some lightweight domain-knowledge into the analysis process. This can enable the tester to answer questions about the (causal) relationships between inputs (or parameters) and outputs. These can be answered without the need for large numbers of controlled tests.

Research Contributions

The CITCOM project runs from 2020-2025. The core conceptual contributions are listed and linked to their relevant papers here. A more complete bibliography is provided below.

Software: The CITCOM Causal Testing Framework

A tool implementing the CITCOM Causal Testing approach has been made available on GitHub under an MIT License.

Target Systems

Causal testing is quite a flexible approach and can be applied to a range of classes of systems. We are currently in a phase of the project where we are exploring its application to a range of types of systems. These include:

Bibliography

Foster, Michael, Andrew Clark, Christopher Wild, Farhad Allian, Robert Turner, Richard Somers, Nicholas Latimer, Neil Walkinshaw, and Robert M Hierons. 2025. “The Causal Testing Framework.” Journal of Open Source Software 10 (107).
Somers, Richard, Neil Walkinshaw, Robert Mark Hierons, Jackie Elliott, Ahmed Iqbal, and Emma Walkinshaw. 2025. “Configuration Testing of an Artificial Pancreas System Using a Digital Twin: An Evaluative Case Study.” Software Testing, Verification and Reliability 35 (2): e70000.
Walkinshaw, Neil, Michael Foster, Jose Miguel Rojas, and Robert M Hierons. 2024. “Bounding Random Test Set Size with Computational Learning Theory.” Proceedings of the ACM on Software Engineering (FSE).
Foster, M, C Wild, R Hierons, and N Walkinshaw. 2024. “Causal Test Adequacy.” In 2024 IEEE Conference on Software Testing, Verification and Validation (ICST). Institute of Electrical; Electronics Engineers (IEEE).
Anness, Abigail R, Michael Foster, Mohammed W Osman, David Webb, Thompson Robinson, Asma Khalil, Neil Walkinshaw, and Hatem A Mousa. 2024. “Do Maternal Haemodynamics Have a Causal Influence on Treatment for Gestational Diabetes?” Journal of Obstetrics and Gynaecology 44 (1): 2307883.
Clark, Andrew G, Michael Foster, Benedikt Prifling, Neil Walkinshaw, Robert M Hierons, Volker Schmidt, and Robert D Turner. 2023. “Testing Causality in Scientific Modelling Software.” ACM Transactions on Software Engineering and Methodology.
Clark, Andrew G, Michael Foster, Neil Walkinshaw, and Robert M Hierons. 2023. “Metamorphic Testing with Causal Graphs.” In 2023 IEEE Conference on Software Testing, Verification and Validation (ICST), 153–64. IEEE.
Somers, Richard J, James A Douthwaite, David J Wagg, Neil Walkinshaw, and Robert M Hierons. 2023. “Digital-Twin-Based Testing for Cyber–Physical Systems: A Systematic Literature Review.” Information and Software Technology 156: 107145.
Walkinshaw, Neil, and Robert M Hierons. 2023. “Modelling Second-Order Uncertainty in State Machines.” IEEE Transactions on Software Engineering 49 (5): 3261–76.
Anness, AR, A Clark, K Melhuish, FMT Leone, MW Osman, D Webb, T Robinson, N Walkinshaw, A Khalil, and HA Mousa. 2022. “Maternal Hemodynamics and Neonatal Birth Weight in Pregnancies Complicated by Gestational Diabetes: New Insights from Novel Causal Inference Analysis Modeling.” Ultrasound in Obstetrics & Gynecology 60 (2): 215–22.
Somers, Richard J, Andrew G Clark, Neil Walkinshaw, and Robert M Hierons. 2022. “Reliable Counterparts: Efficiently Testing Causal Relationships in Digital Twins.” In Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, 468–72.

Acknowledgments

This work has been funded by the EPSRC CITCOM grant.