Researchers at Oxford Population Health’s Demographic Science Unit and the Leverhulme Centre for Demographic Science have developed RobustiPy, a new open-source tool, to ensure research is robust and reliable. RobustiPy was formally introduced in a paper published in Patterns today.
Scientific modelling can produce a range of possible outcomes due to the variation that exists in all environments. To see how reliable their findings are, researchers often complete robustness tests. A robust finding should deliver the same result when slight changes are introduced, such as to the method, or the adding of new data into a model.
Developed for a commonly used programming language called Python, RobustiPy represents the first in the next generation of robustness tools that enable researchers to perform rigorous tests on their findings using an unlimited number of variables and modelling choices. By combining tools that researchers would otherwise have to independently write or assemble separately, RobustiPy makes it easier to perform large-scale robustness analysis.
For a hypothetical study testing whether a new biomarker (a measurable health indicator such as blood pressure or cholesterol level) predicts future disease risk, a typical analysis might show that the biomarker is significant in one model.
RobustiPy enables the researchers to test if the association between the biomarker and disease risk is consistent across different combinations of patient characteristics and using different models. It could offer a clearer picture of the biomarker’s scientific and practical importance.
‘RobustiPy lets the computer do the tedious part; estimating a large number of models and computing a large number of tests’ says Associate Professor and senior author on the paper, Charles Rahal.
‘This means researchers can spend more time on the difficult part: deciding which choices are scientifically defensible.’
The paper demonstrates the open-source software through five simulations and ten empirical examples spanning economics, sociology, psychology and medicine.