System testing in scientific numerical software frameworks using the example of DUNE
Identifiers (Article)
Identifiers (Files)
Abstract
We present dune-testtools, a collection of tools for system testing in scientific soft-ware using the example of the Distributed Unified Numerics Environment (Dune). Testing is
acknowledged as indispensible support for scientific software development and assurance of
software quality to produce trustworthy simulation results. Most of the time, testing in soft-
ware frameworks developed at research facilities is restricted to either unit testing or simple
benchmark programs. However, in a modern numerical software framework, the number of pos-
sible feature combinations constituting a program is vast. Only system testing, meaning testing
within a possible end user environment also emulating variability, can assess software quality
and reproducibility of numerical results. We provide an easy-to-use interface taking workload off
developers and administrators in open-source scientific numerical software framework projects.
In our approach, the large number of possible combinations is reduced using the scientific expert
knowledge of developers to identify the practically relevant combinations. Our approach to
system testing is designed to be integrated in the workflow of a research software developing
scientist.
Statistics
References
IEEE standard glossary of software engineering terminology. IEEE Std 610.12-1990, pages 1–84, Dec 1990.
W. Bangerth, R. Hartmann, and G. Kanschat. deal.II – a General Purpose Object Oriented Finite Element Library. ACM Trans. Math. Softw., 33(4):24/1–24/27, 2007.
W. Bangerth, T. Heister, L. Heltai, G. Kanschat, M. Kronbichler, M. Maier, and B. Turcksin. The deal.II Library, Version 8.3. preprint, 2015.
P. Bastian, M. Blatt, A. Dedner, C. Engwer, R. Klöfkorn, M. Ohlberger, and O. Sander. A Generic Grid Interface for Parallel and Adaptive Scientific Computing. Part I: Abstract Framework. Computing, 82(2–3):103–119, 2008.
P. Bastian, F. Heimann, and S. Marnach. Generic implementation of finite element methods in the distributed and unified numerics environment (dune). Kybernetika, 46(2):294–315, 2010.
I. Burnstein. Practical software testing: a process-oriented approach. Springer Science & Business Media, 2006.
B. Flemisch, M. Darcis, K. Erbertseder, B. Faigle, A. Lauser, K. Mosthaf, S. Müthing, P. Nuske,
A. Tatomir, M. Wolff, et al. DuMux: Dune for multi-{phase, component, scale, physics,...} flow and transport in porous media. Advances in Water Resources, 34(9):1102–1112, 2011.
David Goldberg. What every computer scientist should know about floating-point arithmetic. ACM Computing Surveys (CSUR), 23(1):5–48, 1991.
D. Hook and D. Kelly. Testing for trustworthiness in scientific software. In Software Engineering for Computational Science and Engineering, 2009. SECSE ’09. ICSE Workshop on, pages 59–64, May 2009.
D. Kelly and R. Sanders. Assessing the quality of scientific software. In First International Workshop on Software Engineering for Computational Science and Engineering, 2008.
A. Logg, K.-A. Mardal, and G. Wells. Automated Solution of Differential Equations by the Finite Element Method. Springer, 2012.
G.J. Myers, C. Sandler, and T. Badgett. The art of software testing. John Wiley & Sons, 2011.
W. Oberkampf, T. Trucano, and C. Hirsch. Verification, validation and predictive capability in computational engineering and physics. In Hopkins University, pages 345–384, 2002.
K. Pohl, G. Böckle, and F. van der Linden. Software Product Line Engineering - Foundations, Principles, and Techniques. Springer, 2005.
H. Remmel. Supporting the Quality Assurance of a Scientific Framework. PhD thesis, 2014.
H. Remmel, B. Paech, C. Engwer, and P. Bastian. Supporting the Testing of Scientific Frameworks with Software Product Line Engineering: A Proposed Approach. In Proceedings of the 4th International Workshop on Software Engineering for Computational Science and Engineering, SECSE ’11, pages 10–18, New York, NY, USA, 2011. ACM.
Supplementary Content
-
dune-testtools 2.4 release tarball
DescriptionA tarball containing the dune-testtools module 2.4. The module can be also found as online git repository under https://gitlab.dune-project.org/quality/dune-testtools.git.
