Geant4 Maintainability Assessed with Respect to Software Engineering References

TL;DR

This paper presents a methodology to quantitatively evaluate the maintainability of Geant4 software using established software engineering metrics and reference standards.

Contribution

It introduces a systematic approach to assess Geant4's maintainability based on literature-documented metrics, filling a gap in software quality evaluation.

Findings

Quantitative maintainability scores for Geant4

Identification of key maintainability metrics

Comparison with software engineering standards

Abstract

We report a methodology developed to quantitatively assess the maintainability of Geant4 with respect to software engineering references. The level of maintainability is determined by combining a set of metrics values whose references are documented in literature.

Tables

Table 1. Table 1: Some Metrics of the Size Group

Group	Size Metric	Source
File	Comment Ratio	Lorenz and Kidd
	Declarations in File	Lorenz and Kidd
	File Size	Lorentz and Kidd
	Functions in File	Lorenz and Kidd
	Lines in File	Lorenz and Kidd
	Lines of Source Code	Lorenz and Kidd
	Lines of Comments	Lorenz and Kidd
	Number of Statements	Lorenz and Kidd
	Variables in File	Lorenz and Kidd
Function	Lines in Function	Lorenz and Kidd
	Lines of Source Code	Lorenz and Kidd
	Variables in Function	Lorenz and Kidd

Table 2. Table 4: A Sample of Quality References

Acronym	Reference	Source
Comment Ratio	0.08	McCabe
SLOC (Source Lines Of Code)	60 at file level	McCabe
HPV (Halstead Programme Volume)	1500 at function level	McCabe
	[100,8000] at file level	Verysoft Technology [Tec]
	$>$ 800 too many things at file level	Verysoft Technology [Tec]
	[20, 1000] at function level	Verysoft Technology [Tec]
	$>$ 1000 too many things at function level	Verysoft Technology [Tec]
MI (Maintainability Index)	65 poor maintainability	Coleman, Lowther, Oman [CLO95]
	[65, 84] fair maintainability	Coleman, Lowther, Oman [CLO95]
	$\ge$85 excellent maintainability	Coleman, Lowther, Oman [CLO95]
MCMCC (McCabe’s Maximum Cyclomatic Complexity	[1, 10] low CC	CppDepend [Cpp]
	[11, 15] medium CC	CppDepend [Cpp]
	[16,30] high CC	CppDepend [Cpp]
	$>$31 very high CC	CppDepend [Cpp]
	[1, 10] low CC	McCabe
	[11, 20] medium CC	McCabe
	[21. 50] high CC	McCabe
	$>$51	McCabe

Full text

Geant4 Maintainability Assessed with Respect to Software Engineering References111Presented at 2016 IEEE NSS/MIC - Strasbourg, France, 29 October - 5 November, 2016

Elisabetta Ronchieri INFN CNAF at Bologna. Email: elisabetta.ronchieri@cnaf.infn.it   

Maria Grazia Pia INFN at Genoa. Email: maria.grazia.pia@ge.infn.it   

Tullio Basaglia CERN. Email: Tullio.Basaglia@cern.ch   

Marco Canaparo INFN CNAF at Bologna. Email: marco.canaparo@cnaf.infn.it

We report a methodology developed to quantitatively assess the maintainability of Geant4 with respect to software engineering references. The level of maintainability is determined by combining a set of metrics values whose references are documented in literature.

1 Introduction

This report documents what has been done so far to assess Geant4 maintainability - one of software characteristics defined in software quality standards, such as ISO/IEC 25010:2011 (former ISO/IEC 9126) [ISO11].

Maintainable software allows development teams to fix bugs, add new features, improve usability and increase performance. Organizations that deal with software in different domains, such as telecommunications, aerospace and simulations, monitor such software characteristic to maintain skills and knowledge in order to understand and make changes to their software.

Software characteristics are measured by metrics values. We identified and assessed software metrics tools (both free and under commercial licenses) to collect a large number of measurements [RPG14], [RPG15]. As a result of this assessment, we selected Imagix4D [Ima]: this tool measures several product metrics at different levels, such as file, class, directory, namespace, function and variable, and its vendor positively collaborates with research communities. Metrics used in this study provide code information about size, coupling, inheritance, control-flow structuredness, cohesion, staticness. Existing literature gives references of such metrics for different programming languages. We identified C++ metric thresholds to determine the goodness of code.

2 Method

The methods used to perform this maintainability assessment is characterized by the following steps:

collecting the source code of all Geant4 [AAA*+*03, AAea06] versions from 0 to the current one (10.2); 2. 2.

loading the Geant4 source code into Imagix 4D version 8.0.4 to measure a large number of metrics in order to obtain code information about size, coupling, inheritance, control-flow structuredness, cohesion, staticness; 3. 3.

saving all the collected data at different levels of granularity, such as file, function, class, directory, variable and namespace; 4. 4.

application of statistical methods for the analysis of metric values; 5. 5.

identification of quality (goodness ranges of) references with respect to size, coupling, inheritance, control-flow structuredness, cohesion, staticness, derived from relevant peer-reviewed papers, conference proceedings and technical reports [RC16].

Some of the metrics we collected are listed in Tables 3, 3 and 3.

Size metrics [LK94] quantify code size. They are estimators of software cost and effort.

Complexity metrics, such as McCabe [McC76] and Halstead [Hal77] metrics, measure the simplicity of the system design. McCabe’s complexity, also called cyclomatic complexity, quantifies the control flow within a program by counting the independent paths on a control flow graph. The path indicates a certain degree of well structuredness of an application.

Object-oriented metrics [CK94] measure complexity, maintenance and clarity; they estimate to which extent the system adheres to the object orientation.

3 A sample of quality references and results

Figure 2 shows the trend of McCabe Maximum Cyclomatic Complexity at class level for Geant4 hadronic physics diffraction package, while Figure 2 shows the trend of Halstead’ programme volume for the electromagnetic physics xrays package.

Table 4 shows a sample of quality references.

4 Conclusions

The use of metrics can contribute to monitor the internal quality of software. Further investigation is in progress to identify appropriate ranges of metric values for Geant4 packages by using statistical methods. More extensive results will be discussed in a forthcoming full paper.

Acknowledgment

The authors thank Francesco Giacomini for interesting discussions and INFN CCR for partly funding this work. We also thank the Imagix Corporation that provided an extended free full license of Imagix 4D for performing this work and CERN library for providing papers and books.