Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
ISO/IEC 25020: Quality Measurement Framework — SQuaRE Measurement Reference Model Guide
ISO/IEC 25020:2019 — Systems and software engineering — SQuaRE — Quality measurement framework
1. Understanding the Quality Measurement Framework
ISO/IEC 25020:2019 provides the foundational measurement framework for the entire SQuaRE (Systems and software Quality Requirements and Evaluation) series. It defines the Quality Measurement Reference Model (QM-RM), which establishes the relationships among quality models, quality measures (QMs), and quality measure elements (QMEs). This framework is essential for engineers who need to quantify software product quality, quality-in-use, data quality, and IT service quality in a consistent, repeatable manner.
The QM-RM provides a clear hierarchy: Quality characteristics are measured by QMs, which are constructed from QMEs via measurement functions. Understanding this hierarchy is critical for designing effective quality measurement programs.
This second edition (2019) replaces the first edition from 2007 with significant enhancements. Key additions include explicit relationships among different types of quality measures, guidance on applying measurement results, enhanced documentation elements for QMs in Annex C, a normalized measurement function for QMs in Annex D, and harmonization with ISO/IEC 25022, ISO/IEC 25023, ISO/IEC 25024, and ISO/IEC/IEEE 15939. The standard applies throughout the quality life cycle, spanning development, testing, operation, and maintenance phases.
2. The Quality Measurement Reference Model (QM-RM)
2.1 QM Architecture
The QM-RM defines four layers of measurement abstraction. At the foundation, QMEs (Quality Measure Elements) quantify individual properties using specified measurement methods. QMs (Quality Measures) are derived by applying measurement functions to combine QMEs. These QMs then quantify quality sub-characteristics and characteristics defined in the quality models (ISO/IEC 25010, 25011, 25012, 25019). Finally, quality evaluation reports interpret the results for decision-making.
2.2 Types of Quality Measures
The standard distinguishes between three types of measures aligned with the quality life cycle: QMs on internal property (static attributes like code complexity, measurable during development); QMs on external property (behavioral attributes like response time, measurable during testing and operation); and QMs for quality-in-use (outcomes of system use, measurable in real or simulated operational environments). This three-layer architecture enables early detection of quality issues and continuous improvement.
| Measure Type | Target | Life Cycle Stage | Example |
|---|---|---|---|
| Internal Property QM | Static attributes (code, architecture) | Development, Review | Cyclomatic complexity, Code coverage |
| External Property QM | Behavioral attributes (runtime) | Testing, Operation | Response time, Throughput, Failure density |
| Quality-in-Use QM | Outcomes and stakeholder effects | Operational use, UAT | Task completion rate, User satisfaction |
3. Engineering Design Insights and Practical Application
From a practitioner’s perspective, ISO/IEC 25020 provides a rigorous yet flexible framework for constructing quality measures. The standard emphasizes that QMs must be validated (measuring what they claim to measure) and reliable (producing consistent results under repeated measurement). Engineers should consider face validity, content validity, construct validity, correlation, order preservation, predictive validity, and discrimination when selecting or constructing QMs.
Be aware of the “gaming the system” risk: measures influence human behavior. If team performance is evaluated based on a specific QM, individuals may optimize for that measure at the expense of genuine quality. Mitigate this through training, mentoring, and strategic governance.
The standard introduces normalized measurement functions in Annex D that transform raw measurement values into a standardized 0-to-1 scale. This is particularly valuable when comparing measurements across different systems or contexts. Three function types are provided: (a) when the maximum requirement is the upper bound (e.g., fault correction ratio), (b) when there is an upper bound but no lower bound (e.g., throughput), and (c) when there is a lower bound but no upper bound (e.g., response time). These normalized functions enable consistent quality evaluation across diverse measurement domains.
When implementing quality measurement programs, follow the two-step approach: (1) Select candidate QMs from ISO/IEC 25022 (quality-in-use), ISO/IEC 25023 (product quality), or ISO/IEC 25024 (data quality); (2) Refine and document QMs using the structured template in Annex C, including ID, name, description, measurement function, QMEs, and validity evidence.
4. Frequently Asked Questions
Q1: What is the relationship between ISO/IEC 25020 and ISO/IEC/IEEE 15939?
A1: ISO/IEC 25020 harmonizes with ISO/IEC/IEEE 15939 (Measurement Process) and maps its QM-RM to the measurement information model. The SQuaRE QM-RM extends the generic measurement information model by adding quality-specific concepts such as quality characteristics, sub-characteristics, and the distinction between internal property, external property, and quality-in-use measures.
A1: ISO/IEC 25020 harmonizes with ISO/IEC/IEEE 15939 (Measurement Process) and maps its QM-RM to the measurement information model. The SQuaRE QM-RM extends the generic measurement information model by adding quality-specific concepts such as quality characteristics, sub-characteristics, and the distinction between internal property, external property, and quality-in-use measures.
Q2: Can I define my own QMs not listed in ISO/IEC 25022/25023/25024?
A2: Yes. The standard explicitly allows constructing new QMs to satisfy specific quality requirements. When doing so, you must document the QM according to the template in Annex C, including how it relates to the quality model and how it is constructed from QMEs. The measurement function and validity evidence must be clearly specified.
A2: Yes. The standard explicitly allows constructing new QMs to satisfy specific quality requirements. When doing so, you must document the QM according to the template in Annex C, including how it relates to the quality model and how it is constructed from QMEs. The measurement function and validity evidence must be clearly specified.
Q3: How do normalized measurement functions work in practice?
A3: Normalized functions transform raw QME values to a 0-1 scale. For example, if response time must be under 100ms (lower bound), Formula D.3 maps actual response time (x) to a 0-1 score where x=R (100ms) receives a configurable index E (e.g., 0.6). Values below R score higher than E, values above R score lower, enabling consistent cross-measure comparisons.
A3: Normalized functions transform raw QME values to a 0-1 scale. For example, if response time must be under 100ms (lower bound), Formula D.3 maps actual response time (x) to a 0-1 score where x=R (100ms) receives a configurable index E (e.g., 0.6). Values below R score higher than E, values above R score lower, enabling consistent cross-measure comparisons.
Q4: What is the difference between repeatability and reproducibility in measurement?
A4: Repeatability measures variation when the same method is applied under identical conditions (same tools, same individuals). Reproducibility measures variation when the method is applied under different conditions (different tools, different individuals). Both must be assessed for reliable quality measurement, using statistics like the Kappa statistic or Cohen’s alpha depending on the measurement scale.
A4: Repeatability measures variation when the same method is applied under identical conditions (same tools, same individuals). Reproducibility measures variation when the method is applied under different conditions (different tools, different individuals). Both must be assessed for reliable quality measurement, using statistics like the Kappa statistic or Cohen’s alpha depending on the measurement scale.
