Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
ISO 25099:2026 – Software Quality Evaluation — A Comprehensive Guide
Understanding evaluation methods, metrics selection, and quality assessment per the SQuaRE framework
1. Overview of ISO 25099
ISO 25099 is part of the ISO/IEC 25000 SQuaRE (Software Quality Requirements and Evaluation) series, specifically focusing on software quality evaluation methodologies. This standard provides a structured framework for evaluating software product quality throughout the development lifecycle, from requirements specification through final acceptance. It defines evaluation processes, metrics selection criteria, and reporting formats that enable consistent and repeatable quality assessments across different software projects and organizations. The standard is applicable to all types of software systems, from embedded firmware to large-scale distributed enterprise applications, and it supports evaluation at any stage of the software lifecycle, including interim evaluations during development, acceptance evaluations before deployment, and field evaluations of operational systems. By following the ISO 25099 framework, organizations can establish a common language and methodology for quality evaluation that is understood by all stakeholders, from developers and testers to project managers and customers.
When implementing ISO 25099-compliant evaluations, always begin by identifying the quality model elements (from ISO 25010) that are most relevant to your specific application domain — this targeted approach avoids wasted effort on irrelevant metrics and ensures that evaluation resources are focused on what truly matters for your stakeholders.
2. Key Evaluation Process and Metrics
The evaluation process defined in ISO 25099 follows a systematic sequence: establishing evaluation requirements, specifying the evaluation, designing the evaluation, executing the evaluation, and concluding the evaluation. Each phase has defined inputs, outputs, and quality gates. The standard emphasizes the importance of measurement traceability — every metric must be linked back to a specific quality characteristic defined in the reference quality model. This traceability ensures that evaluation results can be interpreted in terms of stakeholder needs and that coverage gaps in the evaluation can be identified and addressed proactively.
The evaluation requirements phase identifies the purpose of the evaluation, the type of quality to be evaluated, and the stakeholders involved. During the specification phase, evaluators select appropriate metrics from the ISO 25020 measurement reference framework, define measurement methods, and establish rating levels that map raw measurement values to ordinal quality ratings. The design phase produces a detailed evaluation plan including resource allocation, schedule, and tool selection. Execution involves applying the measurement methods to the software product under evaluation, collecting data, and verifying that the measurements are valid and reliable. Finally, the conclusion phase compares results against predefined criteria and produces a comprehensive quality evaluation report.
| Phase | Key Activities | Output Artifacts |
|---|---|---|
| Establish Requirements | Identify stakeholder needs, select quality model | Evaluation requirements specification |
| Specify Evaluation | Define metrics, measurement methods, rating levels | Evaluation plan |
| Design Evaluation | Create measurement procedures, allocate resources | Evaluation design document |
| Execute Evaluation | Apply metrics, collect data, verify results | Measurement results |
| Conclude Evaluation | Assess against criteria, produce quality report | Evaluation report |
A common pitfall is skipping the “specify evaluation” phase and jumping directly to data collection. Without clearly defined rating levels and measurement methods, results from different evaluators will be inconsistent and non-comparable. Always invest adequate time in the specification phase before collecting any measurement data.
3. Engineering Design Insights
From an engineering perspective, ISO 25099 evaluation is most effective when automated measurement tools are integrated into the CI/CD pipeline. Static analysis tools, test coverage analyzers, and performance benchmarks can continuously feed data into the evaluation framework. This shifts quality evaluation from a point-in-time activity to a continuous process, providing early warning of quality degradation as soon as code is committed. Modern DevOps platforms can be configured to trigger evaluation workflows automatically on each build, generating quality dashboards that give teams real-time visibility into the status of each quality characteristic.
The standard also supports modular evaluation — different quality characteristics can be evaluated independently by different teams at different times, as long as the overall evaluation framework remains consistent. This is particularly valuable for large-scale systems developed by distributed teams, where a single monolithic evaluation event would be impractical. For example, the security team might evaluate security characteristics independently while the UX team evaluates usability, with results being integrated later into a consolidated quality profile. This modularity also enables incremental evaluation, where the scope of evaluation expands as the system matures through its development lifecycle.
Another important practical consideration is the selection of appropriate rating levels. ISO 25099 recommends using ordinal scales with clearly defined criteria for each rating level. A typical four-level scale might include: excellent (exceeds requirements), good (meets requirements), marginal (minor deviations), and poor (major deviations). Each level should be operationally defined with concrete examples to ensure consistent application across different evaluators and evaluation cycles. Organizations should also establish clear decision rules for how evaluation results feed into release decisions, risk management processes, and improvement planning activities.
Organizations that embed ISO 25099 evaluation practices into their DevOps pipelines typically reduce post-release defect density by 30-50% compared to traditional test-last approaches, because quality issues are detected and addressed continuously rather than discovered during a final evaluation gate.
4. Frequently Asked Questions
Q: How does ISO 25099 relate to ISO 25010?
A: ISO 25010 defines the quality model (what to measure), while ISO 25099 defines the evaluation process (how to measure). They are complementary standards within the SQuaRE series — the quality model provides the taxonomy of characteristics, and the evaluation process provides the methodology for assessing them.
A: ISO 25010 defines the quality model (what to measure), while ISO 25099 defines the evaluation process (how to measure). They are complementary standards within the SQuaRE series — the quality model provides the taxonomy of characteristics, and the evaluation process provides the methodology for assessing them.
Q: Can ISO 25099 be used for agile projects?
A: Yes. The evaluation phases can be compressed into sprints, with iterative evaluations aligned to each increment delivery. The key is maintaining traceability back to the quality model and ensuring that the evaluation scope is appropriate for the increment size.
A: Yes. The evaluation phases can be compressed into sprints, with iterative evaluations aligned to each increment delivery. The key is maintaining traceability back to the quality model and ensuring that the evaluation scope is appropriate for the increment size.
Q: What tools support ISO 25099 evaluation?
A: Many ALM platforms (Jama, Polarion, Codebeamer) offer traceability features aligned with ISO 25099. For automated metrics, SonarQube, JMeter, and Gatling cover static analysis and performance dimensions. Custom evaluation frameworks can also be built using Python or R for specialized metrics.
A: Many ALM platforms (Jama, Polarion, Codebeamer) offer traceability features aligned with ISO 25099. For automated metrics, SonarQube, JMeter, and Gatling cover static analysis and performance dimensions. Custom evaluation frameworks can also be built using Python or R for specialized metrics.
Q: Is third-party certification available for ISO 25099?
A: Unlike management system standards (e.g., ISO 9001), the SQuaRE series is typically self-declared or customer-required rather than third-party certified. However, independent evaluators can be engaged to conduct unbiased assessments for critical projects.
A: Unlike management system standards (e.g., ISO 9001), the SQuaRE series is typically self-declared or customer-required rather than third-party certified. However, independent evaluators can be engaged to conduct unbiased assessments for critical projects.