Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Comprehensive Guide to CAN/CSA‑ISO/IEC 14598‑6‑02: Software Product Evaluation – Documentation of Evaluation Modules
Understanding the Requirements for Evaluation Module Documentation in Software Quality Assessment
Introduction and Scope
CAN/CSA‑ISO/IEC 14598‑6‑02 is the Canadian adoption of the international standard ISO/IEC 14598‑6:2001, which addresses the documentation of evaluation modules used in software product quality evaluation. This standard is part of the ISO/IEC 14598 series that provides a comprehensive framework for evaluating software product quality throughout the software life cycle. Part 6 specifically defines the content and structure of the documentation that must accompany an evaluation module, ensuring consistency, repeatability, and comparability across different evaluation activities.
The scope of CAN/CSA‑ISO/IEC 14598‑6‑02 encompasses the specification of an evaluation module’s purpose, inputs, outputs, evaluation methods, measurement scales, and interpretation criteria. It applies to any organization or individual responsible for developing, maintaining, or using evaluation modules, including software developers, acquirers, independent evaluators, and certification bodies. The standard does not prescribe specific metrics but rather provides a template for documenting how metrics are applied and how results are interpreted within a given evaluation context.
Although the ISO/IEC 14598 series has been superseded by the ISO/IEC 25000 series (SQuaRE), many organizations continue to reference Part 6 for legacy systems or as a basis for constructing evaluation modules that align with the newer standards. The Canadian adoption (CAN/CSA) ensures alignment with national requirements and may include limited deviations to accommodate local regulatory or industry practices.
Technical Requirements
Structure of an Evaluation Module Document
The core of CAN/CSA‑ISO/IEC 14598‑6‑02 is the specification of the information that shall be included in the documentation of an evaluation module. Each evaluation module must clearly describe its position within the overall evaluation process and provide all the details necessary for a competent evaluator to apply it correctly. The required elements are grouped into several categories as shown in the table below.
| Element Category | Mandatory / Optional | Description |
|---|---|---|
| Module Identification | Mandatory | Unique identifier, version, author, date, and references to the quality model (e.g., ISO/IEC 9126 or 25010) used. |
| Purpose and Scope | Mandatory | Intended use of the module, applicable software characteristics, and limitations. |
| Input Requirements | Mandatory | Type of artefact required (e.g., source code, user manual, specification), preconditions, and data quality constraints. |
| Evaluation Method | Mandatory | Step‑by‑step procedure, including any tools, algorithms, or manual inspection rules. |
| Measurement Scale | Mandatory | Scale type (nominal, ordinal, interval, ratio), units, and range of possible values. |
| Interpretation Criteria | Mandatory | Decision rules for mapping raw measurements to quality levels (e.g., pass/fail, a 1‑5 rating). |
| Output Definition | Mandatory | Structure and format of evaluation results, including any required reporting fields. |
| Validity and Constraints | Optional | Assumptions under which the module produces reliable results, known limitations, and validation evidence. |
| Example Usage | Optional | Illustrative case demonstrating the application of the module and interpretation of outputs. |
Each element must be documented in a manner that is unambiguous and sufficient for independent replication of the evaluation. The standard emphasizes that the documentation should be a standalone artifact – an evaluator should not need to refer to external sources to understand how to apply the module.
Conformance to Quality Models
The evaluation module must reference a recognized software quality model. In the context of CAN/CSA‑ISO/IEC 14598‑6‑02, the quality model from ISO/IEC 9126 (and later ISO/IEC 25010) is typically used. The module must clearly map its metrics to the quality characteristics and sub‑characteristics defined by the chosen model, ensuring traceability between measured attributes and high‑level quality goals.
Tip: When developing an evaluation module, start by selecting a quality model that is appropriate for the software domain (e.g., safety‑critical, information management, real‑time). Mapping each metric to a specific sub‑characteristic from ISO/IEC 25010 can simplify later integration with SQuaRE compliance frameworks.
Implementation Highlights
Creating an Evaluation Module Document
Organizations implementing CAN/CSA‑ISO/IEC 14598‑6‑02 should follow a systematic process:
- Define the evaluation goal – Determine which quality characteristics are to be assessed and the context of use (development, acquisition, certification).
- Select or develop metrics – Choose existing metrics (e.g., from ISO/IEC 25023) or create new ones that capture the relevant attributes.
- Document according to the standard – Use the required element structure to produce a complete evaluation module document.
- Validate the module – Test the module with sample software to ensure that the procedures are reproducible and the interpretation criteria yield consistent results.
- Maintain and version – Keep a history of changes and ensure that each version of the module is clearly identified as required by the standard.
Success factor: Involving both software engineers and measurement specialists during the design of an evaluation module improves the practical relevance of the metrics and reduces the risk of ambiguous instructions. A well‑documented module can be reused across multiple evaluation projects, reducing overall cost.
Relationship to ISO/IEC 25000 (SQuaRE)
CAN/CSA‑ISO/IEC 14598‑6‑02 is considered a precursor to parts of the SQuaRE series, particularly ISO/IEC 25040 (evaluation process) and ISO/IEC 25041 (evaluation guide for developers, acquirers, and evaluators). Modules documented according to Part 6 are compatible with SQuaRE if the quality model references are updated and the documentation aligns with the newer terminology. Many organizations use Part 6 as a practical starting point while planning a migration to the current SQuaRE framework.
Compliance Notes
Claiming Compliance
An organization can claim compliance with CAN/CSA‑ISO/IEC 14598‑6‑02 if its evaluation module documentation includes all mandatory elements listed in the standard and follows the prescribed structure. Compliance can be self‑declared or verified by a third‑party auditor familiar with software quality evaluation standards. It is important to note that the Canadian adoption (CAN/CSA) may include national deviations or additional requirements; users should consult the CAN/CSA version directly rather than relying solely on the ISO/IEC text.
Common pitfall: Some organizations mistakenly claim compliance based on the content of the evaluation itself rather than the documentation of the module. The standard applies specifically to the module documentation, not to the evaluation results or the evaluation process itself.
Auditing Considerations
During an audit, the following aspects will typically be examined:
- Whether the evaluation module document contains a clear version identifier and change history.
- Whether each mandatory element (per the table above) is present and adequately detailed.
- Whether the measurement scale and interpretation criteria are defined in a way that prevents subjective bias.
- Whether the module references the correct quality model and mapping is traceable.
Non‑compliance risk: Failing to document an evaluation module properly can lead to inconsistent evaluation results, inability to reproduce assessments, and loss of confidence in software product certifications. In regulated industries, such as medical device or aviation software, this could have serious legal and safety implications.
Updating Legacy Modules
For organizations that have existing evaluation modules created before the adoption of this standard, a gap analysis should be performed. Missing elements (especially interpretation criteria and measurement scales) should be added. The standard does not require retrospective compliance, but any module used in a current evaluation should conform to CAN/CSA‑ISO/IEC 14598‑6‑02 to ensure defensibility.
The standard was confirmed as a Canadian National Standard in 2002 and has not been formally withdrawn; however, users should be aware that the international community largely transitioned to the SQuaRE series. Compliance with CAN/CSA‑ISO/IEC 14598‑6‑02 is still valid for legacy products and internal quality assurance programs.
Frequently Asked Questions
Q: How does CAN/CSA‑ISO/IEC 14598‑6‑02 relate to ISO/IEC 25040?
A: CAN/CSA‑ISO/IEC 14598‑6‑02 focuses specifically on the documentation of evaluation modules (i.e., the detailed description of how a particular metric or measurement is carried out). ISO/IEC 25040, part of the SQuaRE series, defines the overall evaluation process for software product quality. The two standards complement each other: 25040 provides the process framework, while 14598‑6 provides the template for documenting the individual measurement “building blocks” used within that process. A module documented per 14598‑6 can be used directly in a SQuaRE‑conformant evaluation if the quality model references are updated.
A: CAN/CSA‑ISO/IEC 14598‑6‑02 focuses specifically on the documentation of evaluation modules (i.e., the detailed description of how a particular metric or measurement is carried out). ISO/IEC 25040, part of the SQuaRE series, defines the overall evaluation process for software product quality. The two standards complement each other: 25040 provides the process framework, while 14598‑6 provides the template for documenting the individual measurement “building blocks” used within that process. A module documented per 14598‑6 can be used directly in a SQuaRE‑conformant evaluation if the quality model references are updated.
Q: Is compliance with CAN/CSA‑ISO/IEC 14598‑6‑02 mandatory for software evaluation?
A: No, compliance is voluntary unless required by a contract, regulation, or certification scheme. However, using the standard ensures that evaluation modules are documented in a consistent, auditable, and reproducible manner. Many organizations adopt it to demonstrate due diligence in software quality assessment, especially in domains where third‑party evaluations are common (e.g., independent verification and validation of safety‑critical systems).
A: No, compliance is voluntary unless required by a contract, regulation, or certification scheme. However, using the standard ensures that evaluation modules are documented in a consistent, auditable, and reproducible manner. Many organizations adopt it to demonstrate due diligence in software quality assessment, especially in domains where third‑party evaluations are common (e.g., independent verification and validation of safety‑critical systems).
Q: What are the key differences between CAN/CSA‑ISO/IEC 14598‑6‑02 and the earlier parts of the 14598 series?
A: Parts 1–5 of the 14598 series address the processes for evaluating software quality from different perspectives (general, planning, developer, acquirer, evaluator). Part 6 is unique because it defines the documentation of evaluation modules, which are reusable components that can be employed in any of those processes. In other words, Parts 1–5 tell you what to do and when, while Part 6 tells you how to document the “measurement instruments” you use. Part 6 is also more prescriptive about the structure of the documentation itself, requiring specific sections and level of detail.
A: Parts 1–5 of the 14598 series address the processes for evaluating software quality from different perspectives (general, planning, developer, acquirer, evaluator). Part 6 is unique because it defines the documentation of evaluation modules, which are reusable components that can be employed in any of those processes. In other words, Parts 1–5 tell you what to do and when, while Part 6 tells you how to document the “measurement instruments” you use. Part 6 is also more prescriptive about the structure of the documentation itself, requiring specific sections and level of detail.
Copyright © 2026 – This article is for informational purposes and does not replace the official text of CAN/CSA‑ISO/IEC 14598‑6‑02 or ISO/IEC 14598‑6:2001. For compliance, refer to the authoritative standard published by the Canadian Standards Association.