Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
CAN/CSA-ISO/IEC 15938-13-16: Multimedia Content Description Interface – Image Description Tools
Technical requirements, implementation guidance, and compliance considerations for the MPEG-7-based image annotation standard adopted in Canada
Scope and General Overview
CAN/CSA-ISO/IEC 15938-13-16 is the Canadian adoption of the international standard ISO/IEC 15938-13:2002/Amd 16:2022, which forms part of the MPEG-7 (Multimedia Content Description Interface) family. This particular amendment specifies Image Description Tools that extend the Multimedia Description Schemes (MDS) defined in ISO/IEC 15938-13. The standard provides a formal framework for describing the semantic and structural content of still images, enabling interoperability across different platforms, applications, and content management systems.
The scope of CAN/CSA-ISO/IEC 15938-13-16 covers:
- Definition of image description schemes (DSs) and descriptors (Ds) for visual features and semantic annotations.
- XML Schema bindings for the representation of image metadata.
- Mechanisms for linking image regions to semantic concepts (objects, events, abstract notions).
- Provisions for spatial, temporal, and conceptual relationships among image elements.
The standard targets developers of multimedia authoring tools, digital asset management systems, search engines, and any system that requires standardized image metadata.
Technical Requirements and Core Components
Image Description Schemes and Descriptors
CAN/CSA-ISO/IEC 15938-13-16 defines several new and updated XML schema components within the MPEG-7 MDS framework. The core technical requirements are summarised in the table below.
| Component | Type | Description | Usage Example |
|---|---|---|---|
StillRegion | Description Scheme | Represents a static region within an image and its associated properties. | Describing a face in a photograph. |
SemanticLabel | Descriptor | Assigns a semantic concept (e.g., “car”, “sunset”) to a region or whole image. | Tagging objects in an image. |
MediaLocator | Descriptor | Provides a URI or path to the image file or its fragment. | Linking metadata to the actual image. |
ImageText | Description Scheme | Captures text embedded in or associated with an image (e.g., OCR results). | Extracting captions or labels. |
MatchingHint | Descriptor | Contains features to support similarity matching (e.g., color histogram, SIFT descriptors). | Content-based image retrieval. |
Semantic Annotation Framework
A key requirement of the standard is the ability to create structured semantic annotations. Annotations must conform to the SemanticBase DS and may reference external ontologies or the MPEG-7 classification schemes. The standard mandates that every image description must include at least one MediaLocator and optionally one or more StillRegion elements if regions are annotated.
Representation and Serialisation
All descriptions must be serialised using well-formed XML Schema instances that comply with the namespaces and schema locations defined in ISO/IEC 15938-13:2002/Amd 16. The recommended encoding is UTF-8. Descriptions may be stand-alone or embedded in MPEG-7 metadata containers. The standard also defines compatibility layers with earlier MPEG-7 tools (e.g., the VisualDescriptor set).
Tip: When implementing, validate your XML against the schemas provided in the standard (normative Annex) to ensure consistency with the MPEG-7 reference software.
Implementation Highlights
Adoption in Asset Management Systems
Organisations that manage large image libraries benefit from the standardised metadata model. The semantic labels enable automatic indexing, cross-repository search, and rights management. Implementation typically involves:
- Mapping existing internal metadata fields to the MPEG-7 components (e.g., categories to
SemanticLabel). - Extending XML generation pipelines to produce compliant descriptions.
- Integrating the MPEG-7 reference software (code available from ISO/IEC) for parsing and validation.
Interoperability Considerations
The standard was designed to be extensible. However, to achieve true interoperability, implementers should:
- Use the predefined classification schemes as much as possible.
- Avoid proprietary extensions that diverge from the schema.
- Follow the naming conventions and datatypes specified in the amendment.
Warning: Custom extensions that are not registered with the ISO/IEC may not be recognised by third-party tools and can break conformance claims.
Performance Optimisation
Image description files can become large when many regions are annotated. Consider using the standard’s media profiling mechanism to assign subsets of descriptors to different quality levels. For real-time applications, use the binary representation format (BiM) as defined in ISO/IEC 15938-1 to reduce parsing overhead.
Success Story: A major digital library implemented CAN/CSA-ISO/IEC 15938-13-16 to unify metadata from 10 million images. The upgrade reduced manual cataloguing time by 40% and enabled cross-collection searches.
Compliance and Conformance Notes
Conformance Testing
The standard distinguishes between conformant descriptions (which follow the schema and semantics) and conformant systems (which can produce or consume such descriptions). Conformance testing should be performed using the MPEG-7 Test Model (XM). Specific checks include:
- Schema validation against the normative W3C XML Schemas.
- Verification that mandatory fields (e.g.,
idattributes) are correctly assigned. - Semantic consistency: a label like “sunset” must not be applied to a region that is described as “indoor”.
Legal and Licensing Aspects
CAN/CSA-ISO/IEC 15938-13-16 is a non‑free standard; users must obtain a licensed copy from CSA Group. The reference software (XM) is available under the ISO/IEE copyright, but many implementations use open-source alternatives that claim compatibility. Use caution with non‑certified tools: they may not correctly handle the amendment’s new elements.
Danger: Claiming conformance without proper validation can expose your organisation to legal risks if interoperability failure occurs. Always perform regression tests with the official conformance suite.
Transition from Previous Versions
Systems built on earlier MPEG-7 Image Description (e.g., ISO/IEC 15938-5) must be migrated. The amendment deprecates some older StillRegion attributes. A migration script is provided in the informative Annex B. The recommendation is to create parallel representations during a transition period until the new descriptions are verified.
Frequently Asked Questions
Q: What is the relationship between CAN/CSA-ISO/IEC 15938-13-16 and the base MPEG-7 standard (ISO/IEC 15938-1)?
A: Part 1 defines the system architecture and binary representation. This amendment specifically extends Part 13 (Multimedia Description Schemes) with tools for still images. The textual format remains XML; binary compression (BiM) is optional but recommended for performance.
A: Part 1 defines the system architecture and binary representation. This amendment specifically extends Part 13 (Multimedia Description Schemes) with tools for still images. The textual format remains XML; binary compression (BiM) is optional but recommended for performance.
Q: Is the standard applicable only to natural images, or can it also describe computer‑generated imagery (CGI)?
A: Both natural and synthetic images are within scope. The semantic and visual descriptors are generic. For synthetic scenes, additional fields (e.g.,
A: Both natural and synthetic images are within scope. The semantic and visual descriptors are generic. For synthetic scenes, additional fields (e.g.,
RenderingType) may be used from the MPEG-7 visual tools in Part 7. Q: How does the standard handle images with multiple regions that overlap?
A: Overlapping regions are supported by defining a spatial mask or using a segmentation mask descriptor. The
A: Overlapping regions are supported by defining a spatial mask or using a segmentation mask descriptor. The
SpatialMask descriptor allows arbitrary shapes; otherwise, regions are usually rectangular or polygonal. The schema includes a Relation element to describe topological relationships (e.g., “overlaps”, “contains”). Q: Where can I find the official conformance test data?
A: The ISO/IEC ITTF website provides the conformance bitstreams and reference software as part of the IS packages. For Canadian adopters, CSA Group can supply the standard package along with a guidance document. Open-source projects also maintain MPEG-7 validation tools, but for formal compliance, use the official test suites.
A: The ISO/IEC ITTF website provides the conformance bitstreams and reference software as part of the IS packages. For Canadian adopters, CSA Group can supply the standard package along with a guidance document. Open-source projects also maintain MPEG-7 validation tools, but for formal compliance, use the official test suites.
Last updated: January 2026. This article reflects the technical content of CAN/CSA-ISO/IEC 15938-13-16 as of 2026.