Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
IEC 61164 โ Crow/AMSAA Reliability Growth Statistical Test & Estimation Methods
💡 Standard Overview: IEC 61164 specifies statistical test and estimation methods for reliability growth planning, tracking, and assessment using the Crow/AMSAA model — mathematically formalized as the Power Law Process (PLP), a special class of Non-Homogeneous Poisson Process (NHPP). Originally developed by Dr. Larry Crow at the U.S. Army Materiel Systems Analysis Activity (AMSAA) in 1974, this model has become the de facto industry standard for reliability growth analysis in defense, aerospace, railway signaling, and industrial electronics.
1️⃣ Theoretical Foundations of the Crow/AMSAA Model
1.1 The Power Law Process Formulation
The Crow/AMSAA model posits that during reliability growth development testing, system failures follow a Non-Homogeneous Poisson Process with the following intensity function (instantaneous failure rate):
λ(t) = λβ · tβ−1 (t > 0, λ > 0, β > 0)
Here, λ is the scale parameter and β is the shape parameter — the single most important engineering metric in the model. When β < 1, the failure intensity decreases over time, which is the signature of genuine reliability growth. When β = 1, the process reduces to a homogeneous Poisson process (HPP) indicating no growth. When β > 1, reliability is actually degrading — a critical warning flag that demands immediate engineering intervention.
The expected cumulative number of failures by time t is given by:
E[N(t)] = λ · tβ
This relationship yields a straight line on log-log coordinates, with slope β and intercept log(λ). This elegant linearity property makes the Crow/AMSAA model exceptionally well-suited for graphical validation of field data — a practitioner can visually assess whether the power-law assumption holds by simply plotting cumulative failures against cumulative test time on logarithmic scales.
✅ Engineering Insight: The shape parameter β serves as a direct quantitative measure of engineering effectiveness. In a typical product development cycle spanning 6 to 18 months, β should start in the range of 0.7–0.9 during initial prototype testing and progressively decrease toward 0.4–0.6 as the design matures. A rising β trend is an unambiguous signal that corrective actions are ineffective or that new failure modes are being introduced — the engineering team must halt and reassess the root-cause analysis process.
1.2 Physical Interpretation and Model Assumptions
The Crow/AMSAA model carries several important assumptions that engineers must validate before application:
- Immediate corrective action: Each failure is analyzed and fixed immediately upon occurrence, with the fix incorporated into all subsequent units. This is the “test-analyze-and-fix” (TAAF) paradigm.
- No wear-out during growth testing:The model assumes that the system is in its useful life period — wear-out failures would violate the monotonic intensity assumption.
- Homogeneous improvement rate: The multiplicative improvement effect of each corrective action is approximately constant, which is reflected in the power-law form.
⚠️ Common Violation: In practice, many programs batch corrective actions into periodic design releases (e.g., monthly engineering change orders). This creates a step-function improvement pattern rather than the smooth power-law decay assumed by the model. When this occurs, analysts should segment the data into phases and apply the model piecewise, or use the extended AMSAA model that accommodates discontinuous growth.
2️⃣ Statistical Testing and Parameter Estimation Methodology
2.1 Growth Trend Testing
Before estimating parameters, the engineer must first verify that a statistically significant growth trend exists. IEC 61164 recommends two complementary tests:
| Test | Statistic | Critical Value | Remarks |
|---|---|---|---|
| Laplace (Centered) | U = (Σtᵢ/T − m/2) / √(m/12) | ±1.96 (α=0.05) | U < 0 → growth; sensitive to early failures |
| Cramér–von Mises | C² = 1/(12m) + Σ[(tᵢ/T)^β − (2i−1)/(2m)]² | Tabulated IEC 61164 Annex | More robust; recommended for final confirmation |
| Military Handbook 189 | χ² = 2Σln(T/tᵢ) | χ² distribution, 2m d.o.f. | Equivalent to MLE-based test |
The Laplace test is particularly valuable as an initial screening tool due to its simplicity. However, it is notoriously sensitive to early-life failures — a cluster of failures in the first 10% of test time can dominate the statistic and produce a misleadingly strong “growth” signal even when later-stage growth is negligible.
2.2 Maximum Likelihood Estimation
For failure-truncated testing (test stops at the m-th failure), the MLE of β has a closed-form solution that is both elegant and computationally efficient:
β̂ = m / Σi=1m ln(T / ti)
where tᵢ is the i-th failure time and T is the total test duration. The MLE for λ follows directly: λ̂ = m / Tβ̂. These estimators are asymptotically unbiased, consistent, and efficient. Simulation studies have shown that stable estimates are typically achieved when m ≥ 20, though acceptable engineering precision can be obtained with as few as 10 to 15 failures when using bias-corrected estimators.
For time-truncated testing (test stops at a predetermined time T regardless of failure count), the MLE equations require numerical iteration. IEC 61164 provides explicit iterative algorithms using the Newton-Raphson method, along with starting value guidelines to ensure convergence.
🔥 Critical Engineering Trap: A widespread mistake is comparing the Crow/AMSAA instantaneous MTBF directly against steady-state MTBF requirements (e.g., MIL-HDBK-217 predictions or program-specified reliability targets). The instantaneous MTBF reflects the system’s reliability level at the moment the test stopped — it inherently includes the transient improvement dynamics. The steady-state MTBF after production maturity is typically 1.5 to 2.0 times higher. Programs that terminate growth testing as soon as the instantaneous MTBF meets the target are often disappointed when field reliability falls short. A safety margin of at least 30% above the target is recommended before declaring completion.
2.3 Interval Estimation and Confidence Bounds
The instantaneous MTBF at the end of test time T is estimated as:
MTBF(T) = 1 / (λ̂ β̂ Tβ̂−1)
Confidence intervals are constructed using the conditional chi-squared distribution property of the Crow/AMSAA model. The two-sided 90% confidence interval for the instantaneous MTBF is given by:
- Lower bound: MTBFL = 2m / [χ²0.05, 2m · λ̂ β̂ Tβ̂−1]
- Upper bound: MTBFU = 2m / [χ²0.95, 2m · λ̂ β̂ Tβ̂−1]
For the growth potential — the MTBF achievable if all identified failure modes are corrected — IEC 61164 provides an additional set of estimators that project the asymptotic reliability level assuming no new failure modes are introduced.
3️⃣ Growth Test Planning and Engineering Deployment Strategies
3.1 Designing the Growth Test Program
IEC 61164 Annex A provides a systematic framework for planning a reliability growth test based on the planned growth rate α, defined as:
MTBFi+1 / MTBFi = 1 / (1 − α)
The planning procedure follows these steps:
- Define the target MTBF: Derived from system-level reliability requirements, typically expressed as a mean time between critical failures.
- Establish the starting MTBF: Based on similarity analysis with predecessor systems, preliminary testing, or engineering judgment. Conservative estimation is critical here — overestimating the starting point leads to an under-designed growth program.
- Compute the required growth rate: α = 1 − (MTBF₀ / MTBFtarget)1/n, where n is the planned number of improvement cycles.
- Determine total test duration: Using the approximate relationship T ≈ MTBFtarget · m / (1 − β̂).
| Growth Rate α | Interpretation | Typical Context | Risk Level |
|---|---|---|---|
| 0.2 – 0.3 | Moderate growth | Mature technology, minor modifications | Low |
| 0.3 – 0.5 | Strong growth | New design with known technology base | Medium |
| 0.5 – 0.7 | Aggressive growth | Novel system with intensive TAAF cycles | High |
| > 0.7 | Extreme growth | Rarely sustainable; indicates overly conservative start MTBF | Very High |
3.2 Integration with the TAAF Cycle
The most effective application of IEC 61164 occurs within a structured Test-Analyze-And-Fix (TAAF) framework. Each TAAF cycle typically spans 2 to 4 weeks of continuous testing followed by 1 to 2 weeks of failure analysis and design modification. The Crow/AMSAA model is updated at the end of each cycle to provide quantitative feedback on whether the growth trajectory is on track.
✅ Best Practice — Phase-Based Growth Tracking: Rather than estimating a single β over the entire program, experienced practitioners apply a “rolling window” estimation. A window of the most recent 15–25 failures is used to estimate the current β, and this window is advanced as new data accumulates. This approach reveals changes in growth effectiveness in near-real-time. If the rolling β estimate exceeds 0.85, the TAAF cycle intensity should be increased — more root-cause analysis resources and shorter iteration intervals are indicated.
3.3 Linkage with the IEC Reliability Toolkit
IEC 61164 is one component of a comprehensive reliability engineering standards ecosystem. The Crow/AMSAA model outputs feed directly into:
- IEC 61025 (FTA): Identified critical failure modes from growth testing inform fault tree top events and branch probabilities.
- IEC 60812 (FMEA/FMECA): Risk Priority Numbers (RPN) from FMEA can be validated against observed failure frequencies during growth testing — high-RPN items should correlate with dominant contributors to the Crow/AMSAA intensity function.
- IEC 61078 (RBD): The system reliability model, expressed as a reliability block diagram, is calibrated against the instantaneous MTBF estimates at each growth milestone.
- IEC 61014: Provides the overarching programmatic framework for planning and managing reliability growth activities, with IEC 61164 supplying the quantitative engine.
💡 Cross-Standard Integration Insight: In defense and railway applications, a powerful technique is to link Crow/AMSAA β estimates with FMEA risk metrics. For each dominant failure mode identified in the FMEA with RPN > 200, the engineering team assigns a target failure rate reduction. When these high-RPN modes are systematically eliminated, the program-level β should show a measurable decrease. This bridges qualitative risk assessment with quantitative growth measurement — ensuring that the “numbers move” only when real engineering problems are solved.
❓ Frequently Asked Questions (FAQ)
Q1: What is the fundamental difference between the Crow/AMSAA model and the Duane model?
A: The Duane model is an empirical graphical method that estimates growth by plotting cumulative MTBF against cumulative test time on log-log paper. It provides point estimates only — no confidence intervals, no goodness-of-fit tests, and no rigorous statistical foundation. The Crow/AMSAA model is mathematically equivalent to a stochastic process formulation of the Duane postulate but adds the full apparatus of statistical inference: MLE parameter estimation, chi-squared-based confidence bounds, Cramér–von Mises goodness-of-fit testing, and growth potential projections. For any formal reliability demonstration report, Crow/AMSAA is the required standard; Duane should be reserved for quick-look assessments during the early stages of testing.
Q2: How should multi-unit test data be combined for Crow/AMSAA analysis?
A: IEC 61164 recommends pooling failure times from all units into a single ordered sequence, provided all units are operating under the same design iteration. If units are at different modification levels (e.g., Unit A has received Fix 1 while Unit B has not), the data must be handled using the “calibrated time” approach — mapping each unit’s test time onto a common developmental time axis. A simpler alternative is to analyze each unit separately and then combine the β estimates using a weighted average, with weights proportional to the number of failures per unit.
Q3: When can a reliability growth test be legitimately terminated?
A: Terminating growth testing early carries significant risk, but objective criteria include: (1) The estimated instantaneous MTBF exceeds 130% of the target value; (2) The 90% upper confidence bound of β is below 0.7, indicating sustained strong growth; (3) Three consecutive TAAF cycles show no further decrease in β (growth saturation); (4) Total accumulated test time exceeds 10 times the target MTBF. Before final termination, a “no-fix verification run” of at least 1.5 times the target MTBF should be conducted to confirm stability. If no critical failures occur during this verification period, termination is justified.
Q4: Can IEC 61164 be applied to software reliability growth?
A: Yes — the Crow/AMSAA model has been successfully applied to software reliability growth since the 1980s, particularly in defense avionics and mission-critical systems. However, practitioners must recognize key differences: (a) Software failures are design defects, not wear-out phenomena, so β carries a different physical interpretation related to defect discovery rate rather than hardware degradation; (b) Software corrections can introduce regression defects, violating the “perfect repair” assumption. Extended NHPP models (e.g., the Goel-Okumoto or delayed S-shaped models) may be more appropriate when the software development process follows a non-uniform defect introduction pattern. IEC 61164 is best applied to software growth at the system integration level where hardware and software failures are analyzed jointly.
