Standards & methodology
ASHRAE Guideline 14-2014
ASHRAE Guideline 14
The statistical quality bar for energy savings models — what CV(RMSE), NMBE, and R² mean, the thresholds they must pass, and what to do when they do not.
What the guideline covers
ASHRAE Guideline 14-2014 (Measurement of Energy, Demand, and Water Savings) is published by ASHRAE. It is the dominant US standard for measuring energy savings and is applied internationally through IPMVP.
Its core contribution: three statistical metrics that a regression baseline model should pass before its savings output can be trusted. These thresholds are referenced in IPMVP and shape what “good enough” means in M&V practice.
The full guideline is a purchased document. Key thresholds are reproduced in IPMVP (free) and in practitioner guidance worldwide.
CV(RMSE) — model scatter
Coefficient of Variation of the Root Mean Square Error measures how far the model’s predictions are from actual values, on average.
| Data interval | Threshold |
|---|---|
| Monthly | ≤ 20% |
| Hourly | ≤ 30% |
A CV(RMSE) of 15% means predictions are typically off by 15% of average consumption — acceptable. At 35%, model error drowns out the savings signal.
NMBE — systematic bias
Normalized Mean Bias Error measures whether the model consistently over- or under-predicts.
| Data interval | Threshold |
|---|---|
| Monthly | ≤ ±5% |
| Hourly | ≤ ±10% |
CV(RMSE) and NMBE test different things. A model can pass scatter but fail bias — both must pass independently.
Positive NMBE over-predicts baseline energy and inflates savings. Negative NMBE understates them. Either direction is a problem when payment depends on the verified figure.
R² — explained variation
Coefficient of Determination measures how much consumption variation is explained by the model’s independent variables — typically heating and cooling degree days.
The widely applied threshold is R² ≥ 0.75 (referenced by IPMVP and ISO 50047, though not a bright-line requirement in GL14 itself).
R² = 0.82 means weather explains 82% of month-to-month variation — a strong model. R² = 0.38 means weather alone is insufficient; add production, occupancy, or other variables, or reconsider whether Option C is appropriate.
When a model fails
A failed threshold does not automatically invalidate a project — it signals investigation before results go into a contract or compliance report.
| Metric fails | Common cause | Typical response |
|---|---|---|
| CV(RMSE) too high | Outliers; short baseline; missing variables | Extend baseline; add variables; investigate outliers |
| NMBE out of range | Meter gaps; systematic data error; unusual baseline periods | Check sub-meter coverage; review anomalous months |
| R² too low | Energy not weather-driven; wrong base temperature | Try different base temp; consider IPMVP Option A or B |
The M&V plan should pre-specify what happens if thresholds are not met. Changing methodology after the baseline period closes is an audit risk.
Why it matters in contracts and audits
In guaranteed-savings ESCO contracts, the verified savings figure determines payment. A baseline model that fails GL14 thresholds — or uses an undocumented degree day source — can be challenged.
Counterparties and auditors typically check:
- Does the model meet CV(RMSE) and NMBE thresholds?
- Is the degree day source documented and reproducible?
- Were base temperature and weather station selected before the baseline period ended?
Ed displays CV(RMSE), NMBE, and R² with pass/warn indicators against these widely applied thresholds — based on the principles of ASHRAE Guideline 14, not as a full GL14 reporting package or software certification.