# Literature and provenance check: official CPC/IRI vs IRI objective (plume-based) ENSO probabilities

Compiled 2026-10-03 by a subagent (Claude Opus 5.5) from the public web only.
Labels: **FULL** = I read the page or PDF text myself (downloaded and grepped; nearby text checked). **PARTIAL** = I pulled out the quoted sentence or passage but did not read the whole document. **SNIPPET** = search-engine summary only, not checked.
Local copies are in this directory: `*.html`/`*.txt` (page text), `lheureux2019.pdf`, `ehsan2024.pdf`, `tippett2020_grl.pdf`, `wb_*` (Wayback snapshots), `iri_arch_*` (IRI archive pages).

---

## 1. How the IRI objective (model-based) probabilities are computed, and when the method changed

### 1a. Current method (2026)
- IRI current page, Sept 2026 issue (FULL; `iri_current_2026-10.txt`), https://iri.columbia.edu/our-expertise/climate/forecasts/enso/current/ :
  - "forecasts from all 22 participating models (13 dynamical and 9 statistical) are combined into an equally weighted multi-model average of Niño3.4 SST anomalies. A Gaussian error distribution is then applied to the ensemble-mean forecast, with its width determined by the expected forecast skill for the season and lead time. Higher forecast skill results in a narrower distribution and greater confidence, while lower skill produces a broader range of possible outcomes."
  - Categories: "El Niño (≥ +0.5°C), ENSO-neutral (-0.5°C to +0.5°C), and La Niña (≤ -0.5°C)."
  - The spread comes from hindcasts: "The overall width of the probability distribution is derived from the historical skill of the hindcasts of the models, from 1982 to present, for the specific forecast start time and lead time. This method ... represents one of two general approaches, the other approach being a direct counting of ensemble members". So the method is **Gaussian around the equal-weight mean of all plume models (dynamical + statistical)**. It does not count members, and it does not use the models' own ensemble spread.
  - Caveat: the figure caption on the same page still says "A purely objective ENSO probability forecast, **based on regression**, using as input the model predictions from the plume ... Each of the forecasts is weighted equally ... It does not use any human interpretation or judgment." The word "regression" probably refers to the skill-based (correlation/regression) shrinkage of the spread. It conflicts with the plain "Gaussian" wording elsewhere on the page. I found no document that defines the exact variance formula, e.g. whether it is σ_clim·sqrt(1−r²), as in L'Heureux et al. 2019 (see 2a).
- IRI Feb and Mar 2026 quick-look pages (FULL; `iri_2026-february.txt`, `iri_2026-march.txt`): "using the average value of the NINO3.4 SST anomaly predictions from all models in the plume, equally weighted. A standard Gaussian error is imposed over that averaged forecast, with its width determined by an estimate of overall expected model skill for the season of the year and the lead time." Also: "this approach considers only the mean of the predictions, and not t[he total range across models]". The same pages add "Only models that produce a new ENSO prediction every month are considered in this statement" (this refers to the discussion statement; it is unclear whether it also applies to the probabilities).

### 1b. KEY FINDING: before mid-December 2011 the IRI probabilities were not objective
- The "IRI Probabilistic ENSO Forecast" from 2002 to Nov 2011 was a **judgment-based IRI forecaster assessment** using **quartile-based categories** (25/50/25 climatology). It was not the Gaussian plume rendering.
  - Old definition page, `note_ensoprob.html`, Wayback snapshots 2003-05 and 2009-06 (FULL; `wb_note_*.txt`): "The categories of La Niña, neutral, and El Niño are defined such that for any 3-month period, the historical frequency of being in the La Niña and the El Niño categories is 25% each over the base period of 1950-2002, and the frequency of being in the neutral category is 50%." The thresholds vary by season, e.g. DJF −0.65/+0.65, MAM −0.40/+0.40, SON −0.75/+0.70.
  - Technical ENSO Update, 18 Dec 2008, Wayback (FULL; `wb_2008_technical.html.txt`). The Gaussian method appears there only as an *alternative diagnostic*: "An alternative way to assess the probabilities ... is to use the mean of the forecasts of all models, and to construct a standard error function centered on that mean. The standard error would be Gaussian in shape ...". The issued numbers are then given as "All things considered, during the Dec-Jan-Feb season there is a 48% probability for La Niña ... The above assessment was made in part on the basis of an examination of the current forecasts of ENSO prediction models as well as the observed conditions."
  - The same wording ("The above assessment was made in part on the basis of an examination of the current predictions of ENSO prediction models as well as the observed conditions") appears in the Technical Updates for June 2011 and Nov 2011 (FULL; `iri_arch_201106_technical.txt`, `iri_arch_201111_technical.txt`).
  - The 2004 and 2008 tables show telltale round numbers, e.g. long leads of exactly 25/50/25 and "50% neutral" repeated (Wayback figure3.html, FULL; `wb_fig3_*.txt`).
- The change is announced in the IRI ENSO Update archived as 201201, dated "19 January 2011" in the header, which is evidently a typo for 2012 (FULL; `iri_arch_201201_update.txt`), https://iri.columbia.edu/our-expertise/climate/forecasts/enso/archive/201201/update.html :
  > "NOTICE: This "IRI ENSO Update/Forecast" is being supplanted by the NOAA/Climate Prediction Center ENSO Diagnostic Discussion, issued early in the month. The IRI Technical ENSO Update is still issued by the IRI near the middle of the month, but the probabilistic ENSO forecast created at mid-month is now an objective rendering directly from the IRI/CPC ENSO prediction plume, using equal weighting for all plume models and using a Gaussian probability density function centered on the multi-model mean prediction. Probabilistic ENSO forecasts having a possibly beneficial subjective component ... are made jointly by IRI and CPC at the beginning of the month ... These early month probabilities are to be considered the official CPC/IRI probabilistic ENSO forecast product. The thresholds for El Niño and La Niña are plus and minus 0.45 C Nino3.4 SST anomalies, respectively, as used by NOAA/Climate Prediction Center, rather than the seasonally varying quartile-based values used by IRI through 2011."
- The April 2012 version of the notice (FULL; `iri_arch_201204_update.txt`) adds: "The width of that density function depends on the estimated skill of the multi-model prediction, based on verification of hindcasts." It also gives the ±0.45 °C climatological probabilities, e.g. DJF 37/28/35, MAM 26/54/20.
- First objective issue: the Dec 2011 Technical Update (FULL; `iri_arch_201112_technical.txt`) already says "This alternative method uses the mean of the predictions of all models, equally weighted ... A plot of the probabilities generated from this most recent IRI/CPC ENSO prediction plume using the multi-model mean and the Gaussian standard error method". The Nov 2011 update still uses "The above assessment was made ...". **Best estimate: objective probabilities begin with the mid-Dec 2011 issue.** The numbers support this. Nov 2011 NDJ La Niña was 74%. Dec 2011 DJF La Niña was 98%, and later leads carry 0.1%-level values, which fits a Gaussian rendering.
- Thresholds, step by step:
  - through Nov 2011: seasonal quartile thresholds (about ±0.4 to ±0.75 °C)
  - the Dec 2011 Technical Update text says ±0.5, but the Jan 2012 and Apr 2012 notices say ±0.45 (FULL)
  - by May 2013: "requires that the SST anomaly in the NINO3.4 region ... exceed 0.5 C" (FULL; `iri_arch_201305_technical.txt`)
  - one search summary of a 2026 paper says the threshold was "updated in May 2013" and "from January 2012 to April 2013 ... ±0.45°C" (SNIPPET)
- IRI's own observational index and data: IRI uses OISSTv2 with the 1991–2020 climatology. It also warns that models' differing base periods can cause differences "as high as a quarter of a degree C in the worst cases" (FULL, current page).
- Plume membership changes over time. Ehsan et al. 2024, Table 1 (FULL; `ehsan2024.txt`) shows 30 dynamical and 13 statistical models entering and leaving during 2002–2023. For example, NCEP CFS (v1) ran 2002-01 to 2012-08 and CFSv2 from 2011-06; CMC CanSIPS from 2012-01; LDEO/ECMWF/JMA throughout. Because the mean is equal-weighted, the dynamical/statistical mix (roughly 13:9 now) drifts over time.

### 1c. RONI (CPC switch, Feb 2026) and the IRI objective probabilities
- CPC: "In February 2026 NOAA's Climate Prediction Center adopted the Relative Oceanic Niño Index (RONI)" (drought.gov, 11 Mar 2026; FULL, `drought_roni.txt`). The CPC official probabilities page now reads: "Probabilities are verified using the Relative Oceanic Niño Index (RONI), with El Niño and La Niña defined using -/+ 0.5 degree Celsius thresholds" (FULL; https://cpc.ncep.noaa.gov/products/analysis_monitoring/enso/roni/probabilities/).
- IRI did **not** switch. The Sept 2026 page heads the plume "ENSO Prediction Plume: Model-by-Model Outlook (Traditional Nino3.4 index)" and states "The primary metric used to monitor ... ENSO is the traditional Niño3.4 index (TONI) ... According to the CCSR/IRI definition, El Niño conditions occur when monthly TONI exceeds +0.5 °C" (FULL). The Feb and Mar 2026 IRI pages mention RONI values only as an observational aside (FULL).
- **Implication:** from about Feb/Mar 2026 the official outlook targets RONI and the IRI objective outlook targets traditional Niño3.4. In 2026 RONI is cooler than traditional Niño3.4 (drought.gov: RONI "will dampen the strength of El Niños and amplify La Niñas"). For the same seasons the two forecasts therefore predict different variables.
- Branding: the product is now titled "CCSR/IRI Technical ENSO Update and Model-Based Probabilistic ENSO Forecast" (FULL). It was earlier "IRI/CPC Plume-Based ENSO Forecast" (2012–2013 nav bars, FULL).

### 1d. Barnston et al. 2012 BAMS and Barnston & Tippett papers
- Barnston et al. 2012 (BAMS, "Skill of real-time seasonal ENSO model predictions during 2002–11: Is our capability increasing?"). I could not download it (the AMS site blocks automated access). Ehsan et al. 2024 summarise it as assessing "the real-time prediction capabilities of 20 models individually ... from February 2002 to March 2011. They found that the dynamical models exhibited a statistically significant advantage over the statistical models when predicting ENSO events during the months of March to May" (PARTIAL, via Ehsan 2024). As far as I can tell it verifies the deterministic plume, not the IRI probabilities (SNIPPET-level).
- Barnston & Tippett 2014, Earth Perspectives 1:20, "Climate information, outlooks, and understanding–where does the IRI stand?" (open access, CC-BY, but the Springer download was blocked from here). A search summary says it discusses reliability of IRI's probabilistic forecasts, which IRI has issued "in March of 2002" onward (SNIPPET). It probably verifies IRI's subjective 2002–2011 ENSO probabilities; **worth reading if it can be reached**.
- Barnston et al. 2015, per a search summary: hindcast skill rather than ensemble spread should set the forecast uncertainty (SNIPPET). This matches the IRI rationale quoted above (FULL).

## 2. Has anyone compared official (human) and model-based ENSO probabilities?

### 2a. L'Heureux et al. 2019, WAF 34:165–175, doi:10.1175/WAF-D-18-0126.1 (FULL; `lheureux2019.pdf`)
- Three strategies, verified June 2015 – Sept 2018, 41 forecasts:
  - **CPCoff**: the official subjective three-category probabilities. "At NOAA/CPC, the forecasters have been providing three-category ENSO probabilities since January 2012."
  - **CPCcalib**: the forecasters supply only the mean Niño-3.4 (collected internally "since July 2015"). A Gaussian is centred on that mean. Its SD is the 1982–2010 climatological SD scaled by NMME hindcast correlation skill for each start month and lead. With r near 0 it collapses to climatology.
  - **NMMEcalib**: purely objective, from NMME (~100 equally weighted members), using the same skill-based Gaussian.
- Findings (from the text; the numbers are only in Figs 5–6, which I could not read as numbers):
  - "probabilities for CPCoff are more muted and tend to be more conservative than those from the other methods."
  - NMMEcalib had the worst mean scores. Its "negative skill is in large part due to NMMEcalib assigning high probabilities for the rapid onset of La Niña during summer 2016 even at short lead times". Its skill scores were negative again in late spring/summer 2017, and also "for long-lead forecasts made for targets toward the last quarter of 2017 when La Niña returned and yet probabilities for La Niña remained low."
  - "NMMEcalib frequently outscores CPCoff ... because the NMMEcalib probabilities are more confident overall. This feature ... is therefore more apparent in the median score than the mean score ... Generally, CPCoff has lower median scores".
  - "CPCcalib ... allows forecasters to avoid the large forecast busts of NMMEcalib, yet have better overall skill than CPCoff." It also had a higher mean/SD ("Sharpe") ratio.
  - "when forecasters are relied upon to develop probabilities from model inputs (CPCoff), they tend to err toward more conservative probabilities ... it reduces the frequency of large errors, but it comes at the cost of higher skill scores in the median." The authors cite "loss aversion".
  - The supplement reportedly compares methods "from January 2012 to the present" (not read).
  - The paper does **not** use the IRI plume-based probabilities.
- climate.gov "ENSO forecast mash-ups: What's the best way to combine human expertise with models?" (Di Liberto, July 2019; FULL; `cg_mashups.txt`) summarises the paper:
  - "The final ENSO outlook averages all the forecasters' probabilities."
  - "on average, human forecasters were too conservative"
  - "the model output was the worst in terms of mean skill score ... if you instead looked at the median ... the model output outscores the original human synthesis."
  - Method footnote: "She took the average of the surface temperature anomaly forecasts from all forecasters and applied a normal, bell-shaped or Gaussian statistical distribution."

### 2b. Ehsan et al. 2024, npj Clim Atmos Sci 7:301 (FULL; `ehsan2024.pdf`)
- It scores **only the deterministic plume**: DYN-mean and STAT-mean correlation, RMSE, variance, SESS, and onset accuracy for Feb 2002 – 2023.
- "probabilistic" appears only in the introduction ("disseminated forecasts ... in both deterministic and probabilistic formats"). **It does not score the IRI objective probabilities or the official probabilities.**

### 2c. Others
- climate.gov "How Good Have ENSO Forecasts Been Lately?" (A. Barnston, 25 Sep 2014; FULL; `cg_howgood.txt`): deterministic DYN vs STAT plume means for 2012–2014 only, with no probabilities.
- Tippett et al. 2019, Clim Dyn 53:7497, "Assessing probabilistic predictions of ENSO phase and intensity from the NMME": NMME-only probabilistic reliability (SNIPPET; not read).
- I found **no published study that verifies the IRI plume-based objective three-category probabilities against the official CPC/IRI probabilities**. The search was moderate and I could not reach the Barnston & Tippett 2014 full text. On present evidence the planned comparison looks new, especially over 2012–2026 and across the Nov 2020 change.

## 3. Did CPC document the ~Nov 2020 change in how official probabilities are made?
- I found **no explicit CPC statement that the three-category official probabilities became derived from forecasters' Niño-3.4 means plus a calibrated Gaussian.**
- What is documented:
  - Nov 2020 ENSO Diagnostic Discussion (FULL; `cpc_disc_nov2020.txt`): "A probabilistic strength forecast is available here." This is the first such link I found; the Oct 2020 discussion has none.
  - The linked Nov 2020 strength page (FULL; `cpc_str_nov2020.txt`): "This table shows the forecast probability of Niño-3.4 index exceeding a certain threshold ... The values are based on the analysis published in: L'Heureux ... 2019: Strength Outlooks for the El Niño-Southern Oscillation."
  - climate.gov Nov 2020 update, "just us chickens" (12 Nov 2020; FULL; `cg_nov2020.txt`): "The Climate Prediction Center is now providing a probabilistic outlook for the strength of El Niño and La Niña events. Tom described this new technique in a blog post a little while back" (the link points to the mash-ups post).
- Consistency check (inferred): in the Nov 2020 strength table the ≤ −0.5 °C column (NDJ ~100, DJF 99, JFM 95) matches the official three-category La Niña probabilities ("~95% chance during January-March"). This suggests the three-category numbers came from the same CPCcalib Gaussian from Nov 2020 on, but no text says so explicitly.
- The current CPC pages describe the process only as "The official outlook is determined by a ~10 member team, using available insights from multiple models ... and the latest observations" (FULL). The July 2019 mash-ups post still says "The final ENSO outlook averages all the forecasters' probabilities" (FULL). The April 2023 climate.gov update reportedly says the strength method "has shown promise so far, although we've only been using it for a couple of years" (SNIPPET).

## 4. Documented model biases (over-forecasting El Niño / missing La Niña)
- **2012 false El Niño** (climate.gov, Barnston 2014, FULL): "the dynamical models predicted an El Niño, and only stopped doing so after SSTs in the initial conditions starting cooling. The 4-month lead forecasts ... showed a moderate-strength El Niño. However, neutral-ENSO ... prevailed." Also: "the statistical models predicted weaker warming during both the 2012 and the 2014 event". Also: "The mean absolute error is generally larger for the dynamical models, partly because they averaged too warm during the period". A search summary says the CPC/IRI consensus gave 75% and 65% El Niño for Dec 2012 in Aug and Sep 2012 (SNIPPET).
- **2014 failed El Niño**:
  - Tippett, L'Heureux, Becker & Kumar 2020, GRL 47, e2020GL087008 (FULL; `tippett2020_grl.pdf`): "As late as June 2014, coupled model forecasts were confidently predicting strong El Niño conditions to come that winter ... Those warnings of a strong El Niño turned out to be a false alarm". Also: "Models tend to predict warming when initialized after observed warming conditions and cooling when initialized after observed cooling conditions [excessive momentum]" and "Coupled models failed to capture the correct direction of ENSO evolution in half of last eight springs (2011–2018) including 2014". This is NMME, not the IRI plume, but the dynamical members overlap heavily.
  - Ehsan 2024 (FULL): "during the back-to-back El Niño of 2014–2015 and 2015–2016 ... the DYN forecast was too strong at longer leads, while the STAT forecast correctly suggested a weaker warming signal ... not every individual dynamical model overpredicted; however, the majority of the models did."
- **La Niña under-forecasting**:
  - Ehsan 2024 (FULL): "in the case of cold episodes, both DYN and STAT forecasts show larger differences ... Notable examples include the La Niña events of 2017, 2010, and 2007" and "During cold ENSO episodes, the SESS exhibits low values, even turning negative ... cases of cold episodes in 2007, 2010, and 2017, when the DYN SESS often had more negative values than STAT."
  - L'Heureux 2019 (FULL): for late-2017 targets "La Niña returned and yet probabilities for La Niña remained low". The opposite error also occurred: in summer 2016 the models gave high La Niña probabilities too early.
  - Ehsan 2024 also says both DYN and STAT "underestimate the amplitudes of the warm events in 2002–2003, 2009–2010, 2015–2016, and 2023–2024 especially at longer leads". So the bias depends on the state; it is not a uniform warm bias.

## Implications for the study
1. **The "IRI objective" series is not objective before mid-Dec 2011.** I checked the study's `data/objective_probs_raw.csv` against as-issued Wayback tables for Jan 2004, Jan 2008, Nov 2010 and Jun 2011, and they match exactly (observed). Those numbers are IRI forecasters' judgment-based probabilities with quartile (25/50/25) category definitions. Recommendations:
   - Treat 2003-06 to 2011-11 as "IRI subjective (quartile thresholds)" and analyse it separately, or drop it.
   - Use mid-Dec 2011 onward as the objective (Gaussian, equal-weight plume mean) series.
   - The official CPC/IRI consensus also began in Jan 2012, so a clean head-to-head starts in **Dec 2011 / Jan 2012**.
2. **Threshold and verification-index breaks**:
   - IRI used ±0.45 °C from about Jan 2012 to Apr 2013, then ±0.5.
   - CPC's official outlook targets the ERSSTv5 ONI with rolling climatology. IRI uses OISSTv2 with the 1991–2020 climatology for observations.
   - From Feb 2026 CPC targets RONI while IRI still targets traditional Niño3.4.
   - Verify each product against its own target index, or flag the differences. 2026 scores will differ systematically because RONI runs cooler than ONI.
3. **Expected human-vs-model contrasts** (from L'Heureux 2019): official probabilities more conservative, with fewer busts but lower median skill; the objective probabilities sharper, with occasional large busts (2016 early La Niña, missed 2017 La Niña; 2012 and 2014 false El Niño signals from dynamical models). After about Nov 2020 the official probabilities apparently come from a Gaussian built on the forecasters' mean (strongly suggested, not explicitly documented). From then on, any official-vs-IRI difference mainly reflects (a) the forecasters' mean versus the plume mean and (b) the skill-based spread (NMME 1982–2010 correlations versus IRI plume hindcast skill), not human handling of uncertainty.
4. The IRI spread uses only the multi-model mean and hindcast skill ("considers only the mean ... not the total range"). Model disagreement therefore does not widen the IRI distribution, and the equal-weight membership (DYN:STAT mix) changes over time.
5. **Novelty:** I found no prior paper that verifies the IRI objective probabilities against the official probabilities. The closest are L'Heureux 2019 (official vs NMME-calibrated, 2015–2018) and possibly Barnston & Tippett 2014 (IRI's own 2002-era probabilities; not read).