EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 1

Martin Leutbecher, Roberto Buizza, Renate Hagedorn,

Tim Palmer, Glenn Shutts and Martin Steinheimer

Ensemble forecasting at ECMWF

European Centre for Medium-Range Weather Forecasts

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 2

Outline

1. TIGGE: ensemble inter-comparison, multi-model and calibration2. Model uncertainty representation3. Horizontal resolution increase TL399/255 TL639/319 4. Ensemble data assimilation and prediction

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 3

multi-model

p p

2 4 6 8 10 12 14Lead Time / days

0.0

0.2

0.4

0.6

CR

PSS

TIGGE

CMC

ECMWF

MetOffice

NCEP

T2m, DJF 2008/09NH (20°N - 90°N)BC vs. ERA-interim

Comparing 4 TIGGE ensembles & the Multi-Model

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 4

2

( )( ) ens

ens

q a bxP v qc ds

⎡ ⎤− +⎢ ⎥≤ = Φ⎢ ⎥+⎣ ⎦

with: Φ = CDF of standard Gaussian distribution

• All calibration methods need a training dataset, containing a number of forecast-observation pairs from the past

• Non-homogeneous Gaussian Regression (NGR) provides a Gaussian PDF based on the ensemble mean and variance of the raw forecast distribution

• Calibration process:

Determine optimal calibration coefficients by minimizing CRPS for training dataset

Apply calibration coefficients to determine calibrated PDF from ensemble mean and variance of actual forecast to be calibrated

Create calibrated NGR-ensemble with 51 synthetic members

Calibration using reforecasts

Combine NGR-ensemble with ‘30-day bias corrected’ forecast ensemble

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 5

p p

2 4 6 8 10 12 14Lead Time / days

0.0

0.2

0.4

0.6

CR

PSS

TIGGE

CMC

ECMWF

MetOffice

NCEP

EC-CAL

2m Temperature, DJF 2008/09NH (20°N - 90°N)BC & refc-cali vs. ERA-interim

Comparing 4 TIGGE models, MM, EC-CAL

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 6

TIGGE vs. ECMWF vs. EC-CALp p

2 4 6 8 10 12 14Lead Time / days

0.0

0.2

0.4

0.6

CR

PSS

TIGGE

TIGGE without ECMWF

ECMWF

EC-CAL

2m Temperature, DJF 2008/09Northern Hemisphere (20°N - 90°N)Verification: ERA-interim

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 7

Outline

1. TIGGE: ensemble intercomparison, multi-model and calibration2. Model uncertainty representation3. Horizontal resolution increase TL399/255 TL639/319 4. Ensemble data assimilation and prediction

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 8

Model uncertainty representations

• Starting point: Buizza et al (1999): “stochastic physics”

• Major revision + renaming: Stochastically Perturbed Parameterisation Tendencies (SPPT)

• Stochastic kinetic energy backscatter

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 9

Revision of the SPPT scheme

Original scheme

ΔXp= ( 1+rX) ΔXc

Revised scheme

ΔXp= ( 1+μr) ΔXc

Independent random numbers rX for X=T, q, u, v

Same random number r

for X=T, q, u, v

Perturbations in entire column No perturbations in lowest 300 m and above 50 hPa (0≤ μ ≤1)

Random numbers rX constant in 10o

by 10o lat/lon boxes, and for 6 model time steps (3h in TL399)

Random pattern r varies smoothly in space and time, with de-correlation scales 500 km and 6 h

Uniform distribution between −0.5 and +0.5

Gaussian distribution with stdev 0.5 (limited to ±3stdev)

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 10

Multiplicative Noise Pattern r

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 11

Experimentation and Implementation

• 40 cases with TL399/255 EPS (cycle 35r1) 20 cases in Nov/Dec 2007; 20 in Jul/Aug 2008

• ImpactPositive impact on probabilistic skill for upper air fields in the tropics and neutral to slightly positive impact in the extra-tropicsPrecipitation distribution of perturbed forecasts more similar to that of control (less heavy precipitation events)Prediction of moderate precipitation events improved (SYNOP verification)

• Implemented in cycle 35r3 (8 Sep 2009)

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 12

Revision of SPPT: T850 (Tropics)

• 20 cases in Nov/Dec 2007 and 20 in Jul/Aug 2008 (cycle 35r1)

• Positive impact on probabilistic skill for upper air fields in the tropics (see below) and neutral to slightly positive impact in the extra-tropics (not shown)

EM RMSE

spread

Perfect deterministic forecast

Climatological distribution

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 13

Revision of SPPT: precipitation (NH)

• 20 cases in Nov/Dec 2007 and 20 in Jul/Aug 2008 (cycle 35r1)

• Precipitation distribution of perturbed forecasts is closer to the distribution of the control: less intense precipitation events

• Prediction of moderate precipitation events (<10mm/d) improved (SYNOP verification)

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 14

35r3 E-suite

• Upper air verification:57 cases: April/May & July/AugustVerification with own analyses

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 15

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 16

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 17

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 18

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 19

• SPPT

• Stochastic kinetic energy backscatter: a fraction of the dissipated energy is backscattered upscale and acts as streamfunction forcing for the resolved-scale flow

Shutts & Palmer 2004Shutts (2005) ( cellular automaton)Berner et al (2009) ( spectral AR(1) )

Model uncertainty representations

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 20

• Several improvements Optional wavenumber dependent decorrelation timeBoundary layer tapering (consistent with SPPT)Small change to the forcing spectrum

• Vertical structure random vertical phase shift

Required for use in EnDA context

SPBS progress

SPBS SFFCRM coarse grainingIFS coarse graining

Phase-shift distribution

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 21

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 22

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 23

Summary on scores for SPBS experiments

• TL159 (TL255): SPBS improves skill in extra-tropics and tropics

• TL399: skill improvements in the tropics, extra-tropics overall neutral (some variables slightly worse for long lead times)

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 24

Outline

1. TIGGE: ensemble intercomparison, multi-model and calibration2. Model uncertainty representation3. Horizontal resolution increase TL399/255 TL639/319 4. Ensemble data assimilation and prediction

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 25

Increasing the horizontal resolution

First set of experiments to assess the impact of implementing the higher-resolution system: TL1279 analysis and TL639/319 EPS:

Truncation wavenumbers

EPS AN

current 399 255 799

next 639 319 1279

Vertical discretization remains unchanged:

• analysis L91

• EPS L62 (levels agree with analysis in troposphere)

≤10d >10d

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 26

EPS & analysis resolution increase

35r2 ensembles have been run in the following configurations:

• EPS: 51m399v255a799

• HEPS: 51m639v319a1279

EPS started from the ope an, HEPS from the T1279 analysis f6xh. These experiments have been run for 1 month (J09).

The spread of the two systems is very similar. In terms of rmse(EM), the higher resolution EPS has lower values, especially over SH. 51m399v255a799 (ope)

51m639v319a1279

EM RMSEspread

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 27

EPS & analysis resolution increase – T850

In terms of CRPSS for T850 probabilistic forecasts, HEPS has higher values from day 1.

Differences amount to ~6h gain in predictability at forecast day 7 over NH.

A similar positive signal can be detected over SH and tropics, and over Europe up to day 7.

51m399v255a799 (ope)51m639v319a1279

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 28

Outline

1. TIGGE: ensemble intercomparison, multi-model and calibration2. Model uncertainty representation3. Horizontal resolution increase TL399/255 TL639/319 4. Ensemble data assimilation and prediction

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 29

Ensemble Data Assimilation

• Why? Initial uncertainty is dependent on flow and on observing network. Model this dependency explicitly.

• Applications: EPS, Jb in 4D-Var, QC

• How? Independent 4D-Vars (reduced resolution, one outer loop, less iterations). Perturbations of

observations noise ~N(0,σo2)

model tendencies: revised SPPT, KE backscatterSST

• When? As soon as possible after implementation of higher resolution (TL639)

• EPS initial conditions:high-res. analysis + EnDA-perts + initial SVs

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 30

Tropics

The comparison of EVO-SVINI (ope) and EDA-SVINI ensembles have indicated that the EDA-SVINI EPS has a larger spread, especially over the tropics. The tropics is the area where the benefit of using the EDA is more evident, and also the EM has a slightly lower error.

Ens stdev & EM RMSE– T850

EVO-SVINIEDA-SVINI

EVO-SVINIEDA-SVINI

N.-Hem.

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 31

T850 - NHT850 - NH

CRPSS & IGN : T850 N.-Hem.

Differences can be detected in terms of CRPSS, but are larger in terms of Ignorance Skill Score, a measure more sensitive to the tails of the forecast probability distribution function.

EVO-SVINIEDA-SVINI

EVO-SVINIEDA-SVINI

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 32

T850 - TRT850 - TR

CRPSS & IGN: T850 – TR

Differences can be detected in terms of CRPSS, but are larger in terms of Ignorance Skill Score, a measure more sensitive to the tails of the forecast probability distribution function.

EVO-SVINIEDA-SVINI

EVO-SVINIEDA-SVINI

EMS Toulouse, September 2009 – Leutbecher et al: Ensemble forecasting at ECMWF 33

Summary

• TIGGE comparisonStrength and limitations of the multi-modelValue of reforecasts for calibration (1st and 2nd moment)

• Recent & upcoming implementationsRevision of Stochastically Perturbed Parameterisation Tendencies(September 2009)Resolution upgrade (late 2009/early 2010)Evolved SVs Perturbations from an ensemble of 4D-Vars (2010)

• Further research:Use of coarse-graining approach to constrain stochastic parameterisations