The behaviour of all systems is governed by the laws of physics. If you know the starting conditions, then you can move forward in time applying those laws and you can predict the state of the system at any time in the future. This is what might be called a ‘clockwork’ approach to prediction.
However, many systems are what is known as ‘chaotic’. They obey the laws of physics but to predict their future behaviour you must know their starting conditions to an infinite degree of accuracy. At some time, no matter how small the error, the state of the system in will be fundamentally unpredictable.
Systems do not have to be complicated to be ‘chaotic’. A classic example is the three-body problem. It describes the motion of three-point mass particles moving under their mutual gravitational interactions. Newton’s laws of gravity should predict the position of all particles at anytime in the future. Problem is they don’t! The system’s future behaviour depends on the precise starting conditions. ANY error of ANY size will result in unforecastable ‘chaotic’ behaviour. That is the problem.
The Earth’s atmosphere is one such system. We can never know the position and movement of every molecule and, even at this scale, there will be tiny errors and chaos will eventually rule.
Numerical Weather Forecasting (NWP) uses physics to predict the future development of the atmosphere from an initial starting point usually referred to as its ‘background field’. This gives us one answer. This is called a ‘deterministic’ forecast. In reality, there is not a definitive ‘deterministic’ forecast. It is simply one of many possible forecasts due to errors in the ‘background field’. If we could work out how sensitive the forecast is to its starting conditions, then we would have some idea how ‘predictable’ the atmosphere was today.
‘Ensemble’ forecasting was first developed in the 1970s. Put simply, you take the background field and insert several different sets of variations to the original field. How the variations are chosen was beyond my pay grade, but each one is then fed into the computer and NWP does its work. This produces a forecast derived from each of the slightly different background fields. Then you look at the distribution of the solutions. Put simply, if the solutions remain close together, we don’t have one forecast, but we have higher confidence as the atmosphere is not very sensitive to starting conditions. If they diverge then confidence in our ability is low. Note that this has nothing to do with our forecasting skill. We are trying assess a fundamental property of the atmosphere — its predictability.
The distribution of solutions is known as a ‘probabilistic’ forecast. In truth, this is a more honest appraisal of our ability to predict any situation.
Ensemble forecast is becoming more and more useful. As computer power increases it is now possible to run more and more different background fields at higher and higher resolutions AND have solutions available on a useful time scale. More solutions gives a better understanding of the sensitivity of the atmosphere to starting conditions.
The problem for forecasters is that there is still not one solution, only probabilities. Now, to most of the public, and even to some grumpy old forecasters(!), a ‘probabilistic’ forecast is simply ‘fence-sitting’. They don’t want ‘might’ or ‘could’ or a 30% probability of … . As a forecaster, you can’t say, ‘Sorry, the the atmosphere cannot be precisely forecasted today’. They want forecasters to get it ‘right’. So each day, forecasters look at all the available data and produce a single answer to a question that does NOT have one until it actually occurs — which is too late. Forecasting is ‘challenging’, but that’s why we get the big bucks!
About the Author
Frank Barrow retired from the Met Office in 2020, after a career of 39 years during which he worked as an observer in the 1980s, a forecaster in the 90s and, since 1996, as a trainer at the Met Office College. He describes the latter role as “fitting him like a glove”, so he stayed... for almost 25 years! During that time, he was involved in the training of the vast majority of current Met Office forecasters.
Renaming the Poor Man's Ensemble - by Rebecca Gilbert and Ken Mylne
The comparison of multiple numerical weather prediction models is a valuable forecasting tool and has been documented in literature for more than 24 years. It is often referred to as a Poor Man’s Ensemble, although this term has been queried over the years for being poor use of language. There have been some attempts to address this, including NOAA’s use of the Storm Scale Ensemble of Opportunity, pictured here, to describe a forecast view of several different high resolution model centres and time lagged runs.
The Poor Man’s Ensemble (PME) is a subjective analysis of several independently designed deterministic numerical weather prediction models. It is a valuable technique for the qualified meteorologist and for the amateur at home making use of whatever they have available. A variety of papers have been published about it, dating back 24 years to “The Skill of Ensemble Prediction Systems” (Atger, 1999).
As meteorologists, we seek to understand how the atmosphere behaves. As forecasters, we communicate how it is behaving in a variety of ways so that our knowledge can be useful. We transform the chaos of the atmosphere into words and these words have the power to save lives as well as the power to mislead. Yet we are using terminology to describe a valuable technique which is considered by many to be inaccurate, classist and sexist.
The term Poor could imply poor quality or value. The PME usually only has a few members, however multiple tests have proven that it is an effective forecasting tool (Bowler et al, 2008). While a PME cannot provide a sense of probability, it can give a of range of possible outcomes, and it can showcase an aspect of uncertainty which cannot be achieved by using one model centre alone. It provides information that a typical ensemble cannot, and it can outperform more expensive NWP systems (Bouttier and Marchal, 2019) It is also less prone to systematic biases and errors that cause under dispersive behaviour in single-model EPSs (Ebert, 2001).
It could be argued that it is not a proper ensemble because of its small number of members, however an ensemble is simply defined as a group of things or people brought together. Unlike larger ensembles, it cannot be used to estimate the probabilities of different forecast outcomes, however the experience of many forecasters is that it can sometimes provide a better sampling of the uncertainty which they observe.
The term Poor is also problematic for being classist. These days we try not to refer to poor countries or poor people. We refer to developing countries or low-income communities. When the meteorological community uses Poor Man’s, weimply that a separate or other group use it, but not us. We have a responsibility to be inclusive irrespective of affluence, influence, or wealth. In 2005, the authors of “Test of a Poor Man’s Ensemble Prediction System for Short-Range Probability Forecasting” (Arribas, 2005) considered whether the name was appropriate, and they chose to continue to use it because the name was so well established. It was a fair decision at the time, however, it has been 18 years and we are increasingly questioning whether it is appropriate. Our culture continues to evolve, and we can choose to stop using language that has the power to exclude individuals from engaging.
Use of the word Man to refer to all people is also outdated. Referring to a Weatherman or a Postman is increasingly questioned and altered in preference of terminology that recognises women. In 2019, the author of “Historical Perspective: Earlier Ensembles and Forecasting Forecast Skill” attempted to address the issue by referring to a Poor Person Ensemble Prediction System (Kalnay, 2019). The zeitgeist of today necessitates that we collectively stop using language that needlessly consciously or unconsciously promotes the male while failing to acknowledge the female. When we refer to a weather forecasting tool, there is no need to refer to a male or female at all.
We could choose to use a new term — one that is inclusive, self-explanatory, and not easily confused with the other ensemble model tools. In fact, there is another term that has been used at least since 2007, when a paper called “The Detection and Attribution of Climate Change Using an Ensemble of Opportunity” (Stone et al, 2007) was published. NOAA’s Storm Prediction Centre has been using Ensemble of Opportunity to refer to forecasts from several independently designed deterministic convection allowing models for several years, and it is well documented in a variety of papers. Opportunistic Ensemble or Ensemble of Opportunity, which could be used interchangeably, are self-explanatory terms. A colleague who has called it a PME for their entire career would be able to make an educated guess at what you were referring to, while a newly trained meteorologist would have an easier time remembering which term referred to which tool. Another advantage is that Opportunistic Ensemble is perhaps easier to understand by non-native English speakers.
PME is well documented, and we will not eliminate its use overnight. Shifting our language will take time and practice. Still, if we each make a personal decision to use Ensemble of Opportunity or Opportunistic Ensemble instead of Poor Man’s Ensemble, we will inspire other people around us to do the same and eventually we will nudge our use of language to be as modern and advanced as the tools we use.
Knowledge of things and knowledge of the words for them grow together. If you don’t know the words, you can hardly know the thing (Hazlitt, 1916).
