Skip to main content

Precision and convergence speed of the ensemble Kalman filter-based parameter estimation: setting parameter uncertainty for reliable and efficient estimation

Abstract

Determining physical process parameters in atmospheric models is critical to obtaining accurate weather and climate simulations; estimating optimal parameters is essential for reducing model error. Recently, automatic parameter estimation using the ensemble Kalman filter (EnKF) has been tested instead of conventional manual parameter tuning. To maintain uncertainty for the parameters to be estimated and avoid filter divergence in EnKF-based methods, some inflation techniques should be applied to parameter ensemble spread (ES). When ES is kept constant through the estimation using an inflation technique, the precision and convergence speed of the estimation vary depending on the ES assigned to estimated parameters. However, there is debate over how to determine an appropriate constant ES for estimated parameters in terms of precision and convergence speed. This study examined the dependence of precision and convergence speed of an estimated parameter on the ES to establish a reliable and efficient method for EnKF-based parameter estimation. This was carried out by conducting idealized experiments targeting a parameter in a cloud microphysics scheme. In the experiments, there was a threshold value for ES where any smaller values did not result in any further improvements to the estimation precision, which enabled the determination of the optimal ES in terms of precision. On the other hand, the convergence speed accelerates monotonically as ES increases. To generalize the precision and convergence speed, we approximated the time series of parameter estimation with a first-order autoregression (AR(1)) model. We demonstrated that the precision and convergence speed may be quantified by two parameters from the AR(1) model: the autoregressive parameter and the amplitude of random perturbation. As the ES increases, the autoregressive parameter decreases, while the random perturbation amplitude increases. The estimation precision was determined based on the balance between the two values. The AR(1) approximation provides quantitative guidelines to determine the optimal ES for the precision and convergence speed of the EnKF-based parameter estimation.

Introduction

The accuracy of numerical weather forecasting and climate projections is affected by various factors. In terms of short-term weather forecasting, accurate initial conditions are produced by utilizing a sufficient volume and spatial coverage of observations, alongside effective data assimilation (DA) methods. Additionally, applying smaller grid spacing and accurate discretization schemes to fluid dynamics equations reduce the discretization error in atmospheric simulations. Although these efforts are critical in terms of improving forecasting accuracy, they are not sufficient. Atmospheric models parameterize many physical processes, such as cloud microphysics, turbulence, surface flux, radiation, and moist convection. Such parameterizations contain numerous physical and numerical parameters having their own uncertainties, potentially causing severe forecasting errors. For example, Gilmore et al. (2004) showed that uncertainties for intercept parameters and particle densities in a cloud microphysics scheme largely affected the development of convective clouds. Sanderson et al. (2008) showed that the entrainment coefficient in a convection scheme impacted climate sensitivity for general circulation model (GCM) simulations.

Ideally, physical model uncertainties should be solved by improving the model itself based on an understanding of actual physical processes, and determining model parameters based on experiments and measurements. However, to reduce prediction error, we often “tune” parameters in the existing physical model instead of improving model equations. The conventional parameter tuning method involves parameter sensitivity experiments and comparing each performance for different parameter sets (Mauritsen et al. 2012; Hourdin et al. 2017). This is a meaningful procedure as researchers can directly check the impact of varying parameter values on the prediction and evaluate its effects on the climate. It is important as, in general, the time scale of climate is quite longer than the time scale of physical processes directly governed by the parameters. However, such manual parameter tuning is limited in terms of its evaluation efficiency. As optimal parameter values in weather prediction vary depending on the region, season, and phenomenon of interest, it is not practical to manually tune parameters for various conditions. As such, an advanced optimization method that automatically processes numerous observational data and quantitatively incorporates this information into a physical model is more advantageous than tuning fixed values.

To date, many DA methods have been proposed, such as the four-dimensional variational method (4DVar; Sasaki 1969; Thompson 1969), ensemble Kalman filter (EnKF; Evensen 1994), and four-dimensional ensemble-variational method (4DEnVar; Buehner et al. 2013). These methods have mainly been used to produce the most likely initial conditions, incorporated into numerical weather forecasting systems (e.g., Houtekamer et al. 2005; Lien et al. 2017; JMA 2019; NOAA 2019; ECMWF 2020). These methods have also been applied to parameter optimization for physical models (Ruiz et al. 2013a). Zhu and Navon (1999) attempted to optimize model parameters in a global spectral model using 4DVar with the adjoint model; they reported that optimized parameters reduced the forecast error over a few days. Aksoy et al. (2006b) tested the parameter estimation in a two-dimensional (2D) sea breeze model using the ensemble square root filter (EnSRF; Whitaker and Hamill 2002). They were successful in estimating the true parameter values of the nature run. Aksoy et al. (2006a) also estimated a vertical eddy mixing coefficient in a non-hydrostatic mesoscale model. Tong and Xue (2008) successfully estimated the parameters of a cloud microphysics scheme by applying EnSRF to convective-scale radar data assimilation. Ruiz et al. (2013b) tested the estimation of convective scheme parameters incorporated into a GCM using an ensemble transform Kalman filter (ETKF; Bishop et al. 2001). Kotsuki et al. (2018) optimized the parameter of a convective scheme by applying the ETKF to global precipitation data assimilation. Recently, Kotsuki et al. (2020) attempted to optimize a spatially varying parameter field by using a local ensemble transform Kalman filter (LETKF; Hunt et al. 2007). Ruckstuhl and Janjić (2020) tested the estimation of the roughness length in a convection-permitting model using the LETKF; they showed estimating the roughness length led to the improvement of cloud and precipitation forecasts.

In the EnKF-based parameter estimation, perturbed parameter values are assigned to each ensemble member to artificially generate parameter uncertainty. Unlike state variables such as wind, temperature, and moisture, the parameter values generally remain constant during the forecast and are only updated during the analysis step. The parameter ensemble spread (ES), defined as a standard deviation of the parameter ensemble, is a key metric of parameter uncertainty. This ES also remains constant during the forecast; as such, the updated analysis ES for the parameter is always smaller than its previous one. This leads to filter divergence, in which the parameter ES gradually decreases, and the parameter value finally loses its sensitivity to observations. To avoid filter divergence, previous studies have set a threshold value for the standard deviation of the parameter ensemble, where the standard deviation was adjusted back to the threshold when it fell below the threshold (e.g., Aksoy et al. 2006a, b; Tong and Xue 2008). Kotsuki et al. (2018) applied an inflation technique to parameter ensemble at every analysis step to ensure that the parameter ES was constant.

Thus, the issue centers on determining the optimal threshold or fixed value of the parameter ES. Tong and Xue (2005) stated that an excessively large ES generates oscillations around the true value, reducing estimation precision. By contrast, an ES that is too small leads to slow convergence, while the use of further smaller ES can cause filter divergence. Kotsuki et al. (2018) showed that the estimation reaches a specific value using different ES values for the estimated parameter; however, it takes longer to converge to that value with a smaller ES. These studies suggest that an ES that is not too large nor too small is appropriate for precise and rapidly converging parameter estimation. A possible compromise is to use larger ES for the first few cycles prioritizing fast convergence and then to use smaller ES expecting high precision like Aksoy et al. (2006a, b) and Tong and Xue (2008). In this case, however, the issue about how we set the lower limit of ES value to achieve high precision remains.

Ruiz et al. (2013b) proposed a method to estimate the optimal ES for each analysis step. In their method, the inflation factor for the ES of the parameter was determined based on the analysis error covariance matrix in the ensemble space, including information on state variables and parameters. Ruiz et al. (2013b) stated that regardless of the initial ES, the method can optimize the parameter ES for each analysis step depending on model sensitivity to parameters. While this method provides optimized parameter ES, there is some spin-up time until the ES reaches the optimal value. This is unlikely to be problematic when estimating parameters in GCMs during the integration time of several tens of days, as carried out by Ruiz et al. (2013b). However, this could be fatal if, for example, we want to estimate parameters in cloud microphysics schemes by assimilating radar reflectivity data for less than an hour. Therefore, a suitable ES value is required for the ensemble from the initial time to estimate parameters effectively. Ordinarily, model sensitivity to each parameter is unknown a priori, making it unclear what constitutes a suitable ES value. To establish a reliable and effective estimation method, there is a need to advance the current understanding of how the parameter ES affects the precision and convergence speed of the estimation.

This study aims to clarify how the precision and convergence speed varies depending on the prescribed constant ES. This will help develop a procedure to determine the optimal ES that provides precise and rapidly converging estimation. An observing system simulation experiment (OSSE) which estimates a cloud microphysics parameter with the assimilation of radar reflectivity data was conducted. Different constant ES values were set for an estimated parameter, and we investigated the dependence of precision and convergence speed on the ES value. Finally, we approximated the estimation time series using a simple mathematical model to quantify the precision and convergence speed of the parameter estimation. We also checked the accuracy of parameter estimation because, in general, there is no guarantee that the true value can be estimated accurately in the OSSE. Note that accuracy and precision are different metrics: The accuracy means how close the estimated value is to the true value; the precision means how close the estimated values for different trials are to one another. Thus, it is possible that the estimation shows high precision but poor accuracy due to some systematic errors in the estimated values.

We focused on cloud microphysics parameters as they significantly affect the prediction of moist convective phenomena. Deep moist convection often results in severe weather, such as heavy rain and strong wind gusts. The optimization of cloud microphysics parameters may help preserve life and property by reducing the forecasting error of severe storms and providing accurate weather information. The direct assimilation of convective cloud observations (e.g., radar reflectivity) continues to be challenging because of the possible discrepancy for convection appearance between a numerical model and nature. A quantitative understanding of the precision and convergence speed of parameter estimation for cloud microphysics is essential to expanding the current knowledge on convective-scale data assimilation based on the EnKF.

This paper has been organized into four additional sections. Section 2 describes the methodology of our experiment, Sect. 3 presents the results, Sect. 4 discusses the results, and Sect. 5 presents the conclusions of this study.

Methodology

Overview

This study carried out OSSE within the framework of a perfect-model twin experiment. The nature run was an idealized squall-line simulation using a bulk cloud microphysics scheme with fixed parameter values. One of the microphysics parameters in the nature run was considered the true value and a target of parameter estimation. Figure 1 shows the schematic of the experiment; the DA cycle using the LETKF had commenced from T = 3 h of the nature run. The initial fields of the ensemble simulation were generated by adding random perturbations to the state fields of the nature run at T = 3 h. Pseudo-radar reflectivity fields calculated from the nature run were used as observations for the DA cycle. During the first 1 h DA cycle, only the state estimation was conducted without parameter estimation, where the ensemble used the same parameter value as the true value of the nature run. During this period, the ES of state fields gradually increased and then became steady. Following this, perturbation was added to the target parameter in the ensemble and parameter estimation commenced from T = 4 h of the nature run.

Fig. 1
figure 1

Schematic of the experiment. First, the nature run was performed (horizontal blue arrow). State estimation using the LETKF with 32 ensemble members commenced at T = 3 h (black arrows). Pseudo-radar reflectivity fields calculated from the nature run were assimilated into the ensemble simulation every 5 min. Parameter estimation commenced at T = 4 h (red arrows)

Nature run

A 2D squall-line simulation was carried out to produce the nature run using a non-hydrostatic atmospheric model of the Scalable Computing for Advanced Library and Environment-Regional Model (SCALE-RM; Nishizawa et al. 2015; Sato et al. 2015) version 5.3.3. The horizontal and vertical grid spacing was 5 km and 250 m, respectively. The size of the calculation domain was 2160 and 17.5 km in the horizontal and vertical directions, respectively. The integration time interval for the advection scheme was 2.5 s. Periodic conditions were applied horizontally with 100 km buffer regions from lateral boundaries for potential temperature, wind speed, and water vapor mixing ratio. In the buffer regions, gravity waves and cold air outflow by the squall line were damped. A six-class single-moment microphysics scheme (Tomita 2008) was utilized to simulate moist convective processes, the target of parameter estimation. Sub-grid scale turbulence parameterization was not used to avoid estimation complexity, and only numerical diffusion was added for model stability. Analytical thermodynamic and wind profiles proposed by Weisman and Klemp (1982) were used to provide a favorable environment for the development of the squall line. Figure 2 presents the environmental profiles of the potential temperature, water vapor mixing ratio, and horizontal wind speed. The maximum mixing ratio near the surface was set to 14 g kg−1, while the maximum wind speed at the top of the atmosphere was set to 10 m s−1. Figure 3 shows the development of the cold pool and snapshots of the pseudo-radar reflectivity. This simulation successfully produced a squall line that endured for more than one day.

Fig. 2
figure 2

Environmental profile for squall-line simulation: a potential temperature; b water vapor mixing ratio; and c horizontal wind speed

Fig. 3
figure 3

Cold pool and radar reflectivity of the simulated squall line in the nature run: a Hovmöller diagram of potential temperature at the lowest model level; bd snapshots of radar reflectivity, where (b) T = 6 h; (c) T = 18 h; and (d) T = 24 h. The three red line segments in (a) correspond to the times and locations of (bd)

Initial ensemble

The initial ensemble was generated by adding random perturbations to potential temperature, horizontal and vertical wind speeds, and the mixing ratios of water vapor and hydrometeors of the nature run at T = 3 h. Each perturbation was generated by summing 2D Gaussian kernels whose centers were at each model grid, and horizontal and vertical width (1σ) was 5 km and 250 m, respectively. The amplitude of the Gaussian kernel was randomly sampled from a normal distribution with a zero mean and a specific standard deviation. The standard deviation of the kernel amplitude differed for each variable: 0.2 K for potential temperature, 0.2 m s−1 for wind speed, and 0.06 g kg−1 for mixing ratio. Note that perturbations were added only to a specific domain; 400 km horizontally around the squall line and 10 km vertically from the surface.

State and parameter estimation

The SCALE-LETKF system (Lien et al. 2017) which incorporates the DA system using the LETKF into the SCALE-RM was used. We performed the simultaneous estimation of state variables and a parameter with different localization scales; the ensemble size was 32. Pseudo-radar reflectivity fields obtained from the nature run were assimilated into the SCALE-RM ensemble simulation every 5 min. Model grids with a reflectivity of ≥ 10 dBZ in the nature run were considered precipitation data, while those < 10 dBZ were considered non-precipitation data and replaced with 5 dBZ uniformly prior to assimilation. Radar reflectivity observation error was assumed to be 1 dBZ. Note that precipitation data in the nature run were only assimilated if at least half the ensemble members (16 members) at corresponding grids also experienced precipitation. In contrast, non-precipitation data were assimilated if there is even one member experiencing precipitation to remove precipitation that does not exist in the nature run from the ensemble simulation. Reflectivity data at grids that did not meet these conditions were not assimilated. The localization length scales for state estimation were 4 and 2 km in the horizontal and vertical directions, respectively. The model dynamics, physics, and domain in the SCALE-LETKF system were the same as the nature run. During the first 1 h of simulation (i.e., 12 DA cycles), the ensemble used common parameter values with the nature run. During this period, only state variables were estimated, such as wind speed, temperature, pressure, water vapor, cloud water, rain, cloud ice, snow, and graupel. The inflation technique, known as the relaxation to prior spread (RTPS) method (Whitaker and Hamill 2012), was used with a fixed relaxation parameter of 0.95 for the state estimation. Within this 1 h integration, the ES for the radar reflectivity field gradually increased and reached an almost steady state (Fig. 4). We did not perform a long spin-up period before starting DA but the grows of spread for every 5 min seem to be large enough to increase the spread gradually before starting the parameter estimation. The parameter estimation for the microphysics scheme commenced at T = 4 h of the nature run (Fig. 1). Unlike the state estimation, a non-localized ETKF was applied to estimate a spatially uniform parameter; this was a LETKF with an infinite localization scale. For the parameter ensemble, a relaxation parameter of one was utilized for the RTPS inflation to maintain the ES as a constant parameter throughout the experiment. This was the same approach as adopted by Kotsuki et al. (2018).

Fig. 4
figure 4

Time series of the analysis ensemble spread for radar reflectivity. The ensemble spread was defined as the mean sample standard deviation of the radar reflectivity averaged over model grids where the observations to be assimilated existed. Open squares and open circles indicate the spreads for the first guess and the analysis, respectively. The origin of the time axis corresponds to T = 3 h for the nature run

Parameter to be estimated

The estimation target was a parameter in a six-class single-moment microphysics scheme (Tomita 2008). This scheme predicts mass exchange among six categories of water substances: water vapor, cloud water, rain, cloud ice, snow, and graupel. The mass exchange processes consist of saturation adjustment, auto-conversion, accretion, evaporation, sublimation, deposition, melting, freezing, and the Bergeron process. These processes contain many tunable parameters, such as intercept and slope parameters for the Marshall–Palmer distribution of precipitating hydrometeors (Marshall and Palmer 1948), terminal velocity coefficients for these hydrometeors, collection efficiencies of accretion processes, and parameters controlling auto-conversion rates. As this study investigates the precision and convergence speed of parameter estimation in idealized settings, we selected parameters that were highly sensitive to observation (radar reflectivity). As such, the terminal velocity coefficient of a raindrop, cR, was selected as the unknown parameter:

$$v_{{t{\text{R}}}} = c_{{\text{R}}} \left( {D\frac{{\rho_{0} }}{\rho }} \right)^{1/2} ,$$
(1)

where vtR is the terminal velocity of the raindrop; D is the raindrop diameter; ρ is air density; and ρ0 is the reference air density of 1.28 kg m−3. In this experiment, the true value of cR is 130 in the nature run. We selected cR as the estimated parameter as it controls the falling speed of rain. If cR becomes larger, the falling speed of rain becomes faster (Eq. (1)), and thus, the area of intense radar reflectivity moves downward more quickly. To prevent cR from becoming a value that is too large or too small to be physically practical, a variable transformation was introduced (as per Eq. (2)), and the variable, x, was estimated and converted back to cR after the estimation:

$$c_{{\text{R}}} \left( x \right) = \frac{1}{2}\left( {c_{{{\text{R}}\max }} + c_{{{\text{R}}\min }} } \right) + \frac{1}{2}\left( {c_{{{\text{R}}\max }} - c_{{{\text{R}}\min }} } \right)\tanh x.$$
(2)

Kotsuki et al. (2018) also used this transformation. Note that cR was always between cRmin and cRmax for any x, where cRmin was set to zero as a negative cR causes the ascent of raindrops (Eq. (1)). On the other hand, cRmax was set to 260 such that the transformation of Eq. (2) was symmetric with respect to the true value of cR = 130. Figure 5 shows the cR(x) curve of Eq. (2); in the x space, x = 0 is the true value corresponding to cR = 130 in the nature run.

Fig. 5
figure 5

A variable transformation between cR and x. The curve of cR is plotted against x as per Eq. (2)

Experimental settings

At the initial time of parameter estimation (T = 4 h in Fig. 1), different parameter values were assigned to each member in the ensemble. Parameter values for each member were randomly sampled from a normal distribution with a mean of μinit and a standard deviation of σ in the x space. Note that the parameter ensemble only satisfied a normal distribution in the x space, and not the cR space as the transformation between cR and x was not linear (Eq. (2) and Fig. 5). The parameter perturbation was updated, and the μ of the ensemble was modified at every DA step. An updated μ was considered an estimated parameter value for each step. The standard deviation, σ, is a measure of the parameter ES. As stated in Sect. 2.4, the RTPS method with a coefficient of one was applied to the parameter ensemble; as such, σ was kept constant throughout the estimation.

The time-independent σ value is a key parameter of the experiment. We investigated the dependence of precision and convergence speed of the estimated parameter value (μ) on the parameter ES (σ). Four different ES values were tested: σ = 0.05, 0.1, 0.2, and 0.4 in the x space, corresponding to approximately 6.5, 13, 26, and 49 in the cR space, respectively, at μ = 0 (cR = 130). Note that the spread in the cR space becomes smaller when the cR value approaches cRmin or cRmax even though σ in the x space is kept constant (see Fig. 5). However, to discuss the dependence of precision and convergence speed on the parameter ES as simply as possible, we fixed the parameter ES in the x space throughout the estimation. Under the set σ values, three different initial ensemble means were tested: μinit = 0, 0.55, and − 0.55 in the x space, corresponding to cR = 130, 65, and 195. As the OSSE was a perfect-model twin experiment, it was expected that all experiments may be able to finally attain the true value of the nature run (x = 0), as per previous studies such as Tong and Xue (2008). Thus, experiments where μinit = 0 were expected to provide a statistically stationary time series of μ fluctuating around the true value and illustrate the estimation precision. We also expected that experiments where μinit = 0.55 and − 0.55 would result in convergence from the wrong values to the true value, illustrating the convergence speed of the estimation. All experimental settings are listed in Table 1.

Table 1 List of experimental settings

The left-most column shows the experimental ID; μinit represents the parameter ensemble mean at the initial time; and σ represents the time-independent standard deviation of the parameter ensemble (parameter ensemble spread). The right-most column details the initial time of each experiment corresponding to the simulation time of the nature run.

Results

Overview

Figure 6 shows the time series of the parameter ensemble mean for all experiments listed in Table 1; Fig. 6a–d presents the experiments for μinit = 0 (Exps. 1–4), where 50 trials with different initial parameter ensemble perturbations were carried out for each setting. This figure shows that the parameter ensemble mean successfully remains around the true value of the nature run (x = 0) with some fluctuation; these time series appeared to be statistically stationary. Figures 6a–d clearly illustrate that the fluctuation frequency varies depending on σ: the time series with a small ES fluctuated slowly, while a large ES fluctuated rapidly. The amplitude of the fluctuation was indicative of the precision; the time series with the largest ES (σ = 0.4) fluctuated with the largest amplitude, indicating low precision. The difference in the fluctuation amplitude between σ = 0.05 and 0.2 was unclear, while some extreme values appeared for the experiment of σ = 0.2.

Fig. 6
figure 6

Time series of parameter ensemble mean for all experiments: ad experiments with an initial ensemble mean μinit of 0 (Exps. 1–4). The parameter ensemble spread (σ) for each experiment was: a 0.05; b 0.1; c 0.2; and d 0.4. Black lines represent the time series for a total of 50 trials for each setting. Red lines represent averaged time series of all trials. The origin of the time axis corresponds to T = 4 h for the nature run; eh experiments where μinit = 0.55 (Exps. 5–8: red lines), and − 0.55 (Exps. 9–12: blue lines); e experiments where σ = 0.05 (Exps. 5 and 9); f σ = 0.1 (Exps. 6 and 10); g σ = 0.2 (Exps. 7 and 11); and h σ = 0.4 (Exps. 8 and 12). Initial times of the experiments were every 1 h from 3 h following the commencement of Exps.1–4. The estimation duration was 6 h for each experiment. The beginning and end points of estimation are denoted by circles and squares, respectively. Black lines represent the averaged time series of Exps.1–4, which are the same as the red lines in (ad)

Figure 7 shows the power spectral density of the time series for Exps. 1–4. For the high-frequency range, the spectral amplitude increased with σ. This corresponded to the frequently fluctuating time series for the large σ, as shown in Fig. 6. In all cases, the spectral amplitude decreased with increasing frequency, showing the "red noise" spectrum characteristics. The red noise spectrum is a feature that indicates the first-order autoregression process. In the later section, we approximate the experimental time series as the first-order autoregression model (AR(1) model) in order to understand the behavior of the time series clearly.

Fig. 7
figure 7

Power spectral density of the time series of parameter ensemble mean for Exps. 1–4. The red line is for the parameter ensemble spread (σ) of 0.05, the purple line is for σ = 0.1, the cyan line is for σ = 0.2, and the yellow line is for σ = 0.4. Each spectrum was calculated by applying the discrete Fourier transformation to the time series of each trial and averaging these values

Figure 6e–h shows the results from experiments with μinit = 0.55 (Exps. 5–8: red lines) and − 0.55 (Exps. 9–12: blue lines). For this set of experiments, there were 13 trials conducted for each setting. The initial times of the experiments were every 1 h from 3 h following the commencement of Exps. 1–4; the initial time of these experiments was shifted from Exps. 1–4 because the time series of the parameter ensemble mean for each trial of Exps. 1–4 (black lines in Fig. 6a–d) during the early simulation time did not sufficiently spread out from the initial value provided. As such, it was not appropriate to carry out a statistical comparison of the time series during this period. We also modified the initial times for each trial of Exps. 5–12, as the convergence speed may vary depending on the simulation time. The estimation duration was 6 h for each experiment. The parameters successfully converged toward the true value in the μinit = 0.55 and − 0.55 series; the convergence speed accelerated as σ became larger, which supports the findings from Kotsuki et al. (2018).

Accuracy

Although all experiments appeared to attain the true value of the nature run successfully, it was not clear whether the true value was accurately estimated for the entire simulation time. Figure 8 shows the averaged time series for all 200 trials of Exps. 1–4. Red open circles on the black line indicate that the mean of the estimates differs from the true value (x = 0) with a significance level of 1% based on the Student's t test. From 14 to 20 h following the commencement of parameter estimation, the estimated parameters tended to be higher than x = 0. During this period, time steps where the mean estimates significantly differed from the true value continued intermittently. This indicates that the estimation accuracy varies depending on the observations for each simulation time, despite carrying out the estimation under the twin experiment framework. In this study, we considered the 3 h moving average of the mean time series (blue line in Fig. 8) as the bias component common to all experiments. Sections 3.3 and 3.4 discuss the precision and convergence speed of the estimation, accounting for this estimation bias.

Fig. 8
figure 8

Averaged time series for all the trials of Exps. 1–4. The black line is the averaged time series. Color shade represents the standard deviation of all trials. Red open circles indicate that the mean of the estimates differs from the true value (x = 0) with a significance level of 1% based on the Student's t test. The blue line shows the 3 h moving average of the black line

Precision

The estimation precision is defined as how close the estimated values for different trials are to one another. In this study, the metric of estimation precision was the sample standard deviation of estimation time series with the bias component removed. The sample standard deviation of the estimated parameter was calculated as:

$$\sigma_{\mu } = \sqrt {\frac{1}{T}\mathop \sum \limits_{t = 1}^{T} \left\{ {\mu \left( t \right) - x_{{{\text{bias}}}} \left( t \right)} \right\}^{2} } ,$$
(3)

where μ(t) is the time series of the parameter ensemble mean for each time step, t; xbias(t) is the bias component (blue line in Fig. 8); and T is the number of time steps of the estimation. Here, a smaller σμ indicates higher precision; Fig. 9 presents the mean σμ, averaged for all 50 trials for each parameter ES (σ). The σμ became smallest when σ was 0.1 or less; this implied that σ = 0.1 was the threshold for the parameter ES. There were no further improvements to the estimation precision even if σ was further reduced from a value of 0.1. As the convergence speed accelerates as σ decreases (see Sect. 3.4), a σ of 0.1 was optimal to provide the best precision and better convergence speed in this experimental framework. Section 3.5 presents the quantification of the dependency between precision and the parameter ES using the AR(1) model.

Fig. 9
figure 9

Sample standard deviation of estimated parameters for Exps. 1–4. The standard deviation for each trial was calculated after removing the bias component (blue line in Fig. 8). Vertical bars represent a confidence interval of 99%

Convergence speed

Figure 10 illustrates the composite time series of the parameter ensemble mean for Exps.5–12. Note that the bias component shown in Fig. 8 was removed from each time series prior to compositing. The convergence speed accelerated as σ became larger. The open circles in Fig. 10 indicate that the set of time series did not differ from that for Exps. 1–4, with a significance level of 1% based on the Student's t test. With the exception of Exp. 5 (red line in Fig. 10a), all experiments converged to statistically stationary time series within 6 h.

Fig. 10
figure 10

Composite time series of parameter ensemble mean for Exps. 5–12. Red (blue) lines are for the initial ensemble mean of μinit = 0.55 (− 0.55); a experiments where parameter ensemble spread (σ) was 0.05 (Exps. 5 and 9); b σ = 0.1 (Exps. 6 and 10); c σ = 0.2 (Exps. 7 and 11); and d σ = 0.4 (Exps. 8 and 12). Note that the bias component (xbias) shown as the blue line in Fig. 8 was removed from each time series prior to compositing. Color shade represents the standard deviation of all trials. The origin of the time axis corresponds to the initial times of each trial. Open circles on each line indicate that the set of time series does not differ from Exps.1–4, with a significance level of 1% based on the Student's t test

Figure 11 shows the convergence times for each experiment; here, the convergence time was defined as the period from the initial time to the first open circle in Fig. 10. For experiments where σ = 0.4, the parameter converged within only a few DA cycles; this occurred at ~ 10 min. This means that if σ was not too small, successful parameter estimation occurred within a shorter time than the time scale of mesoscale convective systems (a few hours). The convergence speed is related to the persistence of the time series. This was discussed in quantitative terms by considering the AR(1) model in Sect. 3.5. Notably, when σ ≤ 0.2, the convergence speed for μinit = 0.55 (Exps. 5–7) was slower than for μinit = − 0.55 (Exps. 9–11). This indicates that the convergence speed differs depending on whether parameter estimation begins from a larger or smaller value than the true value. This is likely to imply that model sensitivity to the cR value was not symmetric to cR = 130. In addition, it is possible that the parameter likelihood itself was not symmetric around the true value.

Fig. 11
figure 11

Convergence time plotted against the parameter ensemble spread (σ). Red line shows Exps. 5–8 where the initial ensemble mean of μinit = 0.55; blue line shows Exps. 9–12 where μinit = − 0.55. The time series of Exp. 5 does not converge within 6 h, and as such, was not plotted

Mathematical model of parameter estimation time series

Thus far, this study has demonstrated that although the precision of parameter estimation may be maximized for σ ≤ 0.1 (Fig. 9), the convergence speed for estimation accelerates as σ becomes larger, and as such, the convergence time is shortened (Fig. 11). To quantify the precision and convergence speed of parameter estimation, we attempted to model the temporal variation of the estimated parameter. The spectral amplitude that decreases with increasing frequency in Fig. 7 suggests the estimation time series was close to the AR(1) time series. Here, we approximated the time series of the parameter ensemble mean using the AR(1) model:

$$\mu^{\prime } \left( t \right) = \phi \mu^{\prime } \left( {t - 1} \right) + \varepsilon \left( t \right),$$
(4)

where t is each DA time step; μ′(t) ≡ μ(t) − xbias(t) is the time series of the parameter ensemble mean where the bias component is removed; ϕ is the autoregressive parameter; and ε(t) is a random perturbation. The expected value of ε(t) is assumed as zero (i.e., E[ε] = 0), and its variance is referred to as σε2 = E[ε2], where E[a] indicates the expected value of a. When the time series by Eq. (4) is in a statistically stationary state, E[μ′] = 0. Under this condition, the variance of μ′ is defined as, σμ2 = E[μ2]. As the AR(1) model assumes that μ′ and ε have no correlation (i.e., E[με] = 0), the following relationship between σμ2 and σε2 is satisfied:

$$\sigma_{\mu }^{2} = \phi^{2} \sigma_{\mu }^{2} + \sigma_{\varepsilon }^{2} .$$
(5)

Then, the lag-1 autocorrelation coefficient of μ′, ρ(1), satisfies the following:

$$\rho \left( 1 \right) = \frac{{E\left[ {\mu^{\prime } \left( t \right)\mu^{\prime } \left( {t - 1} \right)} \right]}}{{\sigma_{\mu }^{2} }} = \phi .$$
(6)

Note that the range of ϕ is (− 1, 1). In the AR(1) model, ϕ is a metric of the time series persistence, where if ϕ is close to 1, μ′(t) is highly correlated to the previous value μ′(t − 1). In terms of parameter estimation, this means that if a parameter value at one step deviates considerably from the true value, the parameter at the next step also tends to be highly deviating, resulting in a slow convergence speed.

For each trial, σμ2 was estimated from Eq. (3); then, using Eq. (6), ϕ was estimated as:

$$\phi = \frac{1}{T - 1}\mathop \sum \limits_{t = 2}^{T} \mu^{\prime } \left( t \right)\mu^{\prime } \left( {t - 1} \right)/\sigma_{\mu }^{2} .$$
(7)

Finally, using Eq. (5), we estimated the standard deviation of the random perturbation, σε, as:

$$\sigma_{\varepsilon } = \sigma_{\mu } \sqrt {1 - \phi^{2} } .$$
(8)

Figure 12 shows the ϕ and σε values, estimated by calculating Eqs. (7) and (8) for all trials and averaged for each experimental setting; here, ϕ approaches 1 as σ decreases. This trend corresponds to a slow convergence speed of the estimated parameter for the experiment with a smaller σ. At the same time, σε increases with σ; this trend corresponds to a larger amplitude of the time series fluctuation for experiments with a larger σ.

Fig. 12
figure 12

Two AR(1) model parameters estimated from the time series of Exps. 1–4: a estimated autoregressive parameter ϕ of Eq. (7); and b estimated standard deviation of the random perturbation σε of Eq. (8). Each plot represents the mean value for all the trials of Exp. 1 (σ = 0.05), Exp. 2 (σ = 0.1), Exp. 3 (σ = 0.2), and Exp. 4 (σ = 0.4). The vertical bars represent a confidence interval of 99% for each estimated value

The power spectral density function of AR(1) model is expressed as:

$$S\left( f \right) = \frac{{4\sigma_{\varepsilon }^{2} /T}}{{1 + \phi^{2} - 2\phi \cos \left( {2\pi f} \right)}},$$
(9)

where f is frequency (Wilks 2011). Here, f was discretized as f = n/T for n = 0, 1, …, T/2; f = 1/2 was the Nyquist frequency corresponding to a 10 min period. Figure 13 presents the power spectral density function of Eq. (9) using the estimates of ϕ and σε in Fig. 12 (dotted lines), alongside the power spectral density directly derived from the experimental time series (solid lines). Note that the experimental power spectral density slightly differed from Fig. 7 because the bias component was removed from each time series. For σ = 0.05, the spectrum function was in good agreement with the experimental spectrum, except for the right end of the spectrum (red lines). We inferred that the AR(1) model could suitably approximate the parameter estimation time series for the target parameter cR. As σ increased, the spectrum function deviated slightly from the experimental spectrum, particularly for low-frequency components. This disagreement may be related to the relatively large uncertainty of estimated ϕ, when σ is large (Fig. 12a). As for the disagreement in the low-frequency range, it is also possible that the frequency components having a period of several hours and more were not detected accurately since the simulation time was just 24 h. From the perspective of the AR(1) model, the rapidly fluctuating time series for a large σ is explained in terms of the dependence of power spectral density function on ϕ. As the parameter ES (σ) becomes larger, the autoregressive parameter ϕ value declines (Fig. 12a), resulting in the increase of power spectral density for the high-frequency (larger f) range as per Eq. (9).

Fig. 13
figure 13

Power spectral density function assuming the AR(1) model. The lines of spectral functions were based on Eq. (9) using estimated values of autoregressive parameter ϕ and standard deviation of the random perturbation σε for experiments where σ = 0.05 (red), 0.1 (purple), 0.2 (cyan), and 0.4 (yellow) (dotted lines). Solid lines show power spectral density directly derived from the experimental time series. The experimental power spectral density differed slightly from Fig. 7 because the bias component (blue line in Fig. 8) was removed from each time series prior to calculating power spectral density

Quantitative understanding of precision and convergence speed

This section quantifies the precision and convergence speed of parameter estimation based on the AR(1) model. Figures 9 and 11 present the experimental results, illustrating the dependence of precision and convergence time on the parameter ES (σ). This section reconsiders this in terms of the dependence of ϕ and σε in Eq. (4) for σ. Based on Eq. (4), the convergence from the initial wrong value to the true value is expressed as:

$$\left[ {\mu^{\prime } \left( t \right)} \right] = \phi^{t} \mu^{\prime } \left( 0 \right),$$
(10)

where μ′(0) is the initial parameter ensemble mean; we used E[ε] = 0. The e-folding time was estimated by substituting μ′(0)/e into the left-hand side of Eq. (10) and solving it for t:

$$t = - \frac{1}{\ln \left( \phi \right)}.$$

Here, we assumed ϕ > 0, which is reasonable for this experiment (Fig. 12a). As the DA time step, t, should be an integer, the actual time step where the expected deviation from the true value is < 1/e of its initial deviation assuming the AR(1) model, is:

$$t_{{{\text{AR}}\left( 1 \right)}} = \left\lceil- \frac{1}{\ln \left( \phi \right)}\right\rceil.$$
(11)

Here, \(\lceil a \rceil\) indicates the least integer greater than or equal to a. Figure 14 shows the tAR(1) calculated using Eq. (11), where the estimated ϕ was adopted from Fig. 12a. The right vertical axis of the figure presents the actual time in the correspondence, where one time step is 5 min. The AR(1)-based convergence time in Fig. 14 was in good agreement with the convergence time shown in Fig. 11, indicating that the AR(1) model approximation is reasonable. The convergence speed accelerates as σ increases. In terms of convergence time, a larger σ was more advantageous for parameter estimation; however, the acceptable convergence time was dependent on the time scale of the phenomena (e.g., the time scale of deep convective clouds). If convergence time is within an acceptable range, the optimal parameter ES should be determined to maximize precision.

Fig. 14
figure 14

Time step until the expected deviation from the true value becomes < 1/e assuming the AR(1) model. Each value was calculated based on Eq. (11) using the estimated autoregressive parameter, ϕ, for experiments where σ = 0.05, 0.1, 0.2, and 0.4 (Fig. 12a). The right vertical axis shows the actual time in correspondence where one time step is 5 min

The standard deviation of the estimated σμ is a metric of estimation precision, which is expressed by rewriting Eq. (5) for σμ:

$$\sigma_{{\mu {\text{AR}}\left( 1 \right)}} = \frac{{\sigma_{\varepsilon } }}{{\sqrt {1 - \phi^{2} } }}.$$
(12)

Figure 15 shows the σμAR(1) obtained using Eq. (12), where the estimated ϕ and σε values are shown in Fig. 12; here, σμAR(1) was minimized at σ = 0.05. However, the σμAR(1) values for σ = 0.05 and 0.1 did not differ greatly; σμAR(1) = 9.63 × 10−2 for σ = 0.1, was only 4% larger than σμAR(1) = 9.27 × 10−2 for σ = 0.05. By contrast, σμAR(1) = 1.06 × 10−1 for σ = 0.2, was 11% larger than for σ = 0.1. Thus, we suppressed the estimation uncertainty by setting σ to ≤ 0.1. With a decrease in σ, ϕ and σε were asymptotic to 1 and 0, respectively (Fig. 12). This means that the numerator and denominator on the right-hand side of Eq. (12) becomes zero in the σ = 0 limit. By decreasing σ, the behavior of σμAR(1) is determined by how the numerator and denominator of Eq. (12) approach zero. For the current target parameter, the ratio of the speed at which the denominator and numerator were asymptotic to zero was almost constant for σ ≤ 0.1; σμAR(1) was minimized for this condition. This demonstrates that the parameter ES of σ ≤ 0.1 achieves the best estimation precision.

Fig. 15
figure 15

Standard deviation of estimated values assuming the AR(1) model. This standard deviation was based on Eq. (12) using the estimated ϕ and σε for experiments where σ = 0.05, 0.1, 0.2, and 0.4 (Fig. 12)

Following this, we determined the minimum ES for an acceptable convergence time. The convergence time for σ = 0.1 was approximately 1 h, as shown in Fig. 14. This time scale was comparable or shorter than long-lasting convective systems. On the other hand, the convergence time for σ = 0.05 was approximately 3 h. This is likely to be excessive duration for practical parameter estimation to target deep convection. In terms of convergence speed, we accepted a value of σ ≥ 0.1; and therefore, concluded that σ = 0.1 was the optimal parameter ES to estimate true cR in this twin experiment. Table 2 summarizes the estimated ϕ and σε values for different σ, and the convergence time step and the standard deviation of the estimated σμ, on the basis of the AR(1) fitting.

Table 2 List of parameters assuming the AR(1) model for different parameter ensemble spread

The first and second rows show the estimated autoregressive parameter, ϕ, and standard deviation of the random perturbation, σε, respectively. The third row presents the tAR(1) calculated using Eq. (11); the fourth row presents σμAR(1) obtained using Eq. (12); these values were calculated using the estimated ϕ and σε.

Discussion

As the previous section illustrated, the AR(1) model was able to sufficiently approximate the time series for cR value estimation. As such, the optimal ES may be discussed by considering the autoregressive parameter and the amplitude of random perturbation in the AR(1) model. However, there were also discrepancies with the simple AR(1) time series; for example, for the target parameter, cR, Fig. 11 shows that the convergence speed varies slightly, depending on whether the estimation starts from a value larger than or smaller than the true value. This may be because the autoregressive parameter ϕ in Eq. (4) is asymmetrically distributed around the true value instead of being a constant. If the autoregressive parameter under constant ES varies depending on model sensitivity to the parameter, it is expected that the estimation time series will follow the AR(1) model more closely by using the method in Ruiz et al. (2013b). Their method modified the parameter ES depending on model sensitivity to the parameter. It also raises an interesting question as to whether the optimal parameter ES estimated by their method approaches the ES determined based on our AR(1) model approach. This issue should be addressed in the future by applying the method in Ruiz et al. (2013b) to this twin experiment and comparing the results.

In the present study, we chose the AR(1) model because of its simplicity in interpretation. However, in general, we can approximate the time series as an autoregression model with an arbitrary order of p (AR(p) model) by considering the autocorrelation from lag-1 to lag-p. In this case, we can construct a model fitting better with the experimental time series by using larger p, but we should choose reasonable p value based on some information criteria (e.g., Akaike's information criterion). It is possible that the relative magnitudes for each autoregressive parameter of the AR(p) model are related to the time scales of various physical processes involved in a target phenomenon, in this case a squall-line convection system. In the present study, both state variables and the parameter were updated at each DA step, and thus, the 2nd to pth autoregressive parameters might become smaller than the 1st one. However, if we conduct the state estimation with a longer time interval than that for the parameter estimation, the magnitudes of second and subsequent autoregressive parameters may become larger depending on the time scale of physical processes.

Note that the method of parameter transformation affects the estimation results. In the case of the transformation of Eq. (2), when the cR value approaches its upper or lower limit, the spread in the cR space becomes smaller even though σ in the x space does not change. In this case, the persistence of the estimate could become larger due to lower parameter sensitivity, leading to less convergence to the true value. If we apply more sophisticated method, such that σ in the x space becomes larger when the cR approaches its limit values to keep the cR spread approximately constant, the estimation time series might better fit the AR(1) model. Furthermore, we need to keep in mind the possible effects of constraining the lower limit of the parameter to zero. This could skew the parameter likelihood distribution, which might result in the estimation bias and/or the different convergence speed between larger and smaller initial parameter values. While this issue did not appear to have a significant impact on our results, it can cause some problems in the case of more complex experimental settings. Effects of parameter transformation settings on the precision and convergence speed need to be investigated in future research.

The results for cR estimation do not guarantee that any other parameter estimation follows the AR(1) time series. Parameters that have little or an indirect effect on observed values might show almost random-walk time series that is also the AR(1) process with ϕ = 1. Conversely, it may be suggested that parameters with a direct impact on observed values follow the autoregressive process and may easily be estimated. For parameter optimization using real observations (e.g., operational radar observations), the “true value” cannot be known a priori, unlike the twin experiment in this study. Furthermore, it is possible that a parameter value does not converge to one specific value in a real situation; it is unclear whether the time series of parameter optimization follows an autoregressive process. Thus, we suggest the following steps. First, we attempt some short-period optimization experiments beginning from different initial parameter ensemble means. Here, a relatively large parameter ES should be utilized, because, as shown in Fig. 12a, it is expected that the larger the ES is, stronger the autoregression (smaller the ϕ in Eq. (4)) becomes. Suppose there are some signs of autoregression in a time series during the experiment, such as a tendency for the parameter to begin at different initial values to approach the same value. In this case, we can apply the idea of optimal ES discussed in this study to the parameter to be optimized. Then, by estimating the dependence of the autoregressive parameter and the amplitude of the random perturbation on the parameter ES based on some experiments, we can identify its threshold in terms of precision. Finally, we determine the optimal ES by comparing the expected convergence time and the time scale of the target phenomena. If we determine the optimal ES, we can conduct a longer period estimation with sufficient reliability and efficiency. If the estimated parameter value is trapped near prescribed lower or upper limit of parameter transformation, the true value might be located outside the limit. In this case, we should extend the limit value.

This study also highlighted an interesting albeit problematic issue that the estimated value contains some bias; this was also true for the twin experiment framework using the perfect model (Fig. 8). The overestimation of the cR for T = 14–20 h in the experiment implies that a slightly larger value than the true cR = 130 would reduce prediction error during this period. This is likely a result from the observations during this period. While the nature run of the experiment provided one realization under cR = 130, it is not necessarily the most likely realization in the state space under this condition. The time evolution of the nature run for T = 14–20 h probably went through the path which was the most likely realization when cR value was slightly larger than 130. This estimation bias may be removed if the parameter estimation was carried out over a longer simulation time or the estimates were averaged over many different simulations. For parameter optimization using real observations, the “true value” is likely to vary depending on time and space. The bulk cloud microphysics scheme used was highly simplified compared with real microphysics. Thus, we would be unable to determine the unique optimal parameter values that produce the best prediction accuracy under various atmospheric conditions. The optimized parameter values for the cloud microphysics scheme will vary depending on the type of target convective system, model resolution, and combination with other physical models. In parameter optimization for practical numerical weather prediction, there is a need to clarify the phenomenon to be predicted, the parameter that requires optimization for such a phenomenon, and how the parameter interacts with observed values.

Note that there were also many issues to be considered, even for the simple twin experiment in this study. For example, altering the time interval of the DA may also change the precision and convergence speed of parameter estimation. This study assimilated the radar reflectivity every 5 min; if the time interval is reduced, the effects of different parameter values on the radar reflectivity distribution may not be apparent during the ensemble forecast, making parameter estimation difficult. Conversely, if the time interval is lengthened, nonlinearities of the microphysical processes collapses a clear correlation between the target parameter and radar reflectivity, also making parameter estimation difficult. In this case, however, the possible detrimental effect of nonlinearities may be mitigated by conducting the state estimation with a time interval shorter than that for the parameter updates. Ensemble size is also an important factor of data assimilation; increasing ensemble size may improve the precision and convergence speed of the estimation without altering other settings. Additionally, the simultaneous estimation of multiple parameters may change some results. In reality, the optimal parameter values estimated concurrently may differ from those estimated separately due to interactions among multiple parameters. The estimation of multiple parameters is an area that should be addressed in future research.

In addition to these issues, how to determine the optimal ES for the estimation of time-dependent parameters needs further discussion. When parameters to be estimated are time-dependent, slow convergence speed could be more fatal than low precision. The parameter ensemble mean might never reach the true value at each time step because the estimate cannot follow the time variation of it due to the slow convergence speed. In this case, it is better to prioritize the convergence speed over the precision in determining the better parameter ES. Other compromise solution is to estimate the parameter for a longer period and average the result to determine one optimal value.

Finally, we note that the dependence of the precision and convergence speed on the parameter ES should be explained from the mathematical view of the data assimilation. Large persistence of the time series for the small parameter ES might reflect low parameter sensitivity due to the small parameter uncertainty. On the other hand, large fluctuation amplitude for the large parameter ES seems to be more difficult to be explained. It could reflect some interaction among the parameter uncertainty, the observational errors, and the state variables' uncertainties. In addition, it is important to investigate how the settings for the parameter uncertainty affects the accuracy of state estimation to understand the relation between parameter and state estimations. Theoretical studies on these issues should be addressed in future research.

Conclusions

This study carried out an OSSE, focusing on model parameter estimation; the dependence of the precision and convergence speed of estimation on the parameter ES was investigated. We experimentally demonstrated that the precision deteriorates with increasing parameter ES, while the convergence speed accelerates. However, there is a threshold for ES, where any smaller values do not result in any further improvements to the estimation precision. By using this approach, we determined the optimal parameter ES that provides the best precision and reasonable convergence time shorter than the time scale of target phenomena. At the same time, we checked the accuracy of estimation. We found there exists some systematic errors of estimated values which depend on the observation at each time step. In contrast to the precision and convergence speed, the accuracy cannot be controlled by setting the parameter ES.

In addition, we clarified that the time series of parameter estimation follows a simple mathematical model. We approximated the estimation time series as the AR(1) model, demonstrating that the precision and convergence speed of the estimation may be quantified by two AR(1) model parameters: the autoregressive parameter and the amplitude of random perturbation. As the ES of the parameter increases, the autoregressive parameter decreases, contributing to improving the precision and accelerating convergence speed. By contrast, as the amplitude of random perturbation increases; this reduces precision. From this perspective, the estimation precision was determined based on the balance between autoregression and random perturbation. This viewpoint is also useful in determining the optimal ES for other parameters that were not the focus of this study.

The final objective of this study was to establish a practical procedure to optimize the parameters of atmospheric models by assimilating real observations and improving the accuracy of numerical weather and climate prediction. For example, parameter optimization for cloud-resolving models using state-of-the-art observation equipment, such as phased array weather radars, may improve the accuracy of severe weather predictions. However, there is still a large gap between the findings of this study and achieving this final objective. From this study, we will gradually complicate the problem settings. For example, we will attempt parameter optimization of a low-resolution model by assimilating high-resolution simulation. In addition, optimization for a single-moment cloud microphysics scheme will be carried out by assimilating simulations with a more precise double-moment scheme. The simultaneous estimation of multiple parameters and the estimation of time-dependent parameters are also important issues; by successively building knowledge, practical applications of parameter optimization for actual numerical weather predictions may be achieved.

Availability of data and material

The code for SCALE-RM version 5.3.x is freely available at https://scale.riken.jp under the 2-Clause BSD license. The code for the SCALE-LETKF system is available upon reasonable request from the Data Assimilation Research Team of RIKEN Center for Computational Science (R-CCS) (http://www.data-assimilation.riken.jp/index_e.html). The model configuration files and experimental results were deposited in local storage at the R-CCS, Kobe, Japan. They are available upon request from the corresponding author.

Abbreviations

4DEnVar:

Four-dimensional ensemble-variational method

4DVar:

Four-dimensional variational method

AR(1) model:

First-order autoregression model

DA:

Data assimilation

EnKF:

Ensemble Kalman filter

EnSRF:

Ensemble square root filter

ES:

Ensemble spread

ETKF:

Ensemble transform Kalman filter

GCM:

General circulation model

LETKF:

Local ensemble transform Kalman filter

RTPS:

Relaxation to prior spread

SCALE-RM:

Scalable computing for advanced library and environment-regional model

References

Download references

Acknowledgements

We would like to thank the two reviewers of our manuscript for their helpful suggestions. We would like to thank the Data Assimilation Research Team of RIKEN Center for Computational Science (R-CCS) for providing us with the SCALE-LETKF system and their constructive comments and suggestions. We would also like to thank Editage (www.editage.com) for English language editing.

Funding

This study was supported by JST CREST Grant Number JPMJCR1312, JST AIP Grant Number JPMJCR19U2, and the Foundation for Computational Science (FOCUS) Establishing Supercomputing Center of Excellence, Japan.

Author information

Authors and Affiliations

Authors

Contributions

KS and HT proposed the topic, conceived, and designed the study. SN proposed a basic idea for the analysis. TY prepared and improved DA system. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kenta Sueki.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sueki, K., Nishizawa, S., Yamaura, T. et al. Precision and convergence speed of the ensemble Kalman filter-based parameter estimation: setting parameter uncertainty for reliable and efficient estimation. Prog Earth Planet Sci 9, 47 (2022). https://doi.org/10.1186/s40645-022-00504-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s40645-022-00504-4

Keywords

  • Parameter estimation
  • Data assimilation
  • Ensemble Kalman filter
  • Atmospheric model
  • Cloud microphysics
  • Deep convective system