Outcomes and challenges of global high-resolution non-hydrostatic atmospheric simulations using the K computer
Progress in Earth and Planetary Science volume 4, Article number: 13 (2017)
This article reviews the major outcomes of a 5-year (2011–2016) project using the K computer to perform global numerical atmospheric simulations based on the non-hydrostatic icosahedral atmospheric model (NICAM). The K computer was made available to the public in September 2012 and was used as a primary resource for Japan’s Strategic Programs for Innovative Research (SPIRE), an initiative to investigate five strategic research areas; the NICAM project fell under the research area of climate and weather simulation sciences. Combining NICAM with high-performance computing has created new opportunities in three areas of research: (1) higher resolution global simulations that produce more realistic representations of convective systems, (2) multi-member ensemble simulations that are able to perform extended-range forecasts 10–30 days in advance, and (3) multi-decadal simulations for climatology and variability. Before the K computer era, NICAM was used to demonstrate realistic simulations of intra-seasonal oscillations including the Madden-Julian oscillation (MJO), merely as a case study approach. Thanks to the big leap in computational performance of the K computer, we could greatly increase the number of cases of MJO events for numerical simulations, in addition to integrating time and horizontal resolution. We conclude that the high-resolution global non-hydrostatic model, as used in this five-year project, improves the ability to forecast intra-seasonal oscillations and associated tropical cyclogenesis compared with that of the relatively coarser operational models currently in use. The impacts of the sub-kilometer resolution simulation and the multi-decadal simulations using NICAM are also reviewed.
The K computer began operation in September 2012 at RIKEN Advanced Institute for Computational Science, Japan. The Strategic Programs for Innovative Research (SPIRE), an initiative launched in October 2010 in Japan, aims to leverage the unparalleled power of the K computer to produce the world’s most cutting-edge simulation studies in five key strategic fields. SPIRE also aims more generally to boost the creation of promotional frameworks for computer science and technology and to yield significant social breakthroughs. Field 3 of SPIRE, “Advanced prediction research for natural disaster prevention and reduction,” consists of weather and climate simulations, including earthquake and tsunami simulations (Oishi et al. 2015; Tsuboi et al. 2016; Ando et al. 2016; Hyodo et al. 2016). Studies using meso-scale atmospheric modeling (Saito et al. 2013; Kunii 2014a, 2014b; Chen et al. 2015a, 2015b; Duc et al. 2015; Ito et al. 2015; Kobayashi et al. 2015a; Seko et al. 2015; Yokota et al. 2016) and global modeling are also included under weather simulations. Here, we review the outcome of the global high-resolution atmospheric modeling studies conducted from 2012 to 2016 under Field 3 of SPIRE and preview the challenges for the next K computer project, called the FLAGSHIP2020 project (Future LAtency core-based General-purpose Supercomputer with HIgh Productivity). Running from 2014 to 2019, FLAGSHIP2020 is in preparation for the Post-K supercomputer, which was originally planned to start operation in 2020. (Recently, it was announced that the operation of the Post-K supercomputer has been delayed by at least 2 years.)
For global atmospheric modeling using the K computer, new research using the non-hydrostatic icosahedral atmospheric model (NICAM; Tomita and Satoh 2004; Satoh et al. 2008, 2014) has been pursued by enhancing horizontal resolution, ensemble sizes, or duration of integration time. In particular, we established the following research targets at the beginning of the project: (1) enable extended-range forecasts for a timeframe of 10–30 days, for better simulations of intra-seasonal oscillations (ISO) and (2) enable projection of tropical cyclones. Before the K computer became available, NICAM was known to more accurately reproduce individual ISOs in case studies at a horizontal grid spacing between 3.5 and 14 km (Miura et al. 2007a; Oouchi et al. 2009a, 2009b; Yanase et al. 2010). At that time, simulations such as these were performed mainly on the Earth Simulator at the Japan Agency for Marine-Earth Science and Technology (JAMSTEC). Multi-year simulations were also conducted with the 7 km grid spacing NICAM using the Athena computer (operated by the University of Tennessee’s National Institute for Computational Science and hosted by Oak Ridge National Laboratory in the USA) and showed better simulations of boreal summer ISO (Satoh et al. 2012b; Kinter et al. 2013). The development of the K computer offered the opportunity to obtain more robust performance from NICAM by drastically increasing ensemble size and integration time together with resolution.
This article reviews the outcomes of the NICAM studies using the K computer under Field 3 of SPIRE and previews challenges for the forthcoming Post-K supercomputer under the FLAGSHIP2020 project. In the “Computational aspects” section, we describe how NICAM was adapted for use on the K computer. Following this, we list major outcomes using NICAM with the K computer in the sections “Sub-kilometer global simulation,” “Asian summer monsoon,” “Madden-Julian oscillation,” “Boreal summer intra-seasonal oscillation and tropical cyclogenesis,” “Atmospheric Model Intercomparison Project-type experiments,” and “Projection of tropical cyclones.” The challenges to overcome in the near future are reviewed in “Future perspectives,” and we conclude our review in the final section, “Conclusions.”
The K computer was the first 10 petaflop supercomputer (1015 floating-point operations per second; Yokokawa et al. 2011). This massively parallel scalar machine was twice ranked as the world’s top supercomputer in the TOP500 List in 2011 (Top 500 2011a, 2011b). Furthermore, it showed the top performance in new benchmarks such as the high-performance conjugate gradient (Kumahata et al. 2016) and Graph500 (Graph 500 2014). Two recipients of the Gordon Bell Prize used the K computer to perform their award-winning work (Hasegawa et al. 2011; Ishiyama et al. 2012). In the research area of earth science, which Field 3 of SPIRE also covers, an urban-scale earthquake simulation study using the K computer was selected as a finalist for the Gordon Bell Prize (Ichimura et al. 2014, 2015). These results reflect the K computer’s high performance not only in benchmarks, but also in applications.
The K computer system possesses characteristics advantageous for weather and climate research, which require data-intensive simulation models. The performance of these models is limited not only by the number of calculations processed in the computer’s central processing unit but also by data transfer processes, such as the memory-cache transfer, inter-node communication, and file input/output (I/O). The K computer has relatively high memory throughput (0.5 byte per floating-point operation) compared to the today’s massively parallel scalar supercomputers. The asynchronous, distributed file I/O system of the K computer minimizes waiting time for frequent output of the atmospheric variables during the simulation. The K computer is also highly resilient and has a lower machine failure rate (Yamamoto et al. 2014) compared to other peta-scale supercomputers, which is very important for long-term climate simulations with a large number of computational nodes.
At an early stage of the development of the K computer, NICAM was selected as one of the target applications and was used in many of the computer’s performance optimizations. In experiments with a small number of nodes, we achieved about 10% of the performance efficiency (Terai et al. 2014; Yashiro et al. 2016b), which is almost competitive with that of the state-of-the-art weather and climate models. NICAM also shows good scalability, up to almost the full-node of the K computer in the weak scaling test.
For general model applications, a small number of major calculation parts occupy most of the floating-point operation and elapsed time. These parts are called hot spots, and we usually focus on these hot spots for the optimization of computational performance. However, most weather and climate models do not have a clear hot spot. Every part of the source code contains a small computation that consumes a little time, a characteristic called a flat profile. Weather and climate models generally have a flat profile, where every part of the source code has the potential to deteriorate the total performance. The effort to optimize NICAM on the K computer aimed to overcome the flat profile (Yashiro et al. 2016b). Optimizations were applied mainly to the calculation parts (kernels) such as divergence, gradient, and Laplacian operations. These kernels are a type of stencil calculation, which updates the grid point value using the value of surrounding grids. Because the finite-volume method is used for numerical discretization in NICAM, these kernels are the most important part for solving the fluid dynamics equations of the atmosphere. After optimization, each kernel showed improved performance (Terai et al. 2014). However, we achieved only a 1% gain in sustained performance over the total simulation, because the ratio of elapsed time of the kernels to total time decreased. This result was related to the effect of Amdahl’s law (Amdahl 1967). From this experience, we realized that we needed to find the time-consuming, less computationally intensive parts of the source code and remove them. As a result of these efforts, we achieved about 10% of the sustained performance, which was appropriate to the memory performance of the K computer. We also succeeded in reducing the model execution time by 30%.
Terai et al. (2014) performed a weak scaling test using NICAM on the K computer and showed good scalability of elapsed time and sustained performance of the NICAM (Fig. 1). The test was conducted by changing the number of nodes and the horizontal grid spacing. Horizontal grid spacing was decreased from 56 km to 440 m, which corresponded to an increase in the number of nodes from 5 to 81,920. The total time was divided into three parts: (1) the dynamical step, which covered the time taken to solve the fluid dynamics; (2) the physical step, which covered the physics components of the calculations, such as cloud microphysics, atmospheric radiation, and boundary layer turbulence; and (3) the communication step, which comprised the inter-node data transfer using message passing interface (MPI) (referred to as comm_data_transfer in Fig. 1). The computations and communications were completely separated using barrier statements of MPI (MPI_barrier), in which all tasks must synchronize. The result shows good scalability of total elapsed time. The timing of the dynamical step does not change as the number of nodes is increased, because the number of operations remains almost the same. The increase in the communication time as the number of nodes increases is not significant because most of the communications are conducted node-to-node using halo exchanges. The increase in the total elapsed time is mainly caused by increases in the time of the physical step due to the load imbalance between the processes. Terai et al. (2014) found that the imbalances occurred mainly in the cloud microphysics scheme. They found that the global map of the number of floating-point operations in the cloud microphysics showed inhomogeneous distribution, similar to that of ice hydrometeors. Overall, we achieved a maximum performance of 0.87 petaflops with 81,920 nodes (655,360 cores), as shown by Terai et al. (2014).
The results of the strong scaling test are different from those of the weak scaling test (Fig. 2). In the strong scaling test, we used NICAM with horizontal grid spacing of 14 km and 38 vertical layers. The number of grid points in each process decreased from 33,800 to 100 as the number of nodes increased from 80 to 40,960. The model showed good scalability up to 2560 nodes. The number of simulation days per wall-clock day increased in proportion to the number of nodes. However, the performance saturated beyond 2560 nodes, due to the relative increase in communication time; at 40,960 nodes, the communication time occupied over 50% of total elapsed time. This result led to production runs being conducted with 640 nodes in the studies described in the following sections. For long-term simulations in future studies, strong scalability will need to be improved. Reducing the number of subroutine calls within the MPI communication is one possible solution.
Pre- and post-processing
Owing to the comprehensive optimizations of NICAM, we achieved high efficiency and good scalability. However, the elapsed pre- and post-processing times were relatively increased in the total simulation time for the following reasons. The remapping process has one of the heaviest workloads in post-processing. To use common analysis and visualization tools, we usually remap the meteorological variables from an icosahedral grid to a geodesic (latitude-longitude) grid. For example, the data size of the global 0.87 km horizontal grid spacing simulations was 6 TB for one snapshot of all 3-D and 2-D variables (Miyamoto et al. 2013, 2015; Kajikawa et al. 2016). The required time for remapping and analyzing all data was estimated to be more than 2 months using typical computer clusters, which is much longer than the time required for simulation on the K computer. To solve this problem, we developed programs for parallel analysis on the K computer without remapping, thereby reducing the post-processing time from several months to less than 1 h. This improvement was due to massively parallel I/O rather than parallel computation. In the future, simulation sizes are expected to increase even more drastically. The most important consideration is the total elapsed time of the workflow in simulation studies, which includes pre- and post-processing, analysis, and visualization. All computational work will need to be carried out on the supercomputer to keep this time to a minimum.
NICAM has been selected as the target application for the development of Japan’s next-generation flagship computer (the Post-K supercomputer), together with the local ensemble transform Kalman filter (LETKF). NICAM and LETKF play an important role in the process of application-architecture co-design. Generally, computations of weather and climate models are not expected to be carried out faster than improvement in microprocessors, because the rate of improvement in memory throughput is slower than the floating-point operation speed. Thus, the computational performance of NICAM will be improved by modifications of the algorithm and its optimization (e.g., changes of loop order, data layout, and call structure).
Active use of mixed-precision floating-point operation is one possible approach to further enhance computational performance. The bit length of floating-point value affects the data size of memory transfer and the compression ratio of the output data. The effect of less precise values on the simulation results should be investigated. In recent years, the development of components and schemes such as the atmosphere-ocean coupling system with coupling library Jcup (Arakawa et al. 2014) and the ensemble-based data assimilation system (Terasaki et al. 2015) have expanded the capabilities of the simulations using NICAM. While taking these developments into account, we must increase the speed of total workflow. Maximizing the power of future supercomputers will enable simulations with even more sophisticated physics and/or higher resolution and will ensure that it becomes a powerful tool for further understanding weather and climate phenomena.
Sub-kilometer global simulation
One of the greatest challenges in climate modeling is achieving more accurate simulations of clouds (Stevens and Bony 2013; Palmer 2014). Among various types of naturally occurring clouds, deep moist convective clouds have important roles in determining the structure of atmospheric circulation through the release of latent heat. This deep convection is also part of the cloudy atmospheric disturbances that sometimes cause natural disasters. Since it has been impractical to simultaneously simulate all-scale phenomena in global atmospheric models, from deep convection on a scale of O(1) km to global circulation on a scale of O(104) km, deep convection has been expressed as a parameterization in conventional general circulation models (GCMs) (e.g., Arakawa and Schubert 1974; Kain and Fritsch 1990). NICAM was developed to simulate the global atmospheric circulation without cumulus parameterization (Satoh et al. 2008, 2014). Previous studies using NICAM have shown advanced performance in simulating large-scale convective systems and disturbances mainly using the Earth Simulator (e.g., Miura et al. 2007a; Sato et al. 2009; Nasuno and Satoh 2011; and references in Satoh et al. 2014). However, their grid spacing (several kilometers) was coarser than or similar to that of the observed convection scale. The highest resolution simulations using the Earth Simulator were the 3.5 km horizontal grid spacing simulations by Tomita et al. (2005) and Miura et al. (2007a, 2007b). Meanwhile, studies using cloud-resolving models in a confined area, such as large eddy simulation (LES) models, have indicated a change in deep convection characteristics at grid spacing less than 1 km (Petch et al. 2002; Bryan et al. 2003). Under Field 3 of SPIRE, we successfully conducted the first-ever global atmospheric simulation with sub-kilometer grid spacing using NICAM, which opened the door for simulations resolving deep convective cores in global atmospheric models. This work also allowed us to highlight the resolution dependency of simulated convection in the global model as well as its statistical features.
Miyamoto et al. (2013) conducted a set of grid-refinement experiments with grid spacing that varied from 14 to 0.87 km. Further analysis of these experiments is shown by Miyamoto et al. (2015), Kajikawa et al. (2016), and Yashiro et al. (2016a), and the details of the experimental settings are described in Miyamoto et al. (2013) and Kajikawa et al. (2016). Although the large-scale characteristics of global cloud distributions are almost unchanged between the simulations with 14, 3.5, and 0.87 km grid spacing, the simulation with sub-kilometer resolution provided a more detailed description of the smaller scale cloud structure (Fig. 3). The resolution dependency showed that the properties of simulated convection were qualitatively changed between 3.5 and 1.7 km grid spacing. We defined convective cores in the simulations and made a composite structure of convective cores averaged over the globe. In simulations at finer resolution than the 3.5 km grid spacing, the convective core is expressed by multiple grid points instead of a single grid point for the 3.5 km grid spacing; the dependency of the number of convective cores on resolution changes, with smaller dependency for finer resolutions (Fig. 4); and the minimum distance between convective cores is larger than four grid points for the finer resolutions than the 3.5 km grid spacing (Fig. 4). From these results, the horizontal grid spacing around 2 km can be viewed as the minimum required resolution for a deep convective resolving model over the entire Earth, although the required grid spacing might depend on numerical discretization methods.
Miyamoto et al. (2015) conducted detailed analysis of deep convection in the simulation with the finest 0.87 km grid spacing. They defined cloud disturbances such as the Madden-Julian oscillation (MJO), tropical cyclones (TCs), mid-latitude lows, and fronts and analyzed the statistical features of deep convection associated with these disturbances. The cloud-top height, strength of upward motion, and the environmental field of the convection were different in each type of disturbance. For example, the MJO convection was generated under high convective available potential energy (CAPE) and relatively weak lower convergence, whereas the TC convection was accompanied by low CAPE with strong convergence.
Kajikawa et al. (2016) analyzed the grid-refinement experiments to address the difference in the resolution dependence of the simulated convection by location and environment over the globe. This study extends the work of Miyamoto et al. (2013), which mainly discussed globally averaged convection properties. Kajikawa et al. (2016) showed that convective clouds over the tropics are more dependent on resolution than those in mid-latitudes; on the other hand, no significant difference is found in the resolution dependence of convection properties between land and ocean. Convection within cloud disturbances such as the MJO and TCs also shows large resolution dependency. Consequently, deep convection that is not categorized as cloud disturbances makes a large contribution to the global mean convection properties in Miyamoto et al. (2013). It should be stressed that most deep convection is expressed by multiple grid points in the 0.87 km grid spacing simulation even in tropical cloud disturbances.
Yashiro et al. (2016a) investigated the diurnal cycle of precipitation in this series of grid-refinement experiments and also found that the resolution dependency of its characteristics changed at a grid spacing around 2–3 km. The simulations using 1.7 and 0.87 km grid spacing better reproduced the observed peak time of the diurnal cycle compared to the experiments with coarser grid spacing. This could be a result of the better representation of deep convective cores, as shown by Miyamoto et al. (2013) and Kajikawa et al. (2016).
We showed the resolution dependency of the simulated convection properties and their differences in various environments through a set of grid-refinement experiments. The simulation with sub-kilometer grid spacing revealed new results that differed from previous studies; for example, deep convective cores were expressed by multiple grid points. Higher spatial resolution and more realistic convection simulations are required to better understand cloud disturbances from the meso-scale to the synoptic scale (Prein et al. 2015). Moreover, longer integration at sub-kilometer grid spacing would enable us to investigate the interaction between convection and larger-scale cloud disturbances. Using the Post-K supercomputer, we estimate that we can perform experiments at further finer resolution, such as 220 m, to resolve more detailed structure of clouds.
Asian summer monsoon
The Asian monsoon system is characterized by a seasonal cycle of convective activity and atmospheric circulations (e.g., Murakami and Matsumoto 1994). The variability of the monsoon system’s effects on weather and climate over Asia is largely achieved through local convective activities. This variability is also affected by synoptic scale tropical disturbances. For example, the monsoon onset is often triggered by tropical disturbances or ISO. Recently, global climate modeling has made advances in many aspects of Asian summer monsoon simulations (Sperber et al. 2013; Ogata et al. 2014; Lee and Wang 2014). However, numerical representations of the seasonal transition of the monsoon, such as the monsoon onset, with scale interactions remain challenging. Meanwhile, NICAM has shown an advantage over conventional GCMs in simulating tropical disturbances, such as TCs (Fudeyasu et al. 2008; Taniguchi et al. 2010; Nakano et al. 2015) and MJO (Miura et al. 2007a; Miyakawa et al. 2014). Under Field 3 of SPIRE, we investigated the seasonal transition of the monsoon and anticipated a potential extension of its predictability using a large number of ensemble NICAM simulations, in contrast to previous studies that used simulations of only a single or a few members.
Kajikawa et al. (2015) investigated the process of the Indian summer monsoon onset in 2012 based on ensemble simulations using NICAM with a grid spacing of about 14 km and 38 vertical levels. Moist processes were calculated using the NICAM single-moment water 6-cloud microphysics scheme (NSW6; Tomita, 2008) without any convective parameterization. Sea surface temperature (SST) was forecasted using a slab ocean model to allow diurnal variation and nudged to the persistent SST anomaly at the initial time with an e-folding time (that is, the time in which SST is nudged to specified values) of 7 days. The ensemble simulations were started at 00 UTC each day during the period from 10 May to 10 June 2012, and integrated for 30 days. Using the same set of NICAM ensemble simulations, Yamaura et al. (2013) clarified the role of TCs on the migration of the Baiu front.
The onset of the 2012 Indian summer monsoon was announced on 5 June that year by the Indian Institute of Tropical Meteorology. The NICAM simulations using initial conditions 2 weeks before the onset (15 May) reproduced the onset very well (Fig. 5). Both the observations and simulations included the abrupt onset with northward and eastward migrating tropical disturbances over the Bay of Bengal and the Arabian Sea in late May. Based on a comparison with the Japan Meteorological Agency’s operational Ensemble Prediction System, Kajikawa et al. (2015) pointed out that the better reproducibility of the tropical disturbances seen in the NICAM results extended the predictability of the Asian summer monsoon transition phase.
Yamaura et al. (2013) specifically used the NICAM simulations initialized at 29 and 30 May 2012 to clarify the role of a TC in the migration of the Baiu front. Observations revealed that the Baiu front shifted northward with the TC. A simulation integrated from 29 May reproduced both the TC and the northward migration of the Baiu front, whereas the Baiu front stayed stagnant in another simulation integrated from 30 May. Since the TC in the latter simulation developed farther eastward than the observed TC, Yamaura et al. (2013) suggested that the potential extension of the predictability of the Baiu front migration was due to the realistic reproducibility of TCs.
Seasonal simulations of boreal summer using the global 7 km grid spacing NICAM have captured the Asian summer monsoon as a case study (Oouchi et al. 2009a, 2009b; Satoh et al. 2012b; Kinter et al. 2013). However, these simulations have large biases in precipitation and lower tropospheric wind fields, particularly over the western North Pacific. These biases were partly due to insufficient spin-up of the land surface model, which was upgraded in the experiments conducted during Field 3 of SPIRE.
Another achievement of this project was the first-ever 30-year simulations (see “Atmospheric Model Intercomparison Project-type experiments” section; Kodama et al. 2015). The 30-year simulation successfully simulated the climatology of the annual cycle of the western North Pacific monsoon and its inter-annual variability (see Figures 14 and 15 of Kodama et al. 2015). In Murakami and Matsumoto (1994), the climatological Asian monsoon was defined by differences between the annual maximum and minimum outgoing longwave radiation (OLR) (Fig. 6). The NICAM simulation adequately represents each monsoon sub-system in the observation (e.g., northern Australia-Indonesia monsoon, Southeast Asian monsoon, western North Pacific monsoon); although, the magnitude is stronger by approximately 20–40 W m−2. The NICAM simulation also included a northward extension of the Southeast Asian monsoon and the western North Pacific monsoon which are related to the northward displacement of a low-level westerly and a subtropical high; this in turn led to earlier termination of the Baiu season and to a weaker monsoon trough (Kodama et al. 2015).
Both NICAM ensemble simulations and climate simulation showed realistic behavior of the Asian summer monsoon system including the relationship between TCs and the Baiu front. However, similar to other climate models, NICAM suffers from model biases. In particular, the Asian summer monsoon simulations are severely affected by the warm temperature bias over continents. In addition, the mid-latitude jets were stronger in the simulation than in observations partly because a gravity drag parameterization scheme was turned off in the 14 km grid spacing NICAM simulations. In future, simulations of Asian summer monsoons will need to reduce such biases in order to improve their performance; this is currently being tested through development of the land surface model and the introduction of the gravity wave drag scheme.
NICAM simulations with atmosphere and ocean coupling are also being investigated, because realistic responses of convection to basin-scale ocean surface conditions are strongly related to ocean-atmosphere coupling (Wang et al. 2005). In addition, process-oriented diagnosis in comparison with other climate models (Sperber et al. 2013; Ogata et al. 2014; Kusunoki and Arakawa 2015; Zou and Zhou 2015) is planned to understand the effects of basic-state biases on monsoon systems and to make the necessary improvements. The multi-scale processes of monsoons and the Baiu front under global warming conditions in cloud-system resolving simulations are another ongoing research topic (e.g,. Krishnamurthy et al. 2014).
The MJO, which is often described as an eastward propagating atmospheric pulse with a typical lifespan of 30–60 days, is one of the dominant disturbances in the equatorial atmosphere at the intra-seasonal timescale (Madden and Julian 1972). It is viewed as a key source of predictability for the extended-range forecasts that target the time range of 10–30 days. More than 40 years since its foundation, the majority of GCMs still struggle to produce a satisfactory MJO (Hung et al. 2013). NICAM had enjoyed success in producing a particular MJO event, maintaining a highly accurate eastward propagation speed along with realistic precipitation and zonal wind structures (Miura et al. 2007a). In Field 3 of SPIRE, we utilized the increased computational resource of the K computer to conduct ensemble simulations of MJOs using NICAM to statistically assess the model’s ability to predict MJOs (Miyakawa et al. 2014). Applying a widely used evaluation method proposed by Gottschalck et al. (2010), NICAM was diagnosed to have a prediction limit of 26–28 days. The precipitation patterns associated with MJO phases also compared well with observations.
The following procedure based on the real-time multivariate MJO (RMM) indices (Wheeler and Hendon 2004) was applied to the boreal winter (October–March) MJOs during an evaluation period (2003–2012) to assign initial dates for the experiment. We identified target MJO cases when a succession of observational RMM plots move counterclockwise through phase 2 to phase 5 (corresponding to the MJO convective envelope traveling from the western Indian Ocean to the eastern Indonesian Archipelago) without retreating more than one phase and had an average amplitude greater than 1 over the span between phases 2 and 5. For each of the MJO cases extracted, the first dates on which the RMM plot falls in phases 8, 1, and 2 are assigned as initial dates.For the 19 MJO cases identified, 54 initial dates were assigned.We used the European Centre for Medium-Range Weather Forecasts interim reanalysis (ERA-Interim) dataset (Dee et al. 2011) for initial conditions of the atmosphere and ocean. The resolution and the physical schemes of NICAM were the same as those described in the “Asian summer monsoon” section except for modified cloud microphysics parameters that control the formation and fall speeds of precipitation particles and minor updates of land-process/boundary layer scheme parameters. A mixed-layer ocean model was applied, and its SST was nudged to externally provided SST at an e-folding time of 7 days. The external SST was defined as the sum of the time-varying mean annual cycle and constant anomalous component. The difference from the mean annual cycle was averaged over the week before the initial date to derive the anomalous component. Thus, no information observed after the initial date was used in the simulations. Further details are found in Miyakawa et al. (2014).
NICAM maintained a valid (>0.6) MJO skill score for 26–28 days depending on the initial MJO phase, and for 27 days when all 54 cases were included. Composites of observed and simulated precipitation anomalies for MJO phases 3, 5, and 7 are shown in Fig. 7. The horizontal structure of the simulated precipitation anomalies closely resembles the observations, even for phase 7, for which the average lead time was 28 days. However, precipitation anomalies were overestimated over the central Pacific in phase 5. Such a bias is occasionally found in 14 km grid spacing NICAM, perhaps due to under-resolved local vertical moisture transport, which could be a common problem for models that explicitly calculate cloud systems without cumulus parameterization. The simulation series includes an MJO case that occurred in November–December 2011 during the Cooperative Indian Ocean Experiment on Intraseasonal Variability in Year 2011 and Dynamics of the MJO field campaign (CINDY2011/DYNAMO; Yoneyama et al. 2013). Figure 8 shows the observed and simulated convective signals that clearly propagate at a very similar eastward speed over the tropical Indian to the western Pacific Ocean (10°S–10°N, 40°E–160°W). Time-height sections of observed and simulated zonal wind over Gan Island (not shown) further reveal that the model captures the depth and timing of the intrusion of the westerly winds as well as the deepening of moisture with time, both of which are key aspects of MJO propagation. Despite a lead time of nearly 4 weeks, the model also appears to capture the occurrence of the next MJO-like signal in mid-December. The successful prediction of the event provides the opportunity to compare a global non-hydrostatic model simulation with a major in situ observation for the same MJO event for the first time.
Although NICAM appears to be one of the unique models that can produce MJO events with high accuracy, we believe that the model still does not live up to its potential in terms of MJO prediction. The limit in available resources has prevented us from applying ensemble simulations of significant size for each event. However, a next-generation supercomputer designed to be roughly 20–100 times more powerful than the K computer should be able to accommodate such simulations. The initialization process of this experiment was a simple interpolation from the ERA-Interim dataset, which allows significant initial shock, but a new data assimilation system specifically designed for NICAM is being developed. The use of a simple mixed-layer ocean model limits the model skill in situations in which dynamical features of the ocean drastically affect the SST conditions. A fully coupled version of NICAM with a three-dimensional dynamical ocean is now available and currently being tested (NICAM-COCO coupled model; Satoh et al. 2014).
Boreal summer intra-seasonal oscillation and tropical cyclogenesis
Tropical cyclones cause huge socio-economic impacts, so more accurate TC forecasts are demanded. Because forecasting tropical cyclogenesis (TC genesis) is still a challenge in numerical weather forecasting, the predictability of TCs at sub-seasonal to seasonal time scales (2 weeks to 2 months) is promoted worldwide (Vitart et al. 2012, 2017). TC genesis is affected by large-scale environmental conditions, such as low-level cyclonic vorticity, small vertical wind shear, the convective instability, and mid-level humidity (Gray 1975, 1979). The boreal summer intra-seasonal oscillation (BSISO; Wang and Rui 1990; Wang and Xie 1997) modulates the large-scale environment and TC activity. For example, Nakazawa (2006) pointed out that the active phase of BSISO in the 2004 summer season successively generated TCs, and persistent steering flow led to a record-breaking number of TC landfalls in Japan. Therefore, the accurate prediction of BSISO and the modulation of large-scale environmental conditions associated with BSISO are believed to lead to accurate forecasting of TC genesis.
Studies using NICAM show that the model is a promising tool for accurate BSISO and TC genesis forecasting. For example, Oouchi et al. (2009a) demonstrated that NICAM successfully simulated modulations of BSISO and that TC genesis was captured 3 weeks in advance. However, the authors examined only a limited number of TC genesis cases using a single run because of the limitations in computational resources. The K computer enables us to examine the predictability of the BSISO and TC genesis using the ensemble approach. Here, we examine the predictability of eight cases of TC genesis that occurred successively in August 2004.
Nakano et al. (2015) performed 30-day simulations for each day of August 2004; that is, a total of 31 ensemble 30-day simulations were performed. The initial conditions of the atmosphere and SST were derived from the ERA-Interim dataset (Dee et al. 2011). The resolution and the physical schemes of NICAM were the same as those described in the “Asian summer monsoon” section. This setting is “forecast-mode” because the model never knows the observed SST during the time integration.
A candidate vortex that corresponds to an observed TC should pass within 10° of the location of the observed TC genesis within 1 day of the observed TC genesis time. The candidate vortex should meet TC criteria (e.g., surface wind speed >17.5 m s−1) within 5 days of the observed TC genesis time. Detailed descriptions of the experimental setup and method of TC detection are provided in Nakano et al. (2015).
Table 1 summarizes the results of the TC genesis forecast. The model captured six out of the eight cases of TC genesis 1 week in advance with a low missing rate (all were above 70%). Moreover, four cases of TC genesis in late August were captured 2 weeks in advance with high probability (all were above 40% and 3 of 4 were above 50%). The genesis of TC Songda was well captured 2 weeks in advance, though not 3 weeks in advance. Note that two TC genesis events (Malou and Malakas) were not captured, probably because they were very weak (minimum central pressure >990 hPa) and had short lifespan (less than 3 days).
Figure 9 shows the zonal wind at 850 hPa and the OLR, which is related to convective activities. In the analyzed field, the shear line between easterly wind in the tropics and monsoonal westerly wind extended eastward, accompanied by active convection in mid-August. This eastward extension was maintained throughout late August. TC Songda was generated on the shear line. The experiments initialized between August 5 and 11 (3 weeks before Songda’s genesis) captured the eastward extension of the shear line in mid-August but it retreated in late August. The experiments initialized between August 12 and 18 (2 weeks before Songda’s genesis) captured the eastward extension of the shear line in late August. Therefore, we conclude that the eastward extension of the shear line is closely related to Songda’s genesis, and if models reproduce the longitudinal evolution of the shear line, the associated TC genesis will be simulated effectively.
Once we have achieved more reliable predictability of TC genesis with a longer lead time, our next targets are the probabilities of TC tracks, intensities, and landfall. Such information could be calculated from the large samples of ensemble simulations. Since the NICAM-LETKF system (Terasaki et al. 2015) is an ensemble-based data assimilation system, it can be used to make the large number of initial atmospheric fields fit to the model’s dynamics and physics. It is expected that the probability forecast of TC tracks and landfall probability would be affected by the mid-latitude atmospheric conditions including the structure and evolution of the jet stream and the subtropical high. Given that the mid-latitude jet stream and the subtropical high are affected by the weather systems in the tropics such as MJO (Kawamura et al. 1996; Moon et al. 2013; Molinari and Vollaro 2012), we should further examine the relationship between mid-latitude model biases and those in the tropics to improve probability forecasting of TC tracks. For forecasts of TC intensity, current atmospheric models have typical biases: they are not able to reproduce rapid intensification, and TCs in the models continuously intensify even in the mid-latitude, where TCs begin to weaken, then resulting in overintensification. A higher resolution model is needed to improve the representation of the structure of the TC’s inner core, which results in accurate simulation of rapid intensification (Rogers et al. 2013; Wang and Wang, 2014). Interestingly, a recent study shows that high-resolution global models reduce forecast error in TC intensity and tracking forecasts (Nakano et al. 2017). The atmosphere-ocean interacting processes should also be incorporated in the numerical model to avoid overintensification (Wada et al. 2014; Zarzycki 2016). Simulations of TC intensities are sensitive to details of physical schemes such as the boundary layer scheme and the cloud microphysics scheme (Kanada et al. 2012). More improvement and development of NICAM together with evaluations using available observational data will be necessary to pursue the aforementioned future directions in TC predictability studies.
Atmospheric Model Intercomparison Project-type experiments
Atmospheric Model Intercomparison Project (AMIP) is a standard experimental protocol for atmospheric GCMs to test models’ climatology with multi-decadal simulations. Such climate simulations are now achievable using a high-resolution global non-hydrostatic model, and a new era of climate-system research has begun. For example, the simulated intensities of TCs strongly depend on horizontal resolution (Camargo 2013) and convection scheme (Murakami et al. 2012). Thus, by taking advantage of high-resolution global non-hydrostatic simulations without deep convection schemes, the climatology of TCs is expected to be simulated more informatively, thereby facilitating further investigations of the probability and geographical distributions of TC intensities and structures and their projected changes under a warmer climate (Oouchi et al. 2006; Murakami et al. 2012; Knutson et al. 2015). The global non-hydrostatic climate model is also a powerful tool for simultaneously investigating phenomena at various spatiotemporal scales of atmospheric disturbances, such as planetary-scale atmospheric general circulations, MJO, tropical waves, diurnal meso-scale convective events, severe rainfall, and atmospheric gravity waves and their interactions (Holt et al. 2016). In addition, better representation of clouds, convection, and circulations is key to reducing uncertainties of climate sensitivity (Bony et al. 2015; Stevens and Bony 2013).
Motivated by the aforementioned demands for AMIP-type experiments with high-resolution global models, we performed present (Kodama et al. 2015) and future (Satoh et al. 2015) climate simulations using the 14 km grid spacing NICAM. The model settings were the same as those described in the “Asian summer monsoon” section. Specifically, we should note that instead of following the strict AMIP protocol, the slab ocean model and nudging technique were employed instead of imposing a fixed SST condition to obtain better performance of the distribution of tropical precipitation, as shown in Kodama et al. (2015). The simulations were performed to reproduce the climates of 1979–2008 (Kodama et al. 2015) and to project the climates of 2074–2100 (Satoh et al. 2015).
Kodama et al. (2015) reported the simulated climatology of basic state, TCs, MJO, Asian monsoon, diurnal precipitation cycle, and quasi-biennial oscillation (QBO). In-depth analysis of the MJO and mean and intense precipitation around Japan were presented in Kikuchi et al. (2016) and Fujita et al. (2017, personal communication), respectively. Fukutomi et al. (2015) evaluated the activity and structure of the simulated tropical synoptic scale waves over the western Pacific.
Here, we extend the analysis of Fukutomi et al. (2015) to the various types of convectively coupled equatorial waves (Kiladis et al. 2009), which govern the tropical day-to-day weather. Spatiotemporal spectral and filter analyses were applied to OLR, precipitation, and wind fields to diagnose OLR variances contributed from equatorial Kelvin waves, equatorial Rossby waves, mixed Rossby gravity waves, and tropical depression waves. The National Oceanic and Atmospheric Administration (NOAA) OLR (Liebmann and Smith 1996) and the Japanese 55-year Reanalysis (JRA-55; Kobayashi et al. 2015b) datasets were used to evaluate the results. Here, the equatorial wave-filtered OLR variances are plotted (Fig. 10). Geographical distributions of tropical waves over the Indian and Pacific Oceans were qualitatively simulated, though their overall activities tended to be weaker than the observed ones (Fig. 10). Significant biases were found over Central and South America, the Atlantic, and Africa in all the wave components. Kelvin waves were very weakly simulated, particularly over the Atlantic, whereas simulated equatorial Rossby waves were stronger over the Amazonian region and weaker over the Atlantic than those of the observation. Distribution of the simulated mixed Rossby gravity waves was biased over Central and South America and the Atlantic. Tropical depression waves were weaker over Central and South America and the western Atlantic, whereas the African easterly waves in the simulation were more active over the eastern Atlantic compared with those in the observation. The reasoning behind the model biases in convectively coupled equatorial waves will be further investigated from the perspective of reproducibility of the seasonal scale background fields.
Future projected climate simulations were analyzed by Satoh et al. (2015), in which the response of TCs to global warming was investigated using present and future climate simulation data. Yamada et al. (2017, personal communication) further investigated structural changes in TCs. In addition to TCs, how the precipitation around Japan (Fujita et al., 2017, personal communication) and the precipitation associated with extratropical cyclones respond to global warming (Kodama et al. 2017, personal communication) has been investigated. Before the K computer was available, projection studies using NICAM were limited to short time-slice simulations such as one year or season (Yamada et al. 2010; Satoh et al. 2012a; Tsushima et al. 2014; Noda et al. 2014, 2015), and the insufficient integration period made it difficult to achieve statistical significance of the global warming response. It is now possible to confirm the results derived from shorter-integration NICAM data. For example, Chen et al. (2016) performed seasonal scale sensitivity experiments and showed a large longwave cloud radiative feedback, irrespective of the cloud microphysics schemes that were employed. This result was further supported by an additional analysis of the inter-annual variability found in the NICAM AMIP-type data. Figure 11 shows another example depicting the distribution of high-cloud size that was evaluated using the method described by Noda et al. (2014). Qualitatively, the size distributions under the present and the future conditions show a result similar to those evaluated in a 1-year NICAM simulation data in Noda et al. (2014). Responding to global warming, the number of smaller clouds (<40 km radius) notably increases, while changes to the number of larger clouds (≥40 km) are much smaller and not significant. We found that the difference in the smaller clouds is statistically significant. The result implies less organization of tropical cloud systems in the warmer world. As analyzed by Chen et al. (2016), NICAM shows larger positive high cloud feedback than the CMIP5 models due to the increase of high cloud cover and more positive longwave feedback.
The NICAM AMIP-type simulations described above are the first step toward global cloud-resolving climate simulation. Our next plan, the High Resolution Model Intercomparison Project (HighResMIP), is briefly described in the “Future perspectives” section.
Projection of tropical cyclones
A primary energy source of TCs is latent heat supplied from the warm tropical ocean (Emanuel 2003). The change in TC activities in a warmer climate is one of the important issues of climate change (Walsh et al. 2016). Broccoli and Manabe (1990) used a GCM to project the responses of TCs to global warming for the first time. They used a horizontal grid spacing of a few hundred kilometers, which was too coarse to resolve TC structures such as a warm core. However, their simulation showed the global distribution of simulated TCs in general agreement with the observation. They indicated that climate models have the potential to predict future changes in TCs and advocated reducing the horizontal grid interval in order to realistically reproduce TC structures.
Many GCMs have used finer resolutions than that of Broccoli and Manabe (1990) to assess the response of TCs to global warming, indicating some consensus in the change of TC statistics in a warmer world: decrease in TC genesis frequency, increase in intense TCs, mean lifetime maximum intensity, and rainfall around TCs (Knutson et al. 2010; Christensen et al. 2013; Walsh et al. 2016). However, Emanuel (2013) showed an increase in global annual frequency of TCs under warmer climate conditions using a downscaling method in which incipient vortices are embedded into large-scale climate conditions projected by CMIP5 models. To achieve a better understanding of the relationships between TC genesis and climate change, the Hurricane Working Group (HWG) established by the US Climate and Ocean: Variability, Predictability and Change (CLIVAR) group coordinated common experiments (Walsh et al. 2015). Although the downscaling methodology of Emanuel (2013) projected an increase in TC genesis under the warmer climate condition projected by the HWG models, there were no models that generated a substantial increase in global TC frequency. The reduction of global TC numbers is associated with a decrease in upward mass flux (Sugi et al. 2002, 2012), and an increase in saturation deficit of the mid-troposphere (Emanuel et al. 2008). The mechanism underpinning the decrease in TC numbers, however, remains controversial.
Due to recent advances in supercomputing and GCMs, fine-resolution models with grid spacing less than 25 km have been used to investigate future change in TC activities (Murakami et al. 2012; Manganello et al. 2014; Knutson et al. 2015; Roberts et al. 2015; Wehner et al. 2015). These models reproduced TC structure more realistically. The advent of the K computer enabled researchers to perform long-term simulations using a high-resolution global non-hydrostatic model (Kodama et al. 2015; Satoh et al. 2015).
In this section, future changes in TC activities are investigated using the outputs of the aforementioned NICAM AMIP-type simulation (referred to as the present day (PD) simulation; Kodama et al. 2015) and a warmer climate simulation (referred to as the global warming (GW) simulation; Satoh et al. 2015). Kodama et al. (2015) documented the model settings and experimental design of the PD simulation. The model settings of the GW simulation were identical to those of the PD simulation, and the experimental design is described in Satoh et al. (2015).
Figure 12 shows geographical distributions of TC genesis density for the PD and GW simulations and the future change (GW \( - \) PD). The PD simulation reproduces the observed distribution of TC genesis fairly well; although, the model has clear bias particularly over the north Atlantic for less TC genesis (See Fig. 10 of Kodama et al. 2015). The GW simulation showed a salient decrease in TC genesis over the eastern North Pacific (Fig. 12c). Satoh et al. (2015) documented the annual global TC number of 104.0 for the PD simulation and 81.8 for the GW simulation. The decrement in the TC number due to global warming is 21.3%, which is within the range of previous studies (Knutson et al. 2010; Christensen et al. 2013). Figure 12c shows that TCs are mainly decreased over the north eastern Pacific.
As the mechanism of the decrement was not completely understood, Satoh et al. (2015) used outputs of the PD and GW simulations to propose a new concept based on the relationship between convective mass flux and TCs. They extracted a convective mass flux associated with TCs and defined its contribution rate to convective mass flux in the tropics (R) as follows:
where M T is convective mass flux associated with TCs (defined as integration over the area with a radius smaller than 500 km from the center of TCs), T int is the integration period, and M is the total convective mass flux over the tropical domain (30°N–30°S). Since M T = NTm T , where N is the number of global TCs, T is the average lifetime of TCs, and m T is the average convective mass flux per single TC, we can rewrite this as (Eq. 15 of Satoh et al. 2015)
Satoh et al. (2015) found that the contribution of TC (R) in GW is almost unchanged or less than that in PD. The convective mass flux in the tropics (M) generally decreases in GW (Vecchi and Soden 2007), and our results confirm this change. In the analysis of the TC convective mass flux, Satoh et al. (2015) also found that the convective mass flux associated with TCs and its fractional area become greater as TCs become more intense (Fig. 13a and b, respectively). They also found that the TC convective mass flux and its frictional area in GW are greater than those in PD for the same intensity of TC (Fig. 13a and b, respectively). These results are contrasted by Fig. 13c, where the area-averaged convective mass flux, or vertical velocity, does not show a robust change with warming. Intense TCs increase under a warmer climate, leading to greater convective mass flux associated with TCs. In these simulations, the incidence of intense TCs that develop to less than 945 hPa increases by 38.6% due to global warming (Fig. 14). Thus, Satoh et al. (2015) showed that m T increases as warming. Assuming that R is unchanged between PD and GW, this constraint implies that TC number decreases in order to intensify TCs under a warmer climate.
Walsh et al. (2016) highlighted the importance of the following topics for future research on TCs and climate change: detection methods as a source of uncertainty, lack of climate theory of TC formation, confidence in detection and attribution of observed changes in TCs to date, impact of uncertainty in projections of large-scale climate fields, and correct simulations of TC frequency response in high-resolution climate models. It is expected that high-resolution models will be useful tools for investigating these topics. Thanks to the K computer, we completed the first-ever climate simulation with a non-hydrostatic GCM using 14 km grid spacing (Kodama et al. 2015; Satoh et al. 2015). The outputs from this work will contribute important new findings on TC climatology.
Higher-resolution models have simulated more realistic TC climatology (Walsh et al. 2016). However, TC distributions simulated in high-resolution models still show inconsistency with observed TC activities such as fewer TCs over the north Atlantic (Kodama et al. 2015; Murakami et al. 2015; Roberts et al. 2015), more TCs over the western North Pacific (Manganello et al. 2012; Murakami et al. 2015) and the eastern North Pacific (Kodama et al. 2015; Roberts et al. 2015), weak maximum wind speed compared with low central sea-level pressure (Roberts et al. 2015), and an overestimation of mean TC intensities at higher latitudes (Murakami et al. 2012; Kodama et al. 2015). Reducing these biases will enhance confidence in GCM simulations of future change. For example, Lloyd and Vecchi (2011) investigated the observation that the passage of TCs causes SST cooling, indicating that this oceanic feedback suppresses intensification of TCs. Atmosphere-ocean coupled models will improve the overestimation of TC intensities at higher latitudes.
In addition, high-resolution models simulate TC structures more realistically. An increase in the convective mass flux associated with TCs due to global warming is not caused by an enhancement of updraft velocities (Fig. 13c), but by an increase in the area of the updraft region around TCs (Fig. 13b). This suggests that global warming influences the dynamical structure of TCs. The scale of damage caused by a disaster associated with TCs is measured by TC gale force wind radii as well as its frequency, intensity, and track. Recent studies have focused on future changes in the size of TCs (Kanada et al. 2013; Manganello et al. 2014; Knutson et al. 2015). High-resolution global non-hydrostatic models should also make it possible to statistically evaluate future changes of TC structure.
Ohno and Satoh (2015) revealed that the flow from the lower stratosphere near the TC eyewall plays a key role in constituting the upper-level warm core which develops TC intensity prior to reaching the mature stage. To increase confidence in this theory, it must be compared with an authentic observation with high temporal and spatial resolution. Today, geostationary satellites can observe atmospheric wind vectors associated with TCs in the upper-troposphere with high temporal and spatial resolution (Bessho et al. 2016). Outputs of high-resolution models should be evaluated with such new satellite data, promoting our understanding of TCs.
Post-K supercomputer project
Among the outcomes of Field 3 of SPIRE summarized in this review, the series of studies using the sub-kilometer global atmospheric simulation (Miyamoto et al. 2013, 2015; Kajikawa et al. 2016) are typical examples of “capability computing” using the K computer; that is, a single complex problem solved in the shortest time possible using the maximum computing power available. Although this kind of simulation does not always lead to the maximum output or outcome, the knowledge gained has great potential to bolster next-generation research. In much larger computer system such as the Post-K supercomputer, problems of this size will be easily handled with various configurations. We can expect to get much more information from these supercomputers in terms of “capacity computing,” or the solution of smaller or less complex problems using more efficient computing power.
A research strategy similar to that mentioned above has already been adopted. In the era of the Earth Simulator, Miura et al. (2007a) take a capability computing approach to demonstrate the ability to represent the MJO with 3.5 and 7 km grid spacing using NICAM. Using a large number of ensemble simulations, Miyakawa et al. (2014) statistically confirmed the MJO predictability suggested by Miura et al. (2007a) and suggested that high-resolution global non-hydrostatic models like NICAM generally have better skill in MJO predictability than GCMs. Thus, combining the two strategies of capability computing and capacity computing may be an effective approach for future research using the Post-K supercomputer.
The project to develop the next-generation flagship supercomputer is now underway. The Post-K supercomputer is designed by using homogeneous CPU architecture. The CPU will be a many-core processor, and accelerators are not included in the system. The system will consist of a six-dimensional mesh/torus interconnected network (Tofu; Ajima et al. 2009) and a three-level hierarchical storage system. The co-design is based on collaboration between computational scientists and computer scientists, because a system constructed using commodity computers along current computer trends cannot satisfy the socio-scientific demands of all research areas, including earth sciences. NICAM has been selected as a co-design application from the viewpoint of a structured stencil calculation. In addition, LETKF has also been selected in this co-design project as an application requiring the dense-matrix calculation. NICAM-LETKF (Terasaki et al. 2015) is one of the most important target applications for the development of the new computer system. The current target problem for the NICAM-LETKF is as follows: in the data assimilation cycle, 1000 ensembles by NICAM with 3.5 km grid spacing and 100 vertical levels are conducted every 6 h. The prediction using 440 m horizontal grid spacing is integrated for several months. In terms of the computational performance, the workflow between the simulation code and the data assimilation system, including I/O and communication in a massively parallel supercomputer becomes a serious problem. A framework proposed by Yashiro et al. (2016c), which was designed to reduce data movement and to utilize parallel I/O, showed a reduction in the workflow’s total elapse time. Based on the scientific outcomes summarized in the present review, the big data assimilation by NICAM-LETKF using tremendous volumes of observational data, such as satellite data, is expected to further contribute to the improvement of mid-range predictions around the globe.
Post-K supercomputer studies with NICAM
Following the SPIRE project, the FLAGSHIP2020 project began in 2014 to examine possible maximum use of the Post-K supercomputer. Among the research subjects defined for the weather and climate simulation studies, seamless simulations using NICAM have been proposed. Seamless means beyond a week to seasonal prediction, especially for the predictability of TC statistics such as TC genesis, tracks, and intensities. Furthermore, seamless studies cover a year to decadal scales, and even to the centennial scale, as aimed for by the HighResMIP study, which is introduced below. Successful extended-range forecasts of MJO and BSISO (Miyakawa et al. 2014; Nakano et al. 2015) suggest greater predictability of TCs beyond a week to a few weeks. Through ensemble simulations, many aspects of TCs such as TC genesis, tracks, and intensities will be obtained. In addition, together with the development of the high-resolution atmosphere and ocean coupled model (NICAM-COCO), the seasonal scale predictability of ISO and associated TC characteristics will be explored. For further multi-year simulation, we will examine the potential predictability of the near future from a year to a decadal scale, particularly focusing on TC statistics (as described in the “Projection of tropical cyclones” section). These longer time-scale simulations have also been examined in the “Atmospheric Model Intercomparison Project-type experiments” section and are further pursued in the next subsection, “High Resolution Model Intercomparison Project.”
High Resolution Model Intercomparison Project
As a part of Coupled Model Intercomparison Project phase 6 (CMIP6), the High Resolution Model Intercomparison Project (HighResMIP; Haarsma et al. 2016) is now under way. In this project, climate modeling with a horizontal resolution of 25–50 km will be performed for more than half a century. The aim is that such fine-resolution climate model data will be available for everyone within a few years as CMIP6 continues. Following the HighResMIP protocol, we plan to perform 56 and 28 km grid spacing NICAM for 1950–2014 and 14 km grid spacing NICAM for ten years. Currently, a series of sensitivity experiments are being performed to improve climatology in the simulation. Our aim in participating in the HighResMIP is to investigate multi-scale phenomena including tropical and extratropical cyclones, MJO, tropical waves, convection, and clouds in a seamless manner (see Section 7.2 of Haarsma et al. 2016). Process-based assessment using satellite datasets will also be key to evaluating and constraining the high-resolution climate model.
NICAM-related studies using the K computer under Field 3 of SPIRE have produced a wealth of valuable weather and climate modeling research and present clear challenges that may be solved with the Post-K supercomputer. Key achievements of these studies include high-resolution sub-kilometer global simulations, ensemble simulations for MJO and BSISO, and simulations of TC genesis. The longer multi-decadal simulation by NICAM was conducted for the first time with the global non-hydrostatic model. The climatology of the 14 km grid spacing NICAM shows comparable results with other climate models; although, biases similar to those of other models exist. The results suggest that, even though it has been arguable whether deep convective parameterization is required for the 14 km grid spacing model, the simulation without the parameterization reproduces climatology consistent with observation. Thanks to the K computer, we obtained considerable evidence of NICAM’s usability for weather and climate simulations. Further research using the Post-K supercomputer in the next decade is expected to contribute even more crucial knowledge to this exciting research field.
Boreal summer intra-seasonal oscillation
Coupled Model Intercomparison Project phase 6
European Centre for Medium-Range Weather Forecasts interim reanalysis
Future LAtency core-based General-purpose Supercomputer with HIgh Productivity
General circulation model
The global warming simulation
High Resolution Model Intercomparison Project
Japan Agency for Marine-Earth Science and Technology
Local ensemble transform Kalman filter
Northern Australia-Indonesia monsoon
Non-hydrostatic icosahedral amospheric model
National Oceanic and Atmospheric Administration
NICAM single-moment water 6-cloud microphysics scheme
Outgoing longwave radiation
The present day simulation
Real-time multivariate MJO
Southeast Asian monsoon
Strategic Programs for Innovative Research
Sea surface temperature
Tropical Rainfall Measuring Mission
Western North Pacific monsoon
Ajima Y, Sumimoto S, Shimizu T (2009) Tofu: a 6D mesh/torus interconnect for exascale computers. Computer 42:36–40. doi:10.1109/MC.2009.370
Amdahl GM (1967) Validity of the single processor approach to achieving large scale computing capabilities. In: AFIPS ’67 (Spring) Proceedings of the April 18-20, 1967, spring joint computer conference. ACM, New York, pp 483–485. doi:10.1145/1465482.1465560
Ando K, Hyodo M, Baba T, Hori T, Kato T, Watanabe M, Ichikawa S, Kitahara H, Uehara H, Inoue H (2016) Parallel-algorithm extension for tsunami and earthquake-cycle simulators for massively parallel execution on the K computer. Int J High Perform Comput Appl 30:454–468. doi:10.1177/1094342016636670
Arakawa A, Schubert WH (1974) Interaction of a cumulus cloud ensemble with large-scale environment, Part 1. J Atmos Sci 31:674–701
Arakawa T, Inoue T, Sato M (2014) Performance evaluation and case study of a coupling software ppOpen-MATH/MP. Procedia Comp Sci 29:924–935. doi:10.1016/j.procs.2014.05.083
Bessho K, Date K, Hayashi M, Ikeda A, Imai T, Inoue H, Kumagai Y, Miyakawa T, Murata H, Ohno T, Okuyama A, Oyama R, Sasaki Y, Shimazu Y, Shimoji K, Sumida Y, Suzuki M, Taniguchi H, Tsuchiyama H, Uesawa D, Yokota H, Yoshida R (2016) An introduction to Himawari-8/9—Japan’s new-generation geostationary meteorological satellites. J Meteorol Soc Jpn 94:151–183. doi:10.2151/jmsj.2016-009
Bony S, Stevens B, Frierson DMW, Jakob C, Kageyama M, Pincus R, Shepherd TG, Sherwood SC, Siebesma AP, Sobel AH, Watanabe M, Webb MJ (2015) Clouds, circulation and climate sensitivity. Nat Geosci 8:261–268. doi:10.1038/ngeo2398
Broccoli AJ, Manabe S (1990) Can existing climate models be used to study anthropogenic changes in tropical cyclone climate? Geophys Res Lett 17:1917–1920. doi:10.1029/GL017i011p01917
Bryan GH, Wyngaard JC, Fritsch JM (2003) Resolution requirements for the simulation of deep moist convection. Mon Weather Rev 131:2394–2416. doi:10.1175/1520-0493(2003)131<2394:RRFTSO>2.0.CO;2
Camargo SJ (2013) Global and regional aspects of tropical cyclone activity in the CMIP5 models. J Clim 26:9880–9902. doi:10.1175/JCLI-D-12-00549.1
Chen G, Zhu X, Sha W, Iwasaki T, Seko H, Saito K, Iwai H, Ishii S (2015a) Toward improved forecasts of sea-breeze horizontal convective rolls at super high resolutions. Part I: configuration and verification of a down-scaling simulation system (DS3). Mon Weather Rev 143:1849–1872. doi:10.1175/MWR-D-14-00212.1
Chen G, Zhu X, Sha W, Iwasaki T, Seko H, Saito K, Iwai H, Ishii S (2015b) Toward improved forecasts of sea-breeze horizontal convective rolls at super high resolutions. Part II: the impacts of land use and buildings. Mon Weather Rev 143:1873–1894. doi:10.1175/MWR-D-14-00230.1
Chen Y-W, Seiki T, Kodama C, Satoh M, Noda AT, Yamada Y (2016) High cloud responses to global warming simulated by two different cloud microphysics schemes implemented in the Nonhydrostatic Icosahedral Atmospheric Model (NICAM). J Clim 29:5949–5964. doi:10.1175/JCLI-D-15-0668.1
Christensen JH, Krishna Kumar K, Aldrian E, An S-I, Cavalcanti IFA, de Castro M, Dong W, Goswami P, Hall A, Kanyanga JK, Kitoh A, Kossin J, Lau N-C, Renwick J, Stephenson DB, Xie S-P, Zhou T (2013) Climate phenomena and their relevance for future regional climate change. In: Stocker TF, Qin D, Plattner G-K, Tignor M, Allen SK, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge
Dee DP, Uppala SM, Simmons AJ, Berrisford P, Poli P, Kobayashi S, Andrae U, Balmaseda MA, Balsamo G, Bauer P, Bechtold P, Beljaars ACM, van de Berg L, Bidlot J, Bormann N, Delsol C, Dragani R, Fuentes M, Geer AJ, Haimberger L, Healy SB, Hersbach H, Hólm EV, Isaksen L, Kållberg PK, Köhler M, Matricardi M, McNally AP, Monge-Sanz BM, Morcrette J-J et al (2011) The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q J R Meteorol Soc 137:553–597. doi:10.1002/qj.828
Duc L, Kuroda T, Saito K, Fujita T (2015) Ensemble Kalman Filter data assimilation and storm surge experiments of tropical cyclone Nargis. Tellus A 67:25941. doi:10.3402/tellusa.v67.25941
Emanuel KA (2003) Tropical cyclones. Annu Rev Earth Planet Sci 31:75–104. doi:10.1146/annurev.earth.31.100901.141259
Emanuel KA (2013) Downscaling CMIP5 climate models shows increased tropical cyclone activity over the 21st century. Proc Natl Acad Sci U S A 110:12219–12224. doi:10.1073/pnas.1301293110
Emanuel KA, Sundararajan R, Williams H (2008) Hurricanes and global warming: results from downscaling IPCC AR4 simulations. Bull Am Meteorol Soc 89:347–367. doi:10.1175/BAMS-89-3-347
Fudeyasu H, Wang YQ, Satoh M, Nasuno T, Miura H, Yanase W (2008) Global cloud-system-resolving model NICAM successfully simulated the lifecycles of two real tropical cyclones. Geophys Res Lett 35:L22808. doi:10.1029/2008gl036003
Fukutomi Y, Kodama C, Yamada Y, Noda AT, Satoh M (2015) Tropical synoptic scale wave disturbances over the western Pacific simulated by a global cloud-system resolving model. Theor Appl Climatol 124:737–755. doi:10.1007/s00704-015-1456-4
Gottschalck J, Wheeler M, Weickmann K, Vitart F, Savage N, Lin H, Hendon H, Waliser D, Sperber K, Nakagawa M, Prestrelo C, Flatau M, Higgins W (2010) A framework for assessing operational model MJO forecasts: a project of the CLIVAR Madden-Julian Oscillation working group. Bull Am Meteorol Soc 91:1247–1258
Graph 500 (2014) The Graph 500 List., http://www.graph500.org. Accessed 10 Mar 2017
Gray WM (1975) Tropical cyclone genesis. Atmospheric Science Paper, No. 234. Colorado State University, Fort Collins
Gray WM (1979) Hurricanes: their formation, structure and likely role in the tropical circulation. In: Shaw DB (ed) Meteorology over the tropical oceans. Royal Meteorological Society, Reading, pp 155–218
Haarsma RH, Roberts MJ, Vidale PL, Senior CA, Bellucci A, Bao Q, Chang P, Corti S, Fučkar NS, Guemas V, von Hardenberg J, Hazeleger W, Kodama C, Koenigk T, Leung LR, Lu J, Luo J-J, Mao J, Mizielinski MS, Mizuta R, Nobre P, Satoh M, Scoccimarro E, Semmler T, Small J, von Storch J-S (2016) High Resolution Model Intercomparison Project (HighResMIP v1.0) for CMIP6. Geosci Model Dev 9:4185–4208. doi:10.5194/gmd-9-4185-2016
Hasegawa Y, Iwata JI, Tsuji M, Takahashi D, Oshiyama A, Minami K, Boku T, Shoji F, Uno A, Kurokawa M, Inoue H, Miyoshi I, Yokokawa M (2011) First-principles calculations of electron states of a silicon nanowire with 100,000 atoms on the K computer. In: Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis. ACM Press, New York. doi:10.1145/2063384.2063386
Holt LA, Alexander MJ, Coy L, Molod A, Putman W, Pawson S (2016) Tropical waves and the quasi-biennial oscillation in a 7-km global climate simulation. J Atmos Sci 73:3771–3783. doi:10.1175/JAS-D-15-0350.1
Huffman GJ, Adler RF, Morrissey MM, Bolvin DT, Curtis S, Joyce R, McGavock B, Susskind J (2001) Global precipitation at one-degree daily resolution from multi-satellite observations. J Hydrometeorol 2:36–50. doi:10.1175/1525-7541(2001)002<0036:GPAODD>2.0.CO;2
Huffman GJ, Bolvin DT, Nelkin EJ, Wolff DB, Adler RF, Gu G, Hong Y, Bowman KP, Stocker EF (2007) The TRMM multi-satellite precipitation analysis (TMPA): quasi-global, multiyear, combined-sensor precipitation estimates at fine scale. J Hydrometeorol 8:38–55. doi:10.1175/JHM560.1
Hung M-P, Lin J-L, Wang W, Kim D, Shinoda T, Weaver SJ (2013) MJO and convectively coupled equatorial waves simulated by 20 CMIP5 models. J Clim 26:6185–6214. doi:10.1175/JCLI-D-12-00541.1
Hyodo M, Hori T, Kaneda Y (2016) A possible scenario for earlier occurrence of the nest Nankai earthquake due to triggering by an earthquake at Hyuga-nada, off southwest Japan. Earth Planets Space 68:6. doi:10.1186/s40623-016-0384-6
Ichimura T, Fujita K, Tanaka S, Hori M, Lalith M, Shizawa Y, Kobayashi H (2014) Physics-based urban earthquake simulation enhanced by 10.7 BlnDOF x 30 K time-step unstructured FE non-linear seismic wave simulation. In: SC14: International Conference for High Performance Computing, Networking, Storage and Analysis (2014). IEEE Press, Piscataway, pp 15–26. doi:10.1109/SC.2014.7
Ichimura T, Fujita K, Quinay PEB, Maddegedara L, Hori M, Tanaka S, Shizawa Y, Kobayashi H, Minami K (2015) Implicit nonlinear wave simulation with 1.08T DOF and 0.270T unstructured finite elements to enhance comprehensive earthquake simulation. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. ACM Press, New York. doi:10.1145/2807591.2807674
Ishiyama T, Nitadori K, Makino J (2012) 4.45 Pflops astrophysical N-body simulation on K computer: the gravitational trillion-body problem. In: SC ’12 Proceedings of the International Conference on High Performance Computing, Networking. Storage and Analysis. IEEE CS Press, Los Alamitos, pp 1–10
Ito K, Kuroda T, Saito K, Wada A (2015) A large number of tropical cyclone intensity forecasts around Japan using a coupled high-resolution model. Weather Forecast 30:793–808
Kain JS, Fritsch JM (1990) A one-dimensional entraining detraining plume model and its application in convective parameterization. J Atmos Sci 47:2784–2802
Kajikawa Y, Yamaura T, Tomita H, Satoh M (2015) Impact of tropical disturbance on the Indian summer monsoon onset simulated by a global cloud-system-resolving model. SOLA 11:80–84. doi:10.2151/sola.2015-020
Kajikawa Y, Miyamoto Y, Yoshida R, Yamaura T, Yashiro H, Tomita H (2016) Resolution dependence of deep convections in a global simulation from over 10-kilometer to sub-kilometer grid spacing. Prog Earth Planet Sci 3:16. doi:10.1186/s40645-016-0094-5
Kanada S, Wada A, Nakano M, Kato T (2012) Effect of planetary boundary layer schemes on the development of intense tropical cyclones using a cloud-resolving model. J Geophys Res 117:D03107. doi:10.1029/2011JD016582
Kanada S, Wada A, Sugi M (2013) Future changes in structures of extremely intense tropical cyclones using a 2-km mesh nonhydrostatic model. J Clim 26:9986–10005. doi:10.1175/JCLI-D-12-00477.1
Kawamura R, Murakami T, Wang B (1996) Tropical and midlatitude 45-day perturbations over the western Pacific during the northern summer. J Meteorol Soc Jpn 74:867–890
Kikuchi K, Kodama C, Nasuno T, Miura H, Satoh M, Noda AT, Yamada Y (2016) Tropical intraseasonal oscillation simulated in an AMIP-type experiment by NICAM. Climate Dynam 48:2507-2528. doi:10.1007/s00382-016-3219-z
Kiladis GN, Wheeler MC, Haertel PT, Straub KH, Roundy PE (2009) Convectively coupled equatorial waves. Rev Geophys 47. doi:10.1029/2008RG000266
Kinter JL, Cash B, Achuthavarier D, Adams J, Altshuler E, Dirmeyer P, Doty B, Huang B, Marx L, Manganello J, Stan C, Wakefield T, Jin E, Palmer T, Hamrud M, Jung T, Miller M, Towers P, Wedi N, Satoh M, Tomita H, Kodama C, Nasuno T, Oouchi K, Yamada Y, Taniguchi H, Andrews P, Baer T, Ezell M, Halloy C et al (2013) Revolutionizing climate modeling with Project Athena: a multi-institutional, international collaboration. Bull Am Meteorol Soc 94:231–245. doi:10.1175/Bams-D-11-00043.1
Knutson TR, Mcbride JL, Chan J, Emanuel K, Holland G, Landsea C, Held I, Kossin JP, Srivastava AK, Sugi M (2010) Tropical cyclones and climate change. Nat Geosci 3:157–163. doi:10.1038/NGEO779
Knutson TR, Sirutis JJ, Zhao M, Tuleya RE, Bender M, Vecchi GA, Villarini G, Chavas D (2015) Global projections of intense tropical cyclone activity for the late twenty-first century from dynamical downscaling of CMIP5/RCP4.5 scenarios. J Clim 28:7203–7224. doi:10.1175/JCLI-D-15-0129.1
Kobayashi K, Kitamura D, Ando K, Ohi N (2015a) Parallel computing for high-resolution/large-scale flood simulation using the K supercomputer. Hydrol Res Lett 9:61–68
Kobayashi S, Ota Y, Harada Y, Ebita A, Moriya M, Onoda H, Onogi K, Kamahori H, Kobayashi C, Endo H, Miyaoka K, Takahashi K (2015b) The JRA-55 reanalysis: general specifications and basic characteristics. J Meteorol Soc Jpn 93:5–48. doi:10.2151/jmsj.2015-001
Kodama C, Yamada Y, Noda AT, Kikuchi K, Kajikawa Y, Nasuno T, Tomita T, Yamaura T, Takahashi HG, Hara M, Kawatani Y, Satoh M, Sugi M (2015) A 20-year climatology of a NICAM AMIP-type simulation. J Meteorol Soc Jpn 93:393–424. doi:10.2151/jmsj.2015-024
Krishnamurthy V, Stan C, Randall DA, Shukla RP, Kinter JL (2014) Simulation of the south Asian monsoon in a coupled model with an embedded cloud-resolving model. J Clim 27:1121–1142
Kumahata K, Minami K, Maruyama N (2016) High-performance conjugate gradient performance improvement on the K computer. Int J High Perform Comput Appl 30(1):55–70. doi:10.1177/1094342015607950
Kunii M (2014a) Mesoscale data assimilation for a local severe rainfall event with the NHM-LETKF system. Weather Forecast 29:1093–1105
Kunii M (2014b) The 1000-member ensemble Kalman filtering with the JMA nonhydrostatic mesoscale model on the K computer. J Meteorol Soc Jpn 91:623–633. doi:10.2151/jmsj.2014-607
Kusunoki S, Arakawa O (2015) Are CMIP5 models better than CMIP3 models in simulating precipitation over East Asia? J Clim 28:5601–5621
Lee JY, Wang B (2014) Future change of global monsoon in the CMIP5. Climate Dynam 42:101–119
Liebmann B, Smith CA (1996) Description of a complete (interpolated) outgoing longwave radiation dataset. Bull Am Meteorol Soc 77:1275–1277
Lloyd ID, Vecchi GA (2011) Observational evidence for oceanic controls on hurricane intensity. J Clim 24:1138–1153. doi:10.1175/2010JCLI3763.1
Madden R, Julian P (1972) Description of global-scale circulation cells in the tropics with a 40-50 day period. J Atmos Sci 29:1109–1123
Manganello JV, Hodges KI, Kinter JL III, Cash BA, Marx L, Jung T, Achuthavarier D, Adams JM, Altshuler EL, Huang B, Jin EK, Stan C, Towers P, Wedi N (2012) Tropical cyclone climatology in a 10-km global atmospheric GCM: toward weather-resolving climate modeling. J Clim 25:3867–3893. doi:10.1175/JCLI-D-11-00346.1
Manganello JV, Hodges KI, Dirmeyer B, Kinter JL III, Cash BA, Marx L, Jung T, Achuthavarier D, Adams JM, Altshuler EL, Huang B, Jin EK, Towers P, Wedi N (2014) Future changes in the western North Pacific tropical cyclone activity projected by a multidecadal simulation with a 16-km global atmospheric GCM. J Clim 27:7622–7646. doi:10.1175/JCLI-D-13-00678.1
Miura H, Satoh M, Nasuno T, Noda AT, Oouchi K (2007a) A Madden-Julian oscillation event realistically simulated by a global cloud-resolving model. Science 318:1763–1765. doi:10.1126/science.1148443
Miura H, Satoh M, Tomita H, Nasuno T, Iga S, Noda AT (2007b) A short-duration global cloud-resolving simulation with a realistic land and sea distribution. Geophys Res Lett 34:L02804. doi:10.1029/2006GL027448
Miyakawa T, Satoh M, Miura H, Tomita H, Yashiro H, Noda AT, Yamada Y, Kodama C, Kimoto M, Yoneyama K (2014) Madden-Julian oscillation prediction skill of a new-generation global model demonstrated using a supercomputer. Nat Commun 5:3769. doi:10.1038/Ncomms4769
Miyamoto Y, Kajikawa Y, Yoshida R, Yamaura T, Yashiro H, Tomita H (2013) Deep moist atmospheric convection in a subkilometer global simulation. Geophys Res Lett 40(18):4922–4926. doi:10.1002/grl.50944
Miyamoto Y, Yoshida R, Yamaura T, Yashiro H, Tomita H, Kajikawa Y (2015) Does convection vary in different cloud disturbances? Atmos Sci Lett 16(3):305–309. doi:10.1002/asl2.558
Molinari J, Vollaro D (2012) A subtropical cyclonic gyre associated with interactions of the MJO and the midlatitude jet. Mon Weather Rev 140:343–357. doi:10.1175/MWR-D-11-00049.1
Moon JY, Wang B, Ha KJ, Lee JY (2013) Teleconnections associated with Northern Hemisphere summer monsoon intraseasonal oscillation. Climate Dynam 40:2761–2774. doi:10.1007/s00382-012-1394-0
Murakami T, Matsumoto J (1994) Summer monsoon over the Asian continent and western North Pacific. J Meteorol Soc Jpn 72:719–745
Murakami H, Wang Y, Yoshimura H, Mizuta R, Sugi M, Shindo E, Adachi T, Yukimoto S, Hosaka M, Kusunoki S, Ose T, Kitoh A (2012) Future changes in tropical cyclone activity projected by the new high-resolution MRI-AGCM. J Clim 25:3237–3260. doi:10.1175/JCLI-D-11-00415.1
Murakami H, Vecchi GA, Underwood S, Delworth TL, Wittenberg AT, Anderson WG, Chen JH, Gudgel RG, Harris LM, Lin SJ, Zeng F (2015) Simulation and prediction of category 4 and 5 hurricanes in the high-resolution GFDL HiFLOR coupled climate model. J Clim 28:9058–9079. doi:10.1175/JCLI-D-15-0216.1
Nakano M, Sawada M, Nasuno T, Satoh M (2015) Intraseasonal variability and tropical cyclogenesis in the western North Pacific simulated by a global nonhydrostatic atmospheric model. Geophys Res Lett 42:565–571. doi:10.1002/2014gl062479
Nakano M, Wada A, Sawada M, Yoshimura H, Onishi R, Kawahara S, Sasaki W, Nasuno T, Yamaguchi M, Iriguchi T, Sugi M, Takeuchi Y (2017) Global 7-km mesh nonhydrostatic Model Intercomparison Project for improving TYphoon forecast (TYMIP-G7): Experimental design and preliminary results. Geosci Model Dev 10:1363-1381. doi:10.5194/gmd-2016-184
Nakazawa T (2006) Madden-Julian oscillation activity and typhoon landfall on Japan 2004. SOLA 2:136–139. doi:10.2151/sola.2006-035
Nasuno T, Satoh M (2011) Properties of precipitation and in-cloud vertical motion in a global nonhydrostatic aquaplanet experiment. J Meteorol Soc Jpn 89:413–439. doi:10.2151/jmsj.2011-502
Noda AT, Satoh M, Yamada Y, Kodama C, Seiki T (2014) Responses of tropical and subtropical high-cloud statistics to global warming. J Clim 27:7753–7768. doi:10.1175/JCLI-D-14-00179.1
Noda AT, Yamada Y, Kodama C, Miyakawa T, Seiki T, Satoh M (2015) Cold and warm rain simulated using a global nonhydrostatic model without cumulus parameterization, and their responses to a warmer atmospheric condition. J Meteorol Soc Jpn 93:181–197. doi:10.2151/jmsj.2015-010
Ogata T, Ueda H, Inoue T, Hayasaki M, Yoshida A, Watanabe S, Kira M, Ooshiro M, Kumai A (2014) Projected future changes in the Asian monsoon: a comparison of CMIP3 and CMIP5 model results. J Meteorol Soc Jpn 92:207–225. doi:10.2151/jmsj.2014-302
Ohno T, Satoh M (2015) On the warm core of a tropical cyclone formed near the tropopause. J Atmos Sci 72:551–571. doi:10.1175/JAS-D-14-0078.1
Oishi Y, Imamura F, Sugawara D (2015) Near-field tsunami inundation forecast using the parallel TUNAMI-N2 model: application to the 2011 Tohoku-Oki earthquake combined with source inversions. Geophys Res Lett 42:1083–1091. doi:10.1002/2014GL062577
Oouchi K, Yoshimura J, Yoshimura H, Mizuta R, Kusunoki S, Noda A (2006) Tropical cyclone climatology in a global-warming climate as simulated in a 20 km-mesh global atmospheric model: frequency and wind intensity analyses. J Meteorol Soc Jpn 84:259–276
Oouchi K, Noda AT, Satoh M, Miura H, Tomita H, Nasuno T, Iga S (2009a) A simulated preconditioning of typhoon genesis controlled by a boreal summer Madden-Julian oscillation event in a global cloud-system-resolving model. SOLA 5:65–68. doi:10.2151/sola.2009-017
Oouchi K, Noda AT, Satoh M, Wang B, Xie SP, Takahashi HG, Yasunari T (2009b) Asian summer monsoon simulated by a global cloud-system-resolving model: Diurnal to intra-seasonal variability. Geophys Res Lett 36:L11815. doi:10.1029/2009gl038271
Palmer T (2014) Build high-resolution global climate models. Nature 515:338–339
Petch JC, Brown AR, Gray MEB (2002) The impact of horizontal resolution on the simulations of convective development over land. Q J R Meteorol Soc 128:2031–2044. doi:10.1256/003590002320603511
Prein AF, Langhans W, Fosser G, Ferrone A, Ban N, Goergen K, Keller M, Tölle M, Gutjahr O, Feser F, Brisson E, Kollet S, Schmidli J, van Lipzig NPM, Leung R (2015) A review on regional convection-permitting climate modeling: Demonstrations, prospects, and challenges. Rev Geophys 53:323–361
Roberts MJ, Vidale PL, Mizielinski MS, Strachan J, Hodges K, Bell R, Camp J (2015) Tropical cyclone in the UPSCALE ensemble of high resolution global climate models. J Clim 28:574–596. doi:10.1175/JCLI-D-14-00131.1
Rogers RF, Reasor P, Lorsolo S (2013) Airborne Doppler observations of the inner-core structural differences between intensifying and steady-state tropical cyclones. Mon Weather Rev 141:2970–2991
Saito K, Tsuyuki T, Seko H, Kimura F, Tokioka T, Kuroda T, Duc L, Ito K, Oizumi T, Chen G, Ito J, SPIRE Field 3 Mesoscale NWP group (2013) Super high-resolution mesoscale weather prediction. J Phys Conf Ser 454:012073. doi:10.1088/1742-6596/454/1/012073
Sato T, Miura H, Satoh M, Takayabu YN, Wang Y (2009) Diurnal cycle of precipitation in the tropics simulated in a global cloud-resolving model. J Clim 22:4809–4826. doi:10.1175/2009jcli2890.1
Satoh M, Matsuno T, Tomita H, Miura H, Nasuno T, Iga S (2008) Nonhydrostatic Icosahedral Atmospheric Model (NICAM) for global cloud resolving simulations. J Comp Phys 227:3486–3514. doi:10.1016/j.jcp.2007.02.006
Satoh M, Iga S, Tomita H, Tsushima Y, Noda AT (2012a) Response of upper clouds due to global warming tested by a global atmospheric model with explicit cloud processes. J Clim 25:2178–2191. doi:10.1175/JCLI-D-11-00152.1
Satoh M, Oouchi K, Nasuno T, Taniguchi H, Yamada Y, Tomita H, Kodama C, Kinter J, Achuthavarier D, Manganello J, Cash B, Jung T, Palmer T, Wedi N (2012b) The intra-seasonal oscillation and its control of tropical cyclones simulated by high-resolution global atmospheric models. Clim Dyn 39:2185–2206. doi:10.1007/s00382-011-1235-6
Satoh M, Tomita H, Yashiro H, Miura H, Kodama C, Seiki T, Noda AT, Yamada Y, Goto D, Sawada M, Miyoshi T, Niwa Y, Hara M, Ohno Y, Iga S, Arakawa T, Inoue T, Kubokawa H (2014) The Non-hydrostatic Icosahedral Atmospheric Model: Description and development. Prog Earth Planet Sci 1:18. doi:10.1186/s40645-014-0018-1
Satoh M, Yamada Y, Sugi M, Kodama C, Noda AT (2015) Constraint on future change in global frequency of tropical cyclones due to global warming. J Meteorol Soc Jpn 93:489–500. doi:10.2151/jmsj.2015-025
Seko H, Kunii M, Yokota S, Tsuyuki T, Miyoshi T (2015) Ensemble experiments using a nested LETKF system to reproduce intense vortices associated with tornadoes of 6 May 2012 in Japan. Prog Earth Planet Sci 2:42. doi:10.1186/s40645-015-0072-3
Sperber K, Annamalai H, Kang IS, Kitoh A, Moise A, Turner A, Wang B, Zhou T (2013) The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century. Clim Dyn 41:2711–2744. doi:10.1007/s00382-012-1607-6
Stevens B, Bony S (2013) What are climate models missing? Science 340:1053–1054. doi:10.1126/science.1237554
Sugi M, Noda A, Sato N (2002) Influence of the global warming on tropical cyclone climatology: An experiment with the JMA global model. J Meteorol Soc Jpn 80:249–272. doi:10.2151/jmsj.80.249
Sugi M, Murakami H, Yoshimura J (2012) On the mechanism of tropical cyclone frequency changes due to global warming. J Meteorol Soc Jpn 90A:397–408. doi:10.2151/jmsj.2012-A24
Taniguchi H, Yanase W, Satoh M (2010) Ensemble simulation of cyclone Nargis by a global cloud-system-resolving model - Modulation of cyclogenesis by the Madden-Julian oscillation. J Meteorol Soc Jpn 88:571–591
Terai M, Yashiro H, Sakamoto K, Iga S, Tomita H, Satoh M, Minami K (2014) Performance optimization and evaluation of a global climate application using a 440 m horizontal mesh on the K computer, Abstract presented at the 2014 International Conference for High Performance Computing. Networking, Storage and Analysis, Ernest N. Morial Convention Center, New Orleans, 16-21 November 2014
Terasaki K, Sawada M, Miyoshi T (2015) Local ensemble transform Kalman Filter experiments with the Nonhydrostatic Icosahedral Atmospheric Model NICAM. SOLA 11:23–26. doi:10.2151/sola.2015-006
Tomita H (2008) New microphysical schemes with five and six categories by diagnostic generation of cloud ice. J Meteorol Soc Jpn 86:121–142
Tomita H, Satoh M (2004) A new dynamical framework of nonhydrostatic global model using the icosahedral grid. Fluid Dyn Res 34:357–400
Tomita H, Miura H, Iga S, Nasuno T, Satoh M (2005) A global cloud-resolving simulation: Preliminary results from an aqua planet experiment. Geophys Res Lett 32:L08805. doi:10.1029/2005GL022459
Top 500 (2011a) Top 500 List June 2011., https://www.top500.org/lists/2011/06/. Accessed 10 Mar 2017
Top 500 (2011b) Top 500 List November 2011., https://www.top500.org/lists/2011/11/. Accessed 10 Mar 2017
Tsuboi S, Ando K, Miyoshi T, Peter D, Komatitsch D, Tromp J (2016) A 1.8 trillion degrees-of-freedom, 1.24 petaflops global seismic wave simulation on the K computer. Int J High Perform Comput Appl 30(4):411–422. doi:10.1177/1094342016632596
Tsushima Y, Iga S, Tomita H, Satoh M, Noda AT, Webb M (2014) High cloud increase in a perturbed SST experiment with a global nonhydrostatic model including explicit convective processes. J Adv Model Earth Syst 6:571–585. doi:10.1002/2013MS000301
Vecchi GA, Soden BJ (2007) Global warming and the weakening of the tropical circulation. J Clim 20:4316–4340. doi:10.1175/JCLI4258.1
Vitart F, Robertson AW, Anderson DLT (2012) Subseasonal to Seasonal Prediction Project: Bridging the gap between weather and climate. WMO Bull 61:23–28
Vitart F, Ardilouze C, Bonet A, Brookshaw A, Chen M, Codorean C, Déqué M, Ferranti L, Fucile E, Fuentes M, Hendon H, Hodgson J, Kang H, Kumar A, Lin H, Liu G, Liu X, Malguzzi P, Mallas I, Manoussakis M, Mastrangelo D, MacLachlan C, McLean P, Minami A, Mladek R, Nakazawa T, Najm S, Nie Y, Rixen M, Robertson A et al (2017) The Subseasonal to Seasonal (S2S) Prediction Project Database. Bull Am Meteorol Soc 98:163–173. doi:10.1175/BAMS-D-16-0017.1
Wada A, Uehara T, Ishizaki S (2014) Typhoon-induced sea surface cooling during the 2011 and 2012 typhoon seasons: observational evidence and numerical investigations of the sea surface cooling effect using typhoon simulations. Prog Earth Planet Sci 1:11. doi:10.1186/2197-4284-1-11
Walsh KJE, Camargo SJ, Vecchi GA, Daloz AS, Elsner J, Emanuel K, Horn M, Lim Y-K, Roberts M, Patricola C, Scoccimarro E, Sobel AH, Strazzo S, Villarini G, Wehner M, Zhao M, Kossin JP, LaRow T, Oouchi K, Schubert S, Wang H, Bacmeister J, Chang P, Chauvin F, Jablonowski C, Kumar A, Murakami H, Ose T, Reed KA, Saravanan R et al (2015) Hurricanes and climate: the U.S. CLIVAR working group on hurricanes. Bull Am Meteorol Soc 96:997–1017. doi:10.1175/BAMS-D-13-00242.1
Walsh KJE, Mcbride JL, Klotzbach PJ, Balachandran S, Camargo SJ, Holland G, Knutson TR, Kossin JP, Lee T, Sobel A, Sugi M (2016) Tropical cyclone and climate change. WIREs Clim Change 7:65–89. doi:10.1002/wcc.371
Wang B, Rui H (1990) Synoptic climatology of transient tropical intraseasonal convection anomalies: 1975-1985. Meteorol Atmos Phys 44:43–61
Wang B, Xie X (1997) A model for the boreal summer intraseasonal oscillation. J Atmos Sci 54:72–86
Wang H, Wang Y (2014) A numerical study of typhoon Megi (2010). Part I: rapid intensification. Mon Weather Rev 142:29–48. doi:10.1175/MWR-D-13-00070.1
Wang B, Ding QH, Fu XH, Kang IS, Jin K, Shukla J, Doblas-Reyes F (2005) Fundamental challenge in simulation and prediction of summer monsoon rainfall. Geophys Res Lett 32:L15711. doi:10.1029/2005gl022734
Wehner M, Reed KA, Stone D, Collins WD, Bacmeister J (2015) Resolution dependence of future tropical cyclone projections of CAM5.1 in the US CLIVAR Hurricane Working Group idealized configurations. J Clim 28:3905–3925. doi:10.1175/JCLI-D-14-00311.1
Wheeler MC, Hendon HH (2004) An all-season real-time multivariate MJO index: development of an index for monitoring and prediction. Mon Weather Rev 132:1917–1932
Yamada Y, Oouchi K, Satoh M, Tomita H, Yanase W (2010) Projection of changes in tropical cyclone activity and cloud height due to greenhouse warming: global cloud-system-resolving approach. Geophys Res Lett 37:L07709. doi:10.1029/2010GL042518
Yamamoto K, Uno A, Murai H, Tsukamoto T, Shoji F, Matsui S, Sekizawa R, Sueyasu F, Uchiyama H, Okamoto M, Ohgushi N, Takashina K, Wakabayashi D, Taguchi Y, Yokokawa M (2014) The K computer operations: experiences and statistics. Procedia Comput Sci 29:576–585. doi:10.1016/j.procs.2014.05.052
Yamaura T, Kajikawa Y, Tomita H, Satoh M (2013) Possible impact of a tropical cyclone on the northward migration of the Baiu frontal zone. SOLA 9:89–93. doi:10.2151/sola.2013-020
Yanase W, Taniguchi H, Satoh M (2010) The genesis of tropical cyclone Nargis (2008): environmental modulation and numerical predictability. J Meteorol Soc Jpn 88:497–519. doi:10.2151/jmsj.2010-314
Yashiro H, Kajikawa Y, Miyamoto Y, Yoshida R, Yamaura R, Tomita H (2016a) Resolution dependency of diurnal precipitation cycle simulated by global cloud resolving model. SOLA 12:272–276
Yashiro H, Terai M, Yoshida R, Iga S, Minami K, Tomita H (2016b) Performance analysis and optimization of Nonhydrostatic ICosahedral Atmospheric Model (NICAM) on the K Computer and TSUBAME2.5. In: Proceedings of the Platform for Advanced Scientific Computing Conference (PASC’16). ACM, New York, p Article 3. doi:10.1145/2929908.2929911
Yashiro H, Terasaki K, Miyoshi T, Tomita H (2016c) Performance evaluation of a throughput-aware framework for ensemble data assimilation: the case of NICAM-LETKF. Geosci Model Dev 9:2293–2300. doi:10.5194/gmd-9-2293-2016
Yokokawa M, Shoji F, Uno A, Kurokawa M, Watanabe T (2011) The K computer: Japanese next-generation supercomputer development project. In: IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED) 2011. IEEE, New York, pp 371–372. doi:10.1109/ISLPED.2011.5993668
Yokota S, Seko H, Kunii M, Yamauchi H, Niino H (2016) The tornadic supercell on the Kanto Plain on 6 May 2012: polarimetric radar and surface data assimilation with EnKF and ensemble-based sensitivity analysis. Mon Weather Rev 144:3133–3157. doi:10.1175/MWR-D-15-0365.1
Yoneyama K, Zhang C, Long CN (2013) Tracking pulses of the Madden-Julian oscillation. Bull Am Meteorol Soc 94:1871–1891
Zarzycki CM (2016) Tropical cyclone intensity errors associated with lack of two-way ocean coupling in high-resolution global simulations. J Clim 29:8589–8610
Zou LW, Zhou TJ (2015) Asian summer monsoon onset in simulations and CMIP5 projections using four Chinese climate models. Adv Atmos Sci 32:794–806
All the simulations analyzed in this study were performed on the K computer at the RIKEN Advanced Institute for Computational Science (Proposal number hp120279, hp120313, hp130010, hp140219, and hp150213). This study was supported by Strategic Programs for Innovative Research (SPIRE) Field 3 (Projection of Planet Earth Variations for Mitigating Natural Disasters), which is promoted by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan. All the authors acknowledge Ms. Hisae Tamura for formatting the manuscript and INLEXIO for the English editing.
This study was supported by the Strategic Programs for Innovative Research (SPIRE) Field 3 (Projection of Planet Earth Variations for Mitigating Natural Disasters) and Future LAtency core-based General-purpose Supercomputer with HIgh Productivity (FLAGSHIP2020 project), which are promoted by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan.
Availability of data and materials
The NICAM data simulated by the K computer will be available. The following three papers are to be submitted or in review process and referred to as personal communication in the text: Fujita et al. (2017), Kodama et al. (2017), and Yamada et al. (2017). Please contact the authors for these data and unpublished manuscripts:
Fujita M, Kodama C, Yamada Y, Noda AT, Nakano M, Nasuno T, Satoh M (2017) Change in precipitation around Japan induced by tropical cyclones as projected by the NICAM AMIP-type 25-year simulation.
Kodama C, Stevens B, Mauritsen T, Seiki T, Satoh M (2017) Evidence for larger-than average increases in rainfall from intense cyclones with warming.
Yamada Y, Satoh M, Sugi M, Kodama C, Noda AT, Nakano M, Nasuno T (2017) Response of tropical cyclone activity and structure to a global warming using a high-resolution global non-hydrostatic model.
MS led the studies using NICAM in SPIRE and FLAGSHIP2020 and coordinated this review. HY wrote the “Computational aspects” section. YK and YM wrote the “Sub-kilometer global simulation” section, and the experiments described in this section were carried out by the team at the Advanced Institute for Computational Science: YM, YK, HY, TY, and HT. YK and HY wrote the “Asian summer monsoon” section, and HY conducted the experiments described in this section. TM wrote the “Madden-Julian oscillation” section and conducted the experiments described in this section. MN wrote the “Boreal summer intra-seasonal oscillation and tropical cyclogenesis” section and conducted the experiments described in this section. CK, ATN, TN, YY, and YF wrote the “AMIP-type experiments” section, and YY and MS wrote the “Future projection of tropical cyclones” section. CK, ATN, and YY conducted the experiments described in these two sections. HT, MS, and CK wrote the “Future perspectives” section. MS is responsible for the whole manuscript and wrote the “Introduction” and “Conclusions” sections. All authors read and approved the final manuscript.
MS and HT started the development of NICAM around the year 2000, and the NICAM studies have been conducted mainly in these three institutes since then: Atmosphere and Ocean Research Institute of the University of Tokyo, JAMSTEC, and the Advanced Institute for Computational Science.
Consent for publication
All the figures in this paper are permitted to be used by the copyright holders of the original manuscripts.
Ethics approval and consent to participate
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Satoh, M., Tomita, H., Yashiro, H. et al. Outcomes and challenges of global high-resolution non-hydrostatic atmospheric simulations using the K computer. Prog. in Earth and Planet. Sci. 4, 13 (2017). https://doi.org/10.1186/s40645-017-0127-8
- K computer
- Intra-seasonal oscillations
- Madden-Julian oscillation
- Tropical cyclone
- Global non-hydrostatic model