The Japan Agency for Marine-Earth Science and Technology (JAMSTEC) is a national research and development agency. JAMSTEC is engaged in activities to contribute broadly to the improvement of science and technology, the development of academic research, and the understanding of the Earth and life through ocean research in Japan, a nation surrounded by oceans. To learn more about the development of the network environment and the predictive simulation of global climate change and natural disasters on the JAMSTEC supercomputer “Earth Simulator,” we interviewed Manager Jun Naoi (Information Security Management Section, Information Security and System Department, JAMSTEC); Engineer Ryota Higuchi (Information Systems Section, Information Security and System Department, JAMSTEC); Group Leader Hitoshi Uehara; Deputy Group Leader Satoru Okura; and Engineer Tomoki Sasaki (Supercomputer Engineering and Administration Group, Center for Earth Information Science and Technology, Research Institute for Value-Added Information Generation, JAMSTEC).
(Interview date: July 23, 2024)

Please give an overview of JAMSTEC.

Naoi: JAMSTEC’s mission is to promote an integrated understanding of the oceans, the earth, life, and humanity, and to create the future of the earth through co-creation with society. It is a collaborative system comprising five scientific research institutes and one division focused on technology development working together to advance research and development, and the six locations across Japan are connected by SINET.

The oceanographic research platform managed by JAMSTEC consists of six vessels, including the Support Vessel Yokosuka, and the Deep-sea Submergence Research Vehicle SHINKAI 6500 and other probes, with surveys, observations, and tests conducted with the help of these vessels producing results in climate, earthquakes, life, and various other fields as well as in technological development. Moreover, JAMSTEC manages the supercomputer system that is the Earth Simulator and has made considerable contributions in the fields of global warming prediction research, climate change adaptation prediction research, and disaster prevention research using earthquake and tsunami simulations.

These oceanographic observation data and simulation results are managed systematically while maintaining data quality control, and are made available to researchers in Japan and overseas via the network.

Let me to ask you about the Earth Simulator.

Uehara: The Earth Simulator is a supercomputer managed by JAMSTEC that was developed for the purpose of researching climate change, including global warming and abnormal weather. The first generation (ES1) began operation in 2002 and by 2021 had undergone several upgrades so that it is currently in its fourth generation (ES4). With this fourth generation, the total peak performance has been greatly improved by about 150 times compared to the second generation that we had at the time of the last interview (March 15, 2010). ES4 incorporates three distinct types of node architectures: x86-compatible CPU nodes, nodes equipped with vector engines, and nodes equipped with GPUs. Nodes equipped with GPUs are well-suited for the increasingly popular fields of GPU-based computing and AI research. One of the features is that three node types are connected, so that data can be exchanged at high speed. For example, it is now possible to perform complex simulations and data analysis, such as preprocessing data on CPU nodes, computation on nodes equipped with vector engines, and having AI train with the computation results on nodes equipped with GPUs.

And since it handles volumes of data, the storage capacity consists of 60PB of disk storage for large-scale data and 1.3PB of flash storage for high-speed data I/O.

To manage the Earth Simulator in FY2024, computational resources are categorized into the following three types.

1. JAMSTEC Proposed Projects: Approximately 20% of the total.
2. Proposed Research Projects: Approximately 10% of the total.
3. JAMSTEC Strategic Projects: Approximately 70% of the total.

More specifically, the JAMSTEC Strategic Projects category is divided into the following three sub-categories.

• Contract Research Projects: Research projects commissioned by the government, etc.
• Challenging Use Projects: Exploratory research and short-term intensive projects, etc.
• Fee-based Usage: Non-public use by companies and organizations.

Computational resources are initially assigned at the beginning of the fiscal year for JAMSTEC proposed projects and proposed research projects. However, even after the financial year is underway, each project in the above two categories can get additional computational resources from the JAMSTEC strategic projects category, depending on how research activities progress. This allows us to meet the diverse needs of wide-ranging research fields and industries. The number of users is increasing year by year, and in FY2023, we had about 1,000 users, including some 300 who were JAMSTEC staff. The Earth Simulator is made widely available to outside users.

So, the Earth Simulator is used in a variety of fields?

Uehara: As an example of how the Earth Simulator is used, there was the “Database for Policy Decision-Making for Future Climate Change” (d4PDF) computed on ES3. It is intended to be utilized for impact assessment studies and adaptation planning for global warming, such as for example, “how many times the maximum precipitation in the past 20 years will occur when the temperature rises by 4°C (changes in frequency).” Based on the d4PDF, numerous advanced academic studies have been conducted that contribute to disaster prevention and mitigation. In addition to being used for disaster prevention measures of the Ministry of Land, Infrastructure, Transport and Tourism, it has also helped produce research results connected to the datasets of the Ministry of Education, Culture, Sports, Science and Technology and the Japan Meteorological Agency.

Looking at local governments, Wakayama Prefecture used ES3 to create a dataset on tsunami run-up caused by Tonankai Earthquake. Approximately 20,000 cases running from when the tsunami occurs to when it arrives were simulated for each epicenter and intensity, being used for Wakayama Prefecture’s unique tsunami countermeasure system, including evacuation plans and disaster prevention education.

With the ES4 that we have now, the introduction of CPU nodes and GPU nodes enables types of scientific computing and deep learning for which conventional vector processors weren’t suitable. For example, it is used for research activities such as earthquake-induced landslide calculations in Aso, Kumamoto Prefecture, environmental assessments using image recognition of marine debris washed ashore, and high-precision detection of clouds that precede typhoons near the equator.

As the scope of use expands, the network environment becomes more and more important.

Sasaki: Yes, JAMSTEC has positioned the Earth Simulator as the primary computational resource. The Earth Simulator is located at the Yokohama Institute for Earth Sciences, so we need to exchange data with researchers at other locations. Moreover, a high-speed network environment is important for transferring large volumes of data output from the high-performance Earth Simulator as well as observation data acquired by increasingly high-resolution and large-capacity probes and such.

Okura: Two-thirds of the users of the Earth Simulator belong to external organizations. In addition to simulations, data transfer may also be required over international lines. SINET’s high-speed, low-latency, and stable network environment plays an important role in this.

Please tell us about the concrete connection configuration.

Higuchi: JAMSTEC has six locations nationwide, four of which are connected through SINET6’s L2VPN. Moreover, the Yokohama Institute for Earth Sciences connects L2VPN to SINET via two 100 Gbps lines, enabling high-speed transfer of large data from the Earth Simulator.

Please tell us about the challenges and your future expectations for SINET.

Uehara: The need to make research results using the Earth Simulator publicly available, along with data sharing, is growing year by year. At the same time, the size of the datasets is also growing. To handle them, we need to strengthen the overall network infrastructure. SINET’s network is indispensable for supporting SINET use in various research fields, so I would like to ask for continued stable bandwidth and improved transfer speeds.

Higuchi: I’m in charge of operating JAMSTEC’s backbone network, including the individual locations, and the backbone network is required to operate at high speed and stably. For this reason, SINET has become an indispensable network to advance research at JAMSTEC. As researchers’ network needs are diversifying, we hope to see the development of various services from SINET and would be grateful if SINET would continue to provide stable services in the future.

Uehara: The Earth Simulator plays an important role in climate change prediction, natural disaster countermeasures, and a variety of other fields because of its advanced computing power and wide range of applications. I hope that by further deepening our collaboration with SINET, we can amplify our research outcomes and broaden our contributions to society.

Thank you.