Consideration of a Global Load Balancing System for Large-scale Events (Final Form)

Global Load Balancing Experiments Using the SINET3 Full Route Provision Service


The Kyushu GigaPOP Project (QGPOP), which is researching advanced network technologies primarily across the Hokubu-Kyushu region, conducted global load balancing experiments using the SINET3 Full Route Provision service. We spoke with Associate Professor Koji Okamura of the Kyushu University Research Institute for Information Technology, and Professor Toshihiko Shimokawa of the Department of Social Information Systems at the Kyushu Sangyo University Faculty of Information Science, to get an overview of the experiment and learn what has been achieved.
(Interview date: October 25, 2010)

Firstly, could you please give us an outline of QGPOP?

Okamura: Certainly. QGPOP was established around 1999 as a project to conduct research and development into an interconnection architecture for connecting to a high-speed R&D backbone, and involves the preparation of access lines with the help of universities and research institutions in the Kyushu region. Today, a number of universities and businesses, based primarily in the Hokubu-Kyushu region, are involved in the project. Its specific activities include the presentation of research in a conference-like format and experiments conducted in conjunction with multiple universities and institutions. Although I belong to the Kyushu University Research Institute for Information Technology, we install and operate QGPOP backbone switches and other equipment at the Institute.


Shimokawa: At my laboratory, we have taken up basic technologies of the next-generation internet as a research subject and we are conducting research and development in areas such as networks, web applications, security, global load balancing and virtual machines. Since joint research with various organizations and projects is very important in these activities, we are also an active participant in QGPOP.

Could you tell us about the background to your tackling these global load balancing experiments?

Shimokawa: QGPOP has long had communications with the Cyber Kansai Project (”CKP,” hereafter), and there was talk about whether the two projects could do something together. One of the proposals put forward was the online relaying of the summer high school baseball championships carried out by CKP and Asahi Broadcasting Corporation. What we wanted to do was try conducting a global load balancing experiment using a continuous delivery service for still images, which was one of the types of content. Since global load balancing experiments using DNS Anycast had originally been conducted at the Shimokawa Lab, I felt it was a really interesting subject matter.

In the latest experiment you implemented global load balancing that used routing information.

Okamura: That’s right. Of course there is also the method of using DNS, but this time we wanted to tackle routing-based global load balancing as a new challenge. As for the method, we assign identical IP addresses to the production server installed in Kansai and a mirror server installed at Kyushu University, and send out routing information from each. In principle, since the routing information coming from far away loses out, accesses from close to Kyushu University should come to Kyushu University. In addition, here we had our eye on the SINET full-route provision service. By using the service and sending out routing information from Kyushu University to SINET, we thought that only accesses from SINET subscriber institutions would converge on the mirror server.

Did you encounter any problems in conducting the experiments?

Okamura: The point of this experiment is that the routing information we send out is not output from SINET. If the routing information were to get outside of SINET, users other than SINET subscribing institutions might pick up the routing information and end up at the Kyushu University side. However, one thing we miscalculated is that SINET had many more peers than we had anticipated (laughs).
SINET uses BGP as a route exchange protocol with outside networks, and at first we thought it would be sufficient to have filters applied at around one or two locations. However, when it came time to start the tests, we found that accesses were coming not only from commercial ISPs in Japan but from the United States as well. This doesn’t really mean that people in the United States are accessing the site, but rather that accesses from Japan in which this route is deemed closer in networking terms arrive after making a round-trip to the United States. Since the mirror server is not as high-spec as the production server, if congestion caused by high-volume accesses occurs and cripples content distribution, this would also cause inconvenience Asahi Broadcasting Corporation, the original distribution source. Given this, we also received help from NII and plugged the leaks of routing information one by one.

How was the main experiment conducted?

Shimokawa: For a total of three days on August 11, 13, and 14, 2009, still image delivery for three respective games was carried out from the mirror server. This worked by commencing route advertisements to SINET just before the start of each game, and halting route advertisement after each game had finished.
When we actually conducted the experiments, we observed a tendency that differed greatly from when we performed global load balancing with DNS Anycast. With DNS-based control, since information remains on DNS servers in various locations, accesses did not cease even after the experiment had finished. However, with routing-based control, accesses began as soon as we’d begun route advertising, and stopped completely once route advertising had been halted. While we had predicted this would happen before running the experiment, it was exciting to be able to actually confirm this as data.

Okamura: Another interesting thing was that we were able to check the regional characteristics of the content. Professor Shimokawa had also thought that for content like that for high school baseball, there must be regional characteristics that differ from general content, but until then we’d had no way to substantiate this. However, when we analyzed the results of this experiment, we found that when high schools in the Koshinetsu district were playing, there were indeed more accesses from SINET subscriber institutions in the Koshinetsu district. Although one would be right to expect this, we are glad to have obtained data that fully bears this out.

How did you find the full route provision service used in this experiment?

Okamura: My personal opinion is that at the very least, for the information infrastructure center of a university with some degree of size, the full route provision service should be used. Unless we’d been conducting our own research into routing control from the outset, we wouldn’t have been able to conduct this kind of experiment. In that respect, if full routing is used on the university side and BGP operations and other tasks are carried out, it will be possible to provide support to students when they develop an interest in these fields. Even in the sense of accumulating network technologies, I think the service should be actively used.

Was there a significant educational effect on students at the Shimokawa Lab?

Shimokawa: Yes, there was. Since people from a number of businesses, universities and research institutions participated in this experiment, for students, being able to take part in the same event was a valuable experience. The members said that it was good to use the knowledge and technologies they had learned in the lab. As these efforts also help revitalize the laboratory, I hope to actively participate in future endeavors as well.

Lastly, could you tell us about what areas of future research are exciting for you at present?

Okamura: Today, the internet has become something that is familiar to everyone, and it is now possible to use it in a casual manner without having to know how it works. Even looking at the subjects of students’ research, there is a shifting interest to higher-level layers such as new services that utilize the web. However, if they understand things like how a packet is sent through a router, or whether it goes through North America or Southeast Asia on its way to Europe, then the internet will become even more interesting. For our part, we hope to create an environment that enables students to develop that kind of interest.

Thank you.