Content delivery - practice and theory

Abstract

Akamai has deployed over 13,000 servers on 1,000 networks at 1,500 locations around the world. These servers deliver html documents, static images, and streaming media for over 1,300 of the most popular content providers on the web. Over the years Akamai's systems have had to contend with staggering growth as well as events ranging from network partitions to denial of service attacks. Our experience has been that solid engineering backed by efficient algorithms with provable guarantees is critical for scalable and robust systems. In this talk we consider - mapping and collection/dissemination - two problems central to content delivery networks. We present a study of mapping from an engineering and experiential standpoint and collection/dissemination from a theoretical viewpoint. We explore the issues involved in mapping by considering the problem of directing traffic to website mirrors. Using topology discovery, clustering, congestion measurement and load-balancing algorithms we show how FirstPoint, Akamai's DNS-based application-level anycast service, has measurably improved the performance of many leading websites. The problem of collecting data from and disseminating information to a distributed network of servers can be modeled in terms of confluent flows. We present approximation algorithms and hardness results for the problem of finding confluent flows of minimum congestion. Note: The work on confluent flows is joint with Jiangzhuo Chen and Rajmohan Rajaraman of Northeastern University.