Research Talk: Prioritized GC - Using the Garbage Collector to Support Caching

Abstract

Programmers routinely trade space for time to increase performance,especially in the form of caching and memoization. In managed languageslike Java or JavaScript, however, this trade-off is complex. More space used by the application translates to higher garbage collection overhead due to more frequent collections. The amount of space used is hard to quantify as it depends on the connectivity graph, and its effect on garbage collection overhead will vary as memory demands change over time. Existing runtime systems provide no good support for implementing space-sensitive data structures, forcing programmers into difficult and often brittle choices about provisioning.

In this talk, we will look at how programmers have managed with this issue and present our approach, the prioritized garbage collector. The collector measures the sizes of objects in an order specified by the application. Then we present the sache, a data structure that utilizes this collector to (1) figure out how much space it should use for caching, (2) figure out how much space its values are using, and (3) evict values to enforce that limit

We will look at how each of these are done and conclude with results comparing against the cache in Google's Java library, Guava.