Researchers at MIT’s Computer Science and Artificial Intelligence Lab have designed a system where programs can have access to ad hoc optimally allocated cache memory. In a simulation test system with 36 cores, 'Jenga' CPU cache memory access increased processing speed by 20 to 30 per cent and energy efficiency by as much as 85 per cent.
Computer chips today usually have three to four levels of cache memory on chip with each level being slower but more capacious than the last. The fixed design is obviously a trade off and not optimised for any particular program or application but it's simply unavoidable with current technology. However, MIT scientists have designed a system where cache hierarchies can be allocated on the fly, making the most of the different caches available on chip.
Each core in a multi-core chip usually has two levels of local private cache and then there is a shared third cache, usually in discrete memory banks situated around the chip, plus sometimes a forth DRAM cache on an additional off chip construction. Of course it is faster and more efficient for a core to access nearer by cache memory. Thus Jenga distinguishes between memory banks by location relative to each core and minimises data movement.
Jenga (PDF) has to analyse the following to optimise cache for an application: "What’s the size of the data it accesses? Does it have hierarchical reuse, so that it would benefit from a hierarchy of progressively larger memories? Or is it scanning through a data structure, so we’d be better off having a single but very large level? How often does it access data? How much would its performance suffer if we just let data drop to main memory?" Now, thanks to a computational shortcut that is correct more than 99 per cent of the time, Jenga can provide processes with optimal cache memory access updated every 100 milliseconds.
Explaining the unique appeal of Jenga, David Wood, a professor of computer science at the University of Wisconsin at Madison, said "There have been a number of previous schemes that tried to do some kind of dynamic creation of the hierarchy. Jenga is different in that it really uses the software to try to characterize what the workload is and then do an optimal allocation of the resources between the competing processes. And that, I think, is fundamentally more powerful than what people have been doing before." Jenga is one of the most interesting ways for designing a cache hierarchy seen so far, thought Wood.
