Corundum: statically-enforced persistent memory safety

(Morteza Hoseinzadeh of UC San Diego Presenting on 5/26 at 1:00 p.m. & 7:00 p.m. ET)

Fast, byte-addressable, persistent main memories (PM) make it possible to build complex data structures that can survive system failures. Programming for PM is challenging, not least because it combines well-known programming challenges like locking, memory management, and pointer safety with novel PM-specific bug types. It also requires logging updates to PM to facilitate recovery after a crash. A misstep in any of these areas can corrupt data, leak resources, or prevent successful recovery after a crash. Existing PM libraries in a variety of languages -- C, C++, Java, Go -- simplify some of these problems, but they still require the programmer to learn (and flawlessly apply) complex rules to ensure correctness. Opportunities for data-destroying bugs abound.

This paper presents Corundum, a Rust-based library with an idiomatic PM programming interface and leverages Rust’s type system to statically avoid most common PM programming bugs. Corundum lets programmers develop persistent data structures using familiar Rust constructs and have confidence that they will be free of those bugs. We have implemented Corundum and found its performance to be as good as or better than Intel's widely-used PMDK library, HP's Atlas, Mnemosyne, and go-pmem.

FANS: FPGA-Accelerated Near-Storage Sorting

(Weikang Qiao presenting on Wed. May 19th at 1:00 & 7:00 PM ET)

 
Large-scale sorting is always an important yet demanding task for data center applications. In addition to powerful processing capability, high-performance sorting system requires efficient utilization of the available bandwidth of various levels in the memory hierarchy. Nowadays, with the explosive data size, the frequent data transfers between the host and the storage device are becoming increasingly a performance bottleneck. Fortunately, the emergence of near-storage computing devices gives us the opportunity to accelerate large-scale sorting by avoiding the back and forth data transfer. Near-storage sorting is promising for extra performance improvement and power reduction. However, it is still an open question of how to achieve the optimal sorting performance on the existing near-storage computing device. 

In this work, we first perform an in-depth analysis of the sorting performance on the newly released Samsung SmartSSD platform. Contrary to the previous belief, our analysis shows that the end-to-end sorting performance is bound by not only the bandwidth of the flash, but also the main memory bandwidth, the configuration of the sorting kernel and the intermediate sorting status. Based on our modeling, we propose FANS, an FPGA accelerated near-storage sorting system which selects the optimized design configuration and achieves the theoretically maximum end-to-end performance when using a single Samsung SmartSSD device. The experiments demonstrate more than 3× performance speedup over the state-of-art FPGA-accelerated flash storage. 

Compiler-Driven Simulation of Reconfigurable Hardware Accelerators

(Zhijing Li of Cornell presenting on Wed. 5/12/21 at 1PM & 7PM ET) 

As customized accelerator design has become increasingly popular to keep up with the demand for high performance computing, it poses challenges for modern simulator design to adapt to such a large variety of accelerators. Existing simulators tend to two extremes: low-level and general approaches, such as RTL simulation, that can model any hardware but require substantial effort and long execution times; and higher-level application-specific models that can be much faster and easier to use but require one-off engineering effort.

This work proposes a compiler-driven simulation workflow that can model the performance of arbitrary hardware accelerators, while introducing little programming overhead. The key idea is to separate structure representation from simulation by developing an intermediate language that can flexibly represent a wide variety of hardware constructs. We design the Event Queue dialect of MLIR, a dialect that can model arbitrary hardware accelerators with explicit data movement and distributed event-based control. We also implement a simulator to model EQueue-structured programs. We show the effectiveness of our simulator by comparing it with a state-of-the-art application-specialized simulator, while offering higher extensibility. We demonstrate our simulator can save designers' time with reusable lowering pipeline and visualizable outputs.

Defense against the Dark Arts: New Attacks and Defenses for Shared Last-Level Caches

(Gururaj Saileshwar presenting on Wednesday, March 24 at 1:00 p.m. and 7 p.m. ET)

Caches in modern processors improve performance by enabling fast access to data. However, the sharing of the processor caches (particularly the last-level cache) between different processor-cores, VMs and applications, results in vulnerability to side-channel and covert-channel attacks via caches. Cache side-channel attacks allow malware to infer data-access patterns of sensitive applications based on their cache-accesses, to leak encryption keys, confidential IP like DNN models, etc. Similarly, cache covert-channels allow malicious processes to use timing variation due to caches as a means of covert communication (e.g., to exfiltrate sensitive information from a sandbox). 

In this talk, I will first present our Streamline Attack (to appear in ASPLOS’21), the current fastest cache covert-channel attack and the first to achieve >1MB/s bandwidth (3-6x higher than prior attacks) by leveraging asynchronous communication protocols. With this attack, we demonstrate that cache covert-channels can have higher information capacity than previously understood, while having fewer ISA or micro-architecture specific limitations. Next, I will present MIRAGE, a Defense Against Cache Side-Channel Attacks (to appear in USENIX Security’21) that provides a practical way to enable a fully-associative last-level cache, which is immune to set-conflict based cache attacks. This defense promises to meaningfully end the arms-race between attackers and defenders in the area of conflict-based cache attacks, where successive defenses have been broken by newer attacks in the past few years.

Multiple Instance Learning Network for Whole Slide Image Classification with Hardware Acceleration

(Bin Li, U. Wisconsin-Madison, presenting on 12/9/20 at 11:00 a.m. and 7:00 p.m. ET) 

We propose a novel multiple instance learning (MIL) model for disease detection in whole slide images (WSI) as a driving application for developing hardware acceleration architectures. Our model applies self-attention mechanism on latent representations to model the dependencies between the instances. Our model demonstrates the state-of-the-art performance on both classic MIL benchmark datasets and real-word clinical WSI datasets. Collaborating with hardware research groups, we propose a processing-in-memory (PIM) design for our application as well as general long-sequence attention-based models, which outperforms other PIM-based and GPU-based baselines by a large margin. Meanwhile, we are continuously working with PIM, GPU, and FPGA groups on software-hardware co-design and acceleration performance benchmarking.

ReTail: Request-Level Latency Prediction in Multicore-Enabled Cloud Servers

(Shuang Chen of Cornell presenting December 2, 2020 at 11:00 AM & 8:00 PM ET)

Latency-critical cloud services, such as websearch, have strict quality-of-service (QoS) constraints in terms of tail latency. Improving energy efficiency usually takes second place in an effort to meet these latency constraints. Per-core Dynamic Voltage and Frequency Scaling (DVFS) can offer significant efficiency benefits, however, it is challenging to determine which requests can afford to employ DVFS without hurting the end-to-end tail latency of the entire service. 

We introduce ReTail, a framework for QoS-aware power management for LC services using request-level latency prediction. ReTail is composed of (1) a general and systematic process to collect and select the features of an application that best correlate with the processing latency of its requests, (2) a simple yet accurate request-level latency predictor using linear regression, and (3) a runtime power management system that meets the QoS constraints of LC applications, while maximizing the server's energy savings. Experimental results show that compared to the best state-of-the-art per-core power manager, ReTail achieves an average of 11% (up to 48%) energy savings, while at the same time meeting QoS. 

A case for in-network persistence

(Korakit Seemakhupt, UVA, presenting Wed. October 14 at 11:00 a.m. and 7:00 p.m. ET) 

To guarantee data persistence, storage workloads (such as databases, key-value stores, and file systems) typically use a synchronous protocol that puts network and server stack  latency on the critical path of request processing. The use of the fast and byte-addressable persistent memory (PM) has helped mitigate the storage overhead of the server stack;  yet, networking is still a dominant factor in the end-to-end latency of request processing. Emerging programmable network devices can reduce network latency by moving parts of the applications’ compute into the network (e.g., caching results for read requests); however, for update requests, the client still has to stall on the server to commit the updates, persistently.

In this work, we introduce in-network data persistence that extends the data-persistence domain from servers to the network, and present PMNet, a network device (e.g., switch or NIC) with PM for persisting data in-network. PMNet logs incoming update requests and acknowledges clients directly without having them to wait on the server to commit the request. In case of a failure, the logged requests act as redo logs for the server to recover. We implement PMNet using an FPGA and evaluate its performance against PM-optimized key-value stores and a database. Our evaluation shows that PMNet can improve the throughput of update requests by 4.27x on average, and the 99th-percentile tail latency by 3.23x.