INFaaS: Model-less Inference Serving

(Francisco Romero and Qian Li presenting on Wednesday, April 1 at 11:00 AM and 7:00 PM Eastern Time)

Despite existing work in machine learning inference serving, ease-of-use and cost efficiency remain key challenges. Developers must manually match the performance, accuracy, and cost constraints of their applications to decisions about selecting the right model and model optimizations, suitable hardware architectures, and auto-scaling configurations. These interacting decisions are difficult to make for developers, especially when the application load varies, applications evolve, and the available resources vary over time. Consequently, applications often end up overprovisioning resources.

In this talk, we will introduce INFaaS, a model-less inference-as-a-service system that relieves applications of making these decisions. INFaaS provides a simple interface allowing applications to specify their inference task, and performance and accuracy requirements. To implement this interface, INFaaS generates and leverages model-variants, versions of already trained models that differ in resource footprints, latencies, costs, and accuracies. Based on the characteristics of the model-variants, INFaaS automatically navigates the decision space on behalf of applications to meet their specific objectives: (a) it selects a model, hardware architecture, and any compiler optimizations, and (b) it makes scaling and resource allocation decisions. By sharing hardware resources across models and applications, INFaaS achieves up to 150× cost savings, 1.5× higher throughput, and violates latency objectives 1.5× less frequently, compared to state-of-the-art systems.

Scaling-In General Purpose Computing within the DRAM Hierarchy for Map-Reduce Workloads

(Siddhartha Balakrishna Rai is presenting on Wed. 3/25/20) 
This talk is a design space exploration of the hardware (where? how many? how to interface?) and software (how to place data? how to map computations?) choices for placing RISCV cores within the rank, chip, and bank of the DIMM slots in the DRAM hierarchy to take advantage of the locality vs. parallelism trade-offs for speeding up Map-Reduce workloads.

Cross-Failure Bug Detection in Persistent Memory Programs

(Sihang Liu presenting Wed. 3/11/2020 at 11:00 AM and 7:00 PM Eastern Time.)

Persistent memory (PM) technologies, such as Intel’s Optane memory, deliver high performance, byte-addressability, and persistence, allowing programs to directly manipulate persistent data in memory without any OS intermediaries. An important requirement of these programs is that persistent data must remain consistent across a failure, which we refer to as the crash consistency guarantee. 



However, maintaining crash consistency is not trivial. We identify that a consistent recovery critically depends not only on the execution before the failure, but also on the recovery and resumption after failure. We refer to these stages as the pre- and post-failure execution stages. In order to holistically detect crash consistency bugs, we categorize the underlying causes behind inconsistent recovery due to incorrect interactions between the pre- and post-failure execution. First, a program is not crash-consistent if the post-failure stage reads from locations that are not guaranteed to be persisted in all possible access interleavings during the pre-failure stage — a type of programming error that leads to a race that we refer to as a cross-failure race. Second, a program is not crash-consistent if the post-failure stage reads persistent data that has been left semantically inconsistent during the pre-failure stage, such as a stale log or uncommitted data. We refer to this type of bugs as a cross-failure semantic bug. Together, they form the cross-failure bugs in PM programs. In this work, we provide XFDetector, a tool that detects cross-failure bugs by automatically injecting failures into the pre-failure execution, and checking for cross-failure races and semantic bugs in the post-failure continuation. XFDetector has detected four new bugs in three pieces of PM software: one of PMDK’s examples, a PM-optimized Redis database, and a PMDK library function.   

DIBS: Database Isolation By Scheduling

(Kevin Gaffney, Univ. Wisconsin-Madison, is presenting on Wed. 2/5/20 at 11:00AM & 7:00PM ET)

Preventing concurrent transactions from interfering with each other is a performance challenge in modern relational database systems. The majority of transaction isolation systems achieve isolation through either mid-execution or post-execution validation. As a result, transaction managers are often complex and require developing entire systems around them. We show that database isolation can be guaranteed by analyzing and scheduling declarative transactions outside of the database using pre-execution validation. We provide an implementation that does so with no knowledge of the database system’s implementation or state, achieving competitive performance on transaction processing benchmarks.

Deep Learning Acceleration with Neuron-to-Memory Transformation

Description:
(Yeseong Kim, UCSD, presenting at 11:00AM and 7:00PM Eastern Time on Wednesday, January 22, 2020)

Abstract: 

In this talk, I will discuss our framework for deep neural network (DNN) acceleration, called RAPIDNN, which performs neuron-to-memory transformation for a highly-parallel, memory-centric architecture. RAPIDNN reinterprets a DNN model and maps it into a specialized accelerator, which is designed using non-volatile memory blocks that model four fundamental DNN operations. Our evaluation shows that RAPIDNN achieves 49.5× energy efficiency improvement and 10.9× speedup as compared to PipeLayer, a state-of-the-art DNN accelerator while ensuring less than 0.5% quality loss.

Graph Analytics Accelerator Supporting Sparse Data Representation using Crossbar Architectures

(Nagadastagiri Challapalle presenting on Wednesday, December 18, 2019 at 1:00PM ET)

Graph analytics applications are ubiquitous in this era of a connected world. These applications have very low compute to byte-transferred ratios and exhibit poor locality, which limits their computational efficiency on general purpose computing systems. Conventional hardware accelerators employ custom dataflow and memory hierarchy organization to over- come these challenges. Processing-in-memory (PIM) accelerators leverage massively parallel compute capable memory arrays to perform the in-situ operations on graph data or employ custom compute elements near the memory to leverage larger internal bandwidths. In this work, we present GaaS-X, a graph analytics accelerator that inherently supports the sparse graph data representations using an in-situ compute-enabled crossbar memory architectures. We alleviate the overheads of redundant writes, sparse to dense conversions, and redundant computations on the invalid edges that are present in the state of the art crossbar-based PIM accelerators. GaaS-X achieves 7.7× and 2.4× performance and 22× and 5.7×, energy savings, respectively, over two state-of-the-art crossbar accelerators and offers orders of magnitude improvements over GPU and CPU solutions.

MEG: A RISCV-Based System Simulation Infrastructure for Exploring Memory Optimization Using FPGAs and Hybrid Memory Cube

Emerging 3D memory technologies, such as the Hybrid Memory Cube (HMC) and High Bandwidth Memory (HBM), provide increased bandwidth and massive memory-level parallelism. Efficiently integrating emerging memories into existing systems poses new challenges and require detailed evaluation in a real computing environment. In this paper, we propose MEG, an open-source, configurable, cycle-exact, and RISC-V based full system simulation infrastructure using FPGA and HMC. MEG has three highly configurable design components: (i) an HMC adaptation module that not only enables communication between the HMC device and the processor cores but also can be extended to fit other memories (e.g., HBM, nonvolatile memory) with minimal effort, (ii) a reconfigurable memory controller along with its OS support that can be effectively leveraged by system designers to perform software-hardware co-optimization, and (iii) a performance monitor module that effectively improves the observability and debuggability of the system to guide performance optimization. We provide a prototype implementation of MEG on Xilinx VCU110 board and demonstrate its capability, fidelity, and flexibility on real-world benchmark applications. We hope that our open-source release of MEG fills a gap in the space of publicly-available FPGA-based full system simulation infrastructures specifically targeting memory system and inspires further collaborative software/hardware innovations.