MEG1.1: A RISCV-based System Simulation Infrastructure for Exploring Memory Optimization using FPGAs and High Bandwidth Memory

(Nicholas Beckwith, U Penn., presenting on Wednesday, April 29, 2020)

In this presentation, we propose MEG1.1, a configurable, cycle-exact, and RISC-V based full system emulation infrastructure using FPGA and HBM. MEG1.1 extends MEG1.0 by providing out-of-order RISC-V cores as well as OS and architectural support to help integrate the user’s customized accelerators. Furthermore, MEG1.1 provides an HBM memory interface to fully expose the HBM’s bandwidth to the user. Leveraging MEG1.1, we present a cross-layer system optimization as an illustrative case to demonstrate the usability of MEG1.1. In this case study, we present a reconfigurable memory controller to improve the address mapping of a standard memory controller. This reconfigurable memory controller, along with its OS support, allows the user to improve the memory bandwidth accessible to the out-of-order RISC-V cores as well as to the custom near-memory accelerators. We also present the challenges and research directions for MEG 2.0 that can significantly reduce the cost and improve the portability, flexibility, usability of MEG 1.0 and 1.1 without sacrificing the performance and fidelity.

BaM: Enabling Accelerator Memory Accesses into the SSD

(Zaid Qureshi, David Min, and Vikram Sharma Mailthody presenting 4/22/20.) 

Storage class memories (SCM) have been considered as a prime candidate to address the growing need for applications’ memory footprint. An ideal SCM for tomorrow’s data center has TBs of memory capacity, has a few hundred nanoseconds to a couple of microsecond latency, is energy efficient, offers high memory parallelism, is scalable and is very cheap. Among the several types of SCM, 3DX-point and Flash have shown promising results.  Compared to 3DX-point, Flash offers higher throughput, thanks to several levels of parallelism, has higher density, consumes very low power per memory access, and is proven to be scalable and cost-efficient.

However, studying Flash as part of the main memory system is challenging as the existing simulators and emulators do not provide the needed flexibility and cannot address practical system-level challenges. In this talk, we will discuss our attempt in modeling an SSD using an FPGA. We show that CPUs offer a very low amount of memory parallelism over PCIe and are inefficient in exploiting the massive parallelism offered by these emerging NVM devices. To increase the memory level parallelism, we connect a GPU with an FPGA. We shall then discuss our learnings and several applications and system-level challenges we encountered.

INFaaS: Model-less Inference Serving

(Francisco Romero and Qian Li presenting on Wednesday, April 1 at 11:00 AM and 7:00 PM Eastern Time)

Despite existing work in machine learning inference serving, ease-of-use and cost efficiency remain key challenges. Developers must manually match the performance, accuracy, and cost constraints of their applications to decisions about selecting the right model and model optimizations, suitable hardware architectures, and auto-scaling configurations. These interacting decisions are difficult to make for developers, especially when the application load varies, applications evolve, and the available resources vary over time. Consequently, applications often end up overprovisioning resources.

In this talk, we will introduce INFaaS, a model-less inference-as-a-service system that relieves applications of making these decisions. INFaaS provides a simple interface allowing applications to specify their inference task, and performance and accuracy requirements. To implement this interface, INFaaS generates and leverages model-variants, versions of already trained models that differ in resource footprints, latencies, costs, and accuracies. Based on the characteristics of the model-variants, INFaaS automatically navigates the decision space on behalf of applications to meet their specific objectives: (a) it selects a model, hardware architecture, and any compiler optimizations, and (b) it makes scaling and resource allocation decisions. By sharing hardware resources across models and applications, INFaaS achieves up to 150× cost savings, 1.5× higher throughput, and violates latency objectives 1.5× less frequently, compared to state-of-the-art systems.

Scaling-In General Purpose Computing within the DRAM Hierarchy for Map-Reduce Workloads

(Siddhartha Balakrishna Rai is presenting on Wed. 3/25/20) 
This talk is a design space exploration of the hardware (where? how many? how to interface?) and software (how to place data? how to map computations?) choices for placing RISCV cores within the rank, chip, and bank of the DIMM slots in the DRAM hierarchy to take advantage of the locality vs. parallelism trade-offs for speeding up Map-Reduce workloads.

Cross-Failure Bug Detection in Persistent Memory Programs

(Sihang Liu presenting Wed. 3/11/2020 at 11:00 AM and 7:00 PM Eastern Time.)

Persistent memory (PM) technologies, such as Intel’s Optane memory, deliver high performance, byte-addressability, and persistence, allowing programs to directly manipulate persistent data in memory without any OS intermediaries. An important requirement of these programs is that persistent data must remain consistent across a failure, which we refer to as the crash consistency guarantee. 



However, maintaining crash consistency is not trivial. We identify that a consistent recovery critically depends not only on the execution before the failure, but also on the recovery and resumption after failure. We refer to these stages as the pre- and post-failure execution stages. In order to holistically detect crash consistency bugs, we categorize the underlying causes behind inconsistent recovery due to incorrect interactions between the pre- and post-failure execution. First, a program is not crash-consistent if the post-failure stage reads from locations that are not guaranteed to be persisted in all possible access interleavings during the pre-failure stage — a type of programming error that leads to a race that we refer to as a cross-failure race. Second, a program is not crash-consistent if the post-failure stage reads persistent data that has been left semantically inconsistent during the pre-failure stage, such as a stale log or uncommitted data. We refer to this type of bugs as a cross-failure semantic bug. Together, they form the cross-failure bugs in PM programs. In this work, we provide XFDetector, a tool that detects cross-failure bugs by automatically injecting failures into the pre-failure execution, and checking for cross-failure races and semantic bugs in the post-failure continuation. XFDetector has detected four new bugs in three pieces of PM software: one of PMDK’s examples, a PM-optimized Redis database, and a PMDK library function.   

DIBS: Database Isolation By Scheduling

(Kevin Gaffney, Univ. Wisconsin-Madison, is presenting on Wed. 2/5/20 at 11:00AM & 7:00PM ET)

Preventing concurrent transactions from interfering with each other is a performance challenge in modern relational database systems. The majority of transaction isolation systems achieve isolation through either mid-execution or post-execution validation. As a result, transaction managers are often complex and require developing entire systems around them. We show that database isolation can be guaranteed by analyzing and scheduling declarative transactions outside of the database using pre-execution validation. We provide an implementation that does so with no knowledge of the database system’s implementation or state, achieving competitive performance on transaction processing benchmarks.

Deep Learning Acceleration with Neuron-to-Memory Transformation

Description:
(Yeseong Kim, UCSD, presenting at 11:00AM and 7:00PM Eastern Time on Wednesday, January 22, 2020)

Abstract: 

In this talk, I will discuss our framework for deep neural network (DNN) acceleration, called RAPIDNN, which performs neuron-to-memory transformation for a highly-parallel, memory-centric architecture. RAPIDNN reinterprets a DNN model and maps it into a specialized accelerator, which is designed using non-volatile memory blocks that model four fundamental DNN operations. Our evaluation shows that RAPIDNN achieves 49.5× energy efficiency improvement and 10.9× speedup as compared to PipeLayer, a state-of-the-art DNN accelerator while ensuring less than 0.5% quality loss.