Hopscotch: A Microbenchmark for Memory Performance Evaluation

(Presenting Friday, Feb 01, 2019) Due to the ever-increasing gap between the speed of computing elements and the speed at which memory systems can provide data, we are hitting a memory wall. As CRISP center is coming up with new memory architectures to fight this issue, we need good memory benchmarks to evaluate the performance of these emerging architectures. Ideally, these benchmarks should be able to help us finding memory system bottlenecks, should be able to profile read and write channels independently and combined, and should be tunable to match application of interest. Existing memory benchmarks such as STREAM, ApexMAP, Spatter etc. evaluate memory performance using interesting access patterns. However, they are not flexible enough to allow evaluating with all combinations of read and write patterns. We are developing a micro-benchmark where we can tune the access pattern of read and write channels independently using parameters controlling spatial and temporal locality. The benchmark will include a collection of kernel for exercising different areas of the memory system. To ensure that zero redundancy, we are going to employ diversity analysis using metrics like stall percentage, read/write ratio, median stride length, unit stride percentage etc. The current implementation supports few interesting read and write patterns on CPU and GPU platforms using OpenMP and CUDA. Future extension plan includes exploring the effect of memory controller scheduling and finding new patterns in major application domains by eliminating the patterns we already covered.

A little latency goes a long way: Memory latency and its impact on CPU-driven inference performance

(Ameen Akel of Micron is presenting Mon. Feb 4, 2019)

Memory media latencies are often a hot topic: Systems architects often seek the lowest possible latencies, which, in turn, drives memory manufacturers to architect memories for the minimum possible access latencies.  While some applications may benefit from lower media latencies, most applications exhibit very little sensitivity.  We aim to dispel the memory latency myth: Workloads like DNN inference, across a wide variety of models, are not strongly correlated to memory media latency.  Freeing memory companies of memory latency shackles enables favorable memory media architecture tradeoffs that systems architects may not expect.

Hierarchical and Distributed Machine Learning Inference Beyond the Edge

(Presenting Wednesday Jan 30, 2019) Networked applications with heterogeneous sensors are a growing source of data in the Internet of Things (IoT) environment. Many IoT applications use machine learning (ML) to make real-time predictions. The current dominant approach to deploying ML inference is monolithic, i.e., when inference needs to be performed using data generated by multiple sensors, the features generated by each sensor are joined in a centralized cloud-based tier to perform the inference computation. Since inference typically occurs with high frequency, the monolithic approach can quickly lead to burdensome levels of communication, which wastes energy, reduces data privacy, and often bottlenecks the network violating real-time constraints. In this work, we study a novel approach that mitigates these issues by “pushing” ML inference computations out of the cloud and onto a hierarchy of IoT devices, which compute successively more compressed representations of raw sensor data. We present a new technical challenge of “rewriting” the functional form of an ML inference computation to factor it over a network of devices without significantly reducing prediction accuracy. We present novel “hierarchy aware” neural network architectures which enable users to trade off between communication cost and accuracy. We also present novel exact factoring algorithms for other popular ML models including gradient boosted trees and random forests that preserve accuracy while substantially reducing communication. We evaluate our approach with three real-world problems, urban energy demand prediction, human activity prediction, and server performance prediction. Our approach presents substantial reductions in energy use and latency on IoT devices, while providing the same level of prediction quality to the current monolithic inference. Measurement on a common IoT device shows that energy use and latency can be reduced by up to 63% and 67% respectively without reducing accuracy relative to the full communication setting.

PIMProf: A Performance Profiler for Processing-in-Memory Architectures

Monday, 1/28/19 at 2:00PM ET – Task 1.5 (Evaluation Through Architectural Simulation and Prototyping)

PIM architectures have drawn an increasing research interest as a mitigation to the data movement bottleneck within the current DRAM-based architectures, and a variety of them have been proposed for accelerating various data-intensive workloads. However, for a given workload, it is difficult to determine which part of a program should be offloaded to a given PIM architecture to gain the best performance and how much performance gain is possible. We propose PIMProf, a tool that uses a combination of static and runtime analysis to automatically detect the PIM candidates and estimate the speedup of the program. Our key ideas are as follows: First, PIMProf uses static analysis to capture the dependency between computation and data access, and constructs both a control flow graph and a data dependency graph of the program. Second, PIMProf profiles the computation cost and memory access cost of the program, and attributes the costs to the nodes and edges of the graph. Finally, we show how to formalize the PIM offloading problem into a cost minimization problem of the weighted graph.

Monolithic-3D Integration Augmented Design Techniques for Computing in SRAMs

Presenting Friday, January 25th, at 2PM EST

Task 3.6: Cognitive Architectures

In-memory computing has emerged as a promising solution to address the logic-memory performance gap. We propose design techniques using monolithic-3D integration to achieve reliable multi-row activation which in turn help in computation as part of data readout. Our design is 1.8x faster than the existing techniques for Boolean computations. We quantitatively show no impact to cell stability when multiple rows are activated and thereby requiring no extra hardware for maintaining the cell stability during computations. In-memory digital to analog conversion technique is proposed using a 3D-CAM primitive. The design utilizes relatively low strength layer-2 transistors effectively and provides 7x power savings when compared with a specialized converter in-memory. Lastly, we present a linear classifier system by making use the above-mentioned techniques which is 47x faster while computing vector matrix multiplication using a dedicated hardware engine.

Wearout and active accelerated recovery for processing in emerging technology memories

(Presenting Wednesday, January 23rd, at 2pm EST) Trying to break the memory wall includes various efforts to bring memory closer to the processor, or to push processing into the memory stack, generally called Processing in Memory (PIM). If the memory is one the various "flavors" of non-volatile emerging memory technologies such as: spin-transfer torque RAM (STTRAM), phase-change memory (PCM), resistive RAM (RRAM), memristor, 3DXpoint, etc., limited endurance becomes an important issue in addition to all other general challenges common to PIM. Endurance refers to the fact that most non-volatile memories, including all these new emerging technologies, but also more traditional ones like Flash and EEPROM, have a limited lifetime in terms of how many times they can be written and erased - once the limit is exceeded, the number of faults in the memory increases rapidly and the memory device can no longer be used reliably. For storage applications (the main use of non-volatile memories until now) one way to deal with the limited endurance is to overprovision the device (i.e. leave some of the native capacity unutilized up-front and allocate it later as memory blocks start failing due to the limited endurance) and to use wear leveling by adding a level of memory virtualization in the form of a Flash Translation Layer (FTL) that maps logical blocks to physical blocks in a dynamic way such that the write/erase cycles are more or less equally distributed across the physical blocks such that no single block gets overwritten too many times in a row. Although the concept of an FTL was first introduced for Flash, similar mechanisms will work (and likely be necessary) for all emerging memory technologies with limited endurance (e.g. although not explicitly stated it is likely that the Intel Optane 3D XPoint is using a similar mechanism for wear-leveling, etc.). FTLs are an OK solution for storage applications but are suboptimal (to say the least) for main memory applications, and even more so for processing in memory, both from a latency point of view during normal logical-to-physical mapping, but especially because of the extra long delays necessary for moving data when a logical block needs to be re-allocated to a new physical block. Because of this methods that intrinsically compensate for the limited endurance would be especially preferable for PIM.

One such method is to take advantage of the recovery mechanisms associated with the stress that leads to the limited endurance in the first place. Since stress and wearout are mechanisms that take a device out of physical equilibrium, it turns out that simple thermodynamics tends to partially reverse the effect of stress when the stress is removed. This is a general physics argument that has been experimentally demonstrated for several wearout mechanisms, including Flash wearout, but also other more general ones, such as NBTI/PBTI, hot electrons, electromigration, etc. In this talk I will go over several of these mechanisms and explain the source of stress and ways to reverse it. The main idea is to go beyond simple passive recovery (just remove stress and wait) by reversing the direction of stress (active recovery) and accelerate the process (e.g. by increasing temperature). Such accelerated active recovery can lead to many orders of magnitude improvement in endurance, thus making processing in emerging technology memories practical.

Deep Learning for Pancreatic Cancer Histopathology Image Analysis

(Adib Keikhosravi / Kevin Eliceiri Presenting Fri. 1/18/19) 

Whole slide imaging (WSI) or virtual microscopy is a type of imaging modality which is used to convert animal or human pathology tissue slides to digital images for teaching, research or clinical applications. This method is popular due to education and clinical demands. Although modern whole slide scanners can now scan tissue slides with high resolution in a relatively short period of time, significant challenges, including high cost of equipment and data storage, remain unsolved. Machine learning and deep learning techniques in Computer Aided Diagnosis (CAD) platforms have begun to be widely used for biomedical image analysis by physicians and researchers. We are trying to build a platform for histopathological image super-resolution and cancer grading and staging with the main focus on pancreatic cancer. We present a computational approach for improving the quality of the resolution of images acquired from commonly available low magnification commercial slide scanners. Images from such scanners can be acquired cheaply and are efficient in terms of storage and data transfer. However, they are generally of poorer quality than images from high-resolution scanners and microscopes and do not have the necessary resolution needed in diagnostic or clinical environments, and hence are not used in such settings. First, we developed a deep learning framework that implements regularized sparse coding to smoothly reconstruct high-resolution images, given their low-resolution counterpart. Results show that our method indeed produces images which are similar to images from high resolution scanners, both in quality and quantitative measures and compares favorably to several state-of-the-art methods across a number of test images. To further improve the results, we used a convolutional neural network (CNN) based approach, which is specifically trained to take low-resolution slide scanner images of cancer data and convert it into a high-resolution image. We validate these resolution improvements with computational analysis to show the enhanced images offer the same quantitative results. This project is still ongoing and now we are trying to use middle resolutions for improving the image quality using recurrent neural networks. On the other hand, current approaches for pathological grading/staging of many cancer types such as breast and pancreatic cancer lack accuracy and interobserver agreement. Google research recently used inception for high accuracy tumor cell localization. However, as our group has been discovering the prognostic role of stromal reorganization in different cancer types including pancreatic cancer, which is projected to be the second leading cause of cancer by 2030, we use a wholistic approach that contains both stroma and cell from small TMA punches of different grades of cancer accompanied by normal samples. For this study we used transfer learning from four award winning networks VGG16, VGG19, GoogleNet and Resnet101 for the task of pancreatic cancer grading. Although all these networks have shown great performance for natural image classifications, but Resnet showed the highest performance with 88% accuracy in four-tier grading, and higher for all one by one comparisons among normal and different grades. We fine-trained this network again for different TNM classification and staging tasks and although all the images were selected from small regions from pancreas, the results show the promising capability of CNNs in helping pathologists with diagnosis. To achieve higher accuracies we have almost doubled the size of the dataset and trainings are still running and will update the audience in future talks.