Look-Up Table based Energy Efficient Processing in SRAM for Neural Network Acceleration

(Akshay Krishna Ramanathan presenting on Wed. 7/29/2020 at 11:00 a.m. and 7:00 p.m. ET)

This work presents a Look-Up Table (LUT) based Processing-In-Memory (PIM) technique with the potential for running Neural Network inference tasks. We implement a bitline computing free technique to avoid frequent bitline accesses to the cache sub-arrays and thereby considerably reducing the memory access energy overhead. LUT in conjunction with the compute engines enables sub-array level parallelism while executing complex operations through data lookup which otherwise requires multiple cycles. Sub-array level parallelism and systolic input data flow ensure data movement to be confined to the SRAM slice.

Our proposed LUT based PIM methodology exploits substantial parallelism using look-up tables, which does not alter the memory structure/organization, that is, preserving the bit-cell and peripherals of the existing SRAM monolithic arrays. Our solution achieves 1.72x higher performance and 3.14x lower energy as compared to a state of the art processing-in-cache solution. Sub-array level design modifications to in-corporate LUT along with the compute engines will increase the overall cache area by 5.6%. We achieve 3.97x speedup w.r.t neural network systolic accelerator with a similar area. The re-configurable nature of the compute engines enables various neural network operations and thereby supporting sequential networks (RNNs) and transformer models. Our quantitative analysis demonstrates 101x, 3x faster execution and 91x, 11x energy efficiency than CPU and GPU respectively while running the transformer model, BERT-Base.

Video Analytic Platform and Deep Graph Matching

Feng Shi and Ziheng Xu, UCLA, presenting on July 22, 2020 at 11:00 a.m. and 7:00 p.m. ET)

As a fundamental problem in pattern recognition, graph matching has applications in a variety of fields, especially for computer vision. The utilization of graph matching in video analytics include (but not limited) scene understanding, object keypoints matching, checking the availability of scene generation, etc.. The graph similarity (or matching) problem attempts to find node correspondences between graphs. Traditionally, obtaining an exact solution with heuristic algorithms is NP-hard and spends long latency; recently, the research employing deep graph architectures has been proposed in finding an approximate solution, which leverages the speed and accuracy. Previous works in such research mainly focused on using localized node embeddings to obtain an approximate node alignment. However, investigating only local features cannot reflect the whole structure; the overall graph topology plays a vital role in determining the edge alignment between graphs. Diffusion wavelets, which depict the probability distributions of graph signals over a graph, are powerful tools to assist in the graph topology exploration. In this work, we present WaveSim, a light-weight deep graph matching framework incorporating graph diffusion wavelets to calculate the diffusion distance. We also mathematically prove that it is possible to transform a Quadratic Assignment Programming (QAP) problem with high-order combinatorial nature into a lower dimension problem. Experiments show that WaveSim achieves remarkable and robust performances, and can be extended to matching problems with large graphs.

GCN meets GPU: Decoupling “When to Sample” from “How to Sample"

(Morteza Ramezani, Penn State, presenting on Wed. 7/15/20)

Graphs are powerful and versatile data structures to model many real world problems and learning on graph-based data is attracting growing interest in both academia and industry. Recently, many models have been introduced for learning on graphs. Among these, Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning on graphs. However, due to node dependency in graphs, which leads to "neighborhood explosion" , training GCN for large graphs on GPU is not practical. To address this problem, sampling-based methods have been introduced to couple with stochastic gradient descent for training GCNs. 

While effective in alleviating the neighborhood explosion, due to bandwidth and memory bottlenecks, these methods lead to computational overheads in preprocessing and loading new samples in heterogeneous systems (CPU-GPU), which significantly deteriorate the sampling performance. By decoupling the frequency of sampling from the sampling strategy, we propose LazyGCN, a general yet effective framework that can be integrated with any sampling strategy to substantially improve the training time. The idea behind LazyGCN is to perform sampling periodically and effectively recycle the sampled nodes to mitigate data preparation overhead. We theoretically analyze the proposed algorithm and give corroborating empirical evidence on large real-world graphs, demonstrating that the proposed schema can significantly reduce the number of sampling steps and yield superior speedup without compromising the accuracy.

Mirage: A Highly Parallel And Flexible RRAM Accelerator

(Xiao Liu, UCSD, presenting on July 8, 2020 at 11:00 AM & 7:00 PM ET)

Emerging resistive memory (RRAM) based crossbar is a promising technology to accelerate neural network applications.
Such a structure can support massively parallel multiply-accumulate operations, which are intensively used in convolutional neural networks (CNN).
The novel structure is demonstrated to offer higher performance and power efficiency than the CMOS-based accelerators.
However, previously proposed RRAM-based neural network designs lack several desirable features for neural network accelerators.
First, the pipeline of existing architecture is inefficient. 
as data dependency between different layers of the network significantly can stall the execution.
Second, existing RRAM-based accelerators suffer from limited flexibility.
The oversized networks are not able to be executed on the accelerator, while the undersized networks are not able to utilize all the RRAM crossbar arrays.

To address these issues, we propose Mirage, a novel architectural design to enable high parallelism and flexibility for RRAM-based CNN accelerators.
Mirage consists of a Fine-grained Parallel RRAM Architecture (FPRA) and Auto Assignment (AA).
Motivated by thread block design in the GPU, FPRA addresses the data dependency issue in the pipeline.
When inter layer parallelism is involved, FPRA unifies the data dependency of each layer and handles them with shared input and output memory.
AA provides the ability to execute any-sized network on the accelerator.
When the network is oversized, AA utilizes dynamic reconfiguration to fold the network to fit with available hardware.
When the network is undersized, AA utilizes the FPRA to maximize the use of the extra hardware for higher performance.

We evaluate Mirage on seven popular image recognition neural network models with various network sizes. 

We find that Mirage manages to achieve 2.0x average speedup compares to the state-of-the-art RRAM-based accelerator. 

Additionally, Mirage can adopt network into RRAM-based accelerators of various sizes and we show that Mirage can deliver better performance scalability over prior works.

Efficient and Reliable Digital Processing in-Memory Acceleration

(Presented by Minxuan Zhou, UCSD, on June 24th.)

Digital processing in-memory (PIM) is a promising technology that minimizes data movement overhead while enabling an extremely high degree of parallelism. These characteristics provide a great opportunity for accelerating emerging big-data applications.  Such optimization requires that both software and hardware be considered in order to get an efficient and reliable solution. In this talk we first illustrate our software-hardware co-design methodology for digital PIM by using an example of graph processing workloads. We next introduce several new PIM-specific methods that improve the reliability of the system by efficiently managing its thermal issues.   Our results show that it is possible to get orders of magnitude improvement in speed over the state of the art graph processing algorithms, while ensuring thermal and reliability constraints are met.

AI-PiM – PiM accelerators for AI applications for RISC-V

(Vaibhav Varma, UVA, presenting on Wed. 6/17/2020)

Artificial intelligence (AI) and machine learning (ML) have emerged as the fastest growing workloads in the last few years ranging from applications like object detection and face recognition to self-driving smart cars. This rise in AI applications combined with IoT infrastructure is leading to a new paradigm of Artificial Intelligence of Things (AIoT) where IoT edge devices are augmented with AI/ML capabilities to enable smart sensing applications. This has fuelled an increased interest towards integrating AI accelerators in edge devices and Processing-in-Memory (PiM) accelerators are prime candidates for this integration. PiM accelerators promise improved performance and power characteristics by breaking the memory wall but they are notoriously difficult to program which resists their integration in the traditional computing stack. In this talk, we present AI-PiM as a solution to this problem. AI-PiM is a hardware/software codesign methodology which helps with efficient integration of PiM accelerators in the RISC-V processor pipeline as functional units. Along with hardware integration, AI-PiM also focusses on RISC-V ISA extensions which target the PiM functional units directly, resulting in a tight integration of PiM accelerators with the processor.

Sieve: Scalable In-situ DRAM-based Accelerator Designs for Massively Parallel k-mer Matching

(Lingxi Wu presenting on May 27, 2020 at 11:00 a.m. and 7:00 p.m.)

The rapid influx of biosequence data, coupled with the stagnation of the processing power of conventional computing systems, highlight the critical need for exploring high-performance accelerator designs that can meet the ever-increasing throughput demands of modern bioinformatics pipelines.  This work argues that processing in memory (PIM) is a viable and effective solution to alleviate the bottleneck of k-mer matching, a widely used genome sequence comparison and classification algorithm, characterized by highly random access patterns and low computational intensity.

This work proposes and evaluates three DRAM-based in-situ k-mer matching accelerator designs (one optimized for area, one optimized for throughput, and one that strikes a balance between hardware cost and performance), dubbed Sieve, that leverage a novel data mapping scheme to allow for simultaneous comparisons of millions of DNA base pairs, lightweight matching circuitry for fast pattern matching, and an early termination mechanism that prunes unnecessary DRAM row activation to reduce latency and save energy.  Evaluation of Sieve using state-of-the-art workloads with real-world datasets shows that the most aggressive design provides an average of 408x/41x speedup and 93X/61x energy savings over multi-core-CPU/GPU baselines for k-mer matching. Sieve's performance scales linearly with the reference sequence data, substantially boosting the efficiency of modern genome sequence pipelines.