The era of ?big data? is leading to changes in the compute paradigm, in particular to the notion of moving computation to data, known as Near Data Processing (NDP). Technological advancements have enabled the application of NDP at many levels of the memory hierarchy from cache to DRAM, from non-volatile storage-class memory to processors embedded in storage devices. This dissertation explores the effectiveness of data-centric compute architectures using Active Storage, Processing-in-Memory and

more »

... oherent Access Processor Interface (CAPI) accelerated Flash storage. We developed a compute framework Active In-Storage (AiSTOR) that enables scalable distributed Big Data Processing by directly performing the computations on active storage devices. AiSTOR has the following three major advantages: (i) active storage utilizes the processor capabilities on the storage devices and this significantly reducing the bandwidth requirement of the network, (ii) the computations can take advantage of the inherent map/reduce parallelism by using the array of the distributed storage processors available on the active data storage devices, thereby aggregating the processing power of a cluster of active devices, (iii) it can perform coherent processing of streaming data as it arrives on the storage devices. We define a generic NDP architecture which is well-suited for memory-bound computations and implement the software kernels for NDP-based algorithmic mapping.We show for a modest sized NDP system, that the AiSTOR architecture framework employing distributed processing algorithms can yield efficient and accurate computational processing performance. In comparison with Hadoop based MapReduce, the compute times on AiSTOR has significant performance benefits by up to 18%, while providing very competitive results compared to Spark-based in-memory processing. The effectiveness of the NDP architecture is demonstrated by evaluating the row-buffer management policies (open-page and closed-page) with the controller modifications and a generic un [...]