‹ Back to Video Series: Systems Track

Accelerate Big Data Processing with High-Performance Computing Technologies

  • Overview
  • Resources

Speakers: D. K. Panda and Xiaoyi Lu, Ohio State University

This talk discusses opportunities and challenges with accelerating big data middleware on modern high-performance computing (HPC) clusters while fully using HPC technologies. Using the publicly available software packages in the High-Performance Big Data (HiBD) project, we provide case studies and benefits of new designs for several Apache Hadoop*, Apache Spark*, and Memcached components. We also examine the interplay between high-performance interconnects, storage systems—such as hard disk drive (HDD) and solid-state drive (SSD)—and multicore platforms to achieve the best solutions.