Programming Models for Parallel Computing (Paperback)

With the coming of the parallel computing era, computer scientists have turned their attention to designing programming models that are suited for high-performance parallel computing and supercomputing systems. Programming parallel systems is complicated by the fact that multiple processing units are simultaneously computing and moving data. This book offers an overview of some of the most prominent parallel programming models used in high-performance computing and supercomputing systems today.

The chapters describe the programming models in a unique tutorial style rather than using the formal approach taken in the research literature. The aim is to cover a wide range of parallel programming models, enabling the reader to understand what each has to offer. The book begins with a description of the Message Passing Interface (MPI), the most common parallel programming model for distributed memory computing. It goes on to cover one-sided communication models, ranging from low-level runtime libraries (GASNet, OpenSHMEM) to high-level programming models (UPC, GA, Chapel); task-oriented programming models (Charm++, ADLB, Scioto, Swift, CnC) that allow users to describe their computation and data units as tasks so that the runtime system can manage computation and data movement as necessary; and parallel programming models intended for on-node parallelism in the context of multicore architecture or attached accelerators (OpenMP, Cilk Plus, TBB, CUDA, OpenCL). The book will be a valuable resource for graduate students, researchers, and any scientist who works with data sets and large computations.

ContributorsTimothy Armstrong, Michael G. Burke, Ralph Butler, Bradford L. Chamberlain, Sunita Chandrasekaran, Barbara Chapman, Jeff Daily, James Dinan, Deepak Eachempati, Ian T. Foster, William D. Gropp, Paul Hargrove, Wen-mei Hwu, Nikhil Jain, Laxmikant Kale, David Kirk, Kath Knobe, Ariram Krishnamoorthy, Jeffery A. Kuehn, Alexey Kukanov, Charles E. Leiserson, Jonathan Lifflander, Ewing Lusk, Tim Mattson, Bruce Palmer, Steven C. Pieper, Stephen W. Poole, Arch D. Robison, Frank Schlimbach, Rajeev Thakur, Abhinav Vishnu, Justin M. Wozniak, Michael Wilde, Kathy Yelick, Yili Zheng

隨著並行計算時代的到來，計算機科學家們開始關注設計適用於高性能並行計算和超級計算系統的編程模型。多處理單元同時計算和移動數據使得編程並行系統變得複雜。本書概述了當今高性能計算和超級計算系統中使用的一些著名並行編程模型。

本書以獨特的教程風格描述了這些編程模型，而不是使用研究文獻中的正式方法。目的是涵蓋各種並行編程模型，使讀者能夠了解每個模型的特點。本書首先介紹了消息傳遞接口（MPI），這是分佈式內存計算中最常用的並行編程模型。接著介紹了單向通信模型，從低級運行時庫（GASNet、OpenSHMEM）到高級編程模型（UPC、GA、Chapel）；任務導向的編程模型（Charm++、ADLB、Scioto、Swift、CnC），允許用戶將計算和數據單元描述為任務，以便運行時系統根據需要管理計算和數據移動；以及針對多核架構或附加加速器的節點內並行性的並行編程模型（OpenMP、Cilk Plus、TBB、CUDA、OpenCL）。本書將是研究生、研究人員和任何處理數據集和大型計算的科學家的寶貴資源。

貢獻者：Timothy Armstrong、Michael G. Burke、Ralph Butler、Bradford L. Chamberlain、Sunita Chandrasekaran、Barbara Chapman、Jeff Daily、James Dinan、Deepak Eachempati、Ian T. Foster、William D. Gropp、Paul Hargrove、Wen-mei Hwu、Nikhil Jain、Laxmikant Kale、David Kirk、Kath Knobe、Ariram Krishnamoorthy、Jeffery A. Kuehn、Alexey Kukanov、Charles E. Leiserson、Jonathan Lifflander、Ewing Lusk、Tim Mattson、Bruce Palmer、Steven C. Pieper、Stephen W. Poole、Arch D. Robison、Frank Schlimbach、Rajeev Thakur、Abhinav Vishnu、Justin M. Wozniak、Michael Wilde、Kathy Yelick、Yili Zheng