close
Skip to main content
Springer Nature Link
Log in

A vision of post-exascale programming

  • Perspective
  • Published:
Save article
View saved research
Frontiers of Information Technology & Electronic Engineering Aims and scope

Abstract

Exascale systems have been under development for quite some time and will be available for use in a few years. It is time to think about future post-exascale systems. There are many main challenges with regard to future post-exascale systems, such as processor architecture, programming, storage, and interconnect. In this study, we discuss three significant programming challenges for future post-exascale systems: heterogeneity, parallelism, and fault tolerance. Based on our experience of programming on current large-scale systems, we propose several potential solutions for these challenges. Nevertheless, more research efforts are needed to solve these problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  • Bahmani A, Mueller F, 2014. Scalable performance analysis of exascale MPI programs through signature–based clustering algorithms. 28th ACM Int Conf on Supercomputing, p.155–164. https://doi.org/10.1145/2597652.2597676

    Book  Google Scholar 

  • Balaji P, Snir M, Amer A, et al., 2013. Exascale MPI. https://www.exascaleproject.org/project/exascalempi/[Accessed on Sept. 10, 2018].

    Google Scholar 

  • Bland W, Du P, Bouteiller A, et al., 2012. A checkpointon–failure protocol for algorithm–based recovery in standard MPI. European Conf on Parallel Processing, p.477–488. https://doi.org/10.1007/978–3–642–32820–6.48

    Google Scholar 

  • Bland W, Bouteiller A, Herault T, et al., 2013. Post–failure recovery of MPI communication capability: design and rationale. Int J High Perform Comput Appl, 27(3):244–254. https://doi.org/10.1177.1094342013488238

    Article  Google Scholar 

  • Bouteiller A, Cappello F, Herault T, et al., 2003. MPICHV2: a fault tolerant MPI for volatile nodes based on pessimistic sender based message logging. ACM/IEEE Conf on Supercomputing, p.1–17. https://doi.org/10.1145/1048935.1050176

    Google Scholar 

  • Cappello F, 2009. Fault tolerance in petascale/exascale systems: current knowledge, challenges, and research opportunities. Int J High Perform Comput Appl, 23(3):212–226. https://doi.org/10.1177.1094342009106189

    Article  Google Scholar 

  • Chen Z, 2013. Online–ABFT: an online algorithm based fault tolerance scheme for soft error detection in iterative methods. ACM SIGPLAN Not, 48(8):167–176. https://doi.org/10.1145/2442516.2442533

    Article  Google Scholar 

  • Dagum L, Menon R, 1998. OpenMP: an industry standard API for shared–memory programming. IEEE Comput Sci Eng, 5(1):46–55. https://doi.org/10.1109/99.660313

    Article  Google Scholar 

  • Dean J, Ghemawat S, 2008. MapReduce: simplified data processing on large clusters. Commun ACM, 51(1):107–113. https://doi.org/10.1145/1327452.1327492

    Article  Google Scholar 

  • Dong X, Muralimanohar N, Jouppi N, et al., 2009. Leveraging 3D PCRAM technologies to reduce checkpoint overhead for future exascale systems. Int Conf on High Performance Computing Networking, Storage, and Analysis, p.1–12. https://doi.org/10.1145/1654059.1654117

    Book  Google Scholar 

  • Fu H, Liao J, Yang J, et al., 2016. The Sunway Taihulight supercomputer: system and applications. Sci Chin Inform Sci, 59(7):072001. https://doi.org/10.1007/s11432–016–5588.7

    Article  Google Scholar 

  • Gropp W, 2009. MPI at exascale: challenges for data structures and algorithms. In: Ropo M, Westerholm J, Dongarra J (Eds.), Recent Advances in Parallel Virtual Machine and Message Passing Interface. Springer Berlin Heidelberg. https://doi.org/10.1007/978–3–642–03770–2.3

  • Huang KH, Abraham JA, 1984. Algorithm–based fault tolerance for matrix operations. IEEE Trans Comput, C–33(6):518–528. https://doi.org/10.1109/TC.1984.1676475

    Google Scholar 

  • Jeffers J, Reinders J, 2013. Intel Xeon Phi Coprocessor High Performance Programming. Morgan Kaufmann Publishers Inc., San Francisco, USA.

    Google Scholar 

  • Lee S, Vetter JS, 2012. Early evaluation of directive–based GPU programming models for productive exascale computing. Int Conf on High Performance Computing, Networking, Storage, and Analysis, p.1–11. https://doi.org/10.1109/SC.2012.51

    Book  Google Scholar 

  • Lin H, Tang X, Yu B, et al., 2017. Scalable graph traversal on Sunway Taihulight with ten million cores. Int Parallel and Distributed Processing Symp, p.635–645. https://doi.org/10.1109/IPDPS.2017.53

    Book  Google Scholar 

  • Munshi A, 2009. The OpenCL specification. 21st IEEE Hot Chips Symp, p.1–314. https://doi.org/10.1109/HOTCHIPS.2009.7478342

    Book  Google Scholar 

  • Ragan–Kelley J, Adams A, 2012. Halide. http://halide–lang. org [Accessed on Sept. 10, 2018].

    Google Scholar 

  • Ragan–Kelley J, Barnes C, Adams A, et al., 2013. Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. ACM SIGPLAN Not, 48(6):519–530. https://doi.org/10.1145/2499370.2462176

    Article  Google Scholar 

  • Schroeder B, Gibson G, 2010. A large–scale study of failures in high–performance computing systems. IEEE Trans Depend Sec Comput, 7(4):337–350. https://doi.org/10.1109/TDSC.2009.4

    Article  Google Scholar 

  • Stone JE, Gohara D, Shi G, 2010. OpenCL: a parallel programming standard for heterogeneous computing systems. Comput Sci Eng, 12(3):66–73. https://doi.org/10.1109/MCSE.2010.69

    Article  Google Scholar 

  • Tang X, Zhai J, Yu B, et al., 2017. Self–checkpoint: an in–memory checkpoint method using less space and its practice on fault–tolerant HPL. 22nd ACM SIGPLAN Symp on Principles and Practice of Parallel Programming, p.401–413. https://doi.org/10.1145/3155284.3018745

    Google Scholar 

  • Tang X, Zhai J, Qian X, et al., 2018. VSensor: leveraging fixed–workload snippets of programs for performance variance detection. 23rd ACM SIGPLAN Symp on Principles and Practice of Parallel Programming, p.124–136. https://doi.org/10.1145/3200691.3178497

    Book  Google Scholar 

  • Vetter JS, Glassbrook R, Dongarra J, et al., 2011. Keeneland: bringing heterogeneous GPU computing to the computational science community. Comput Sci Eng, 13(5):90–95. https://doi.org/10.1109/MCSE.2011.83

    Article  Google Scholar 

  • Xin RS, Gonzalez JE, Franklin MJ, et al., 2013. GraphX: a resilient distributed graph system on Spark. 1st Int Workshop on Graph Data Management Experiences and Systems, p.1–6. https://doi.org/10.1145/2484425.2484427

    Book  Google Scholar 

  • Yao E, Wang R, Chen M, et al., 2012. A case study of designing efficient algorithm–based fault tolerant application for exascale parallelism. 26th Int Parallel and Distributed Processing Symp, p.438–448. https://doi.org/10.1109/IPDPS.2012.48

    Book  Google Scholar 

  • Zhu X, Chen W, Zheng W, et al., 2016. Gemini: a computation–centric distributed graph processing system. 12th USENIX Symp on Operating Systems Design and Implementation, p.301–316.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, 100084, China

    Ji-Dong Zhai & Wen-Guang Chen

Authors
  1. Ji-Dong Zhai
    View author publications

    Search author on:PubMed Google Scholar

  2. Wen-Guang Chen
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Ji-Dong Zhai.

Additional information

Project supported by the National Key Technology R&D Program of China (No. 2016YFB0200100)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhai, JD., Chen, WG. A vision of post-exascale programming. Frontiers Inf Technol Electronic Eng 19, 1261–1266 (2018). https://doi.org/10.1631/FITEE.1800442

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • Issue date:

  • DOI: https://doi.org/10.1631/FITEE.1800442

Key words

CLC number