close
Skip to main content
Springer Nature Link
Log in

Expansion of the memory pyramid in the era of large models: compute-intensive compute-in-memory and memory-intensive compute-in-memory

  • Review
  • Published:
Save article
View saved research
Science China Information Sciences Aims and scope Submit manuscript

Abstract

The rise of large-scale models has significantly increased the demand for high computational power and throughput in artificial intelligence (AI) chips, presenting challenges for existing architectures. Compute-in-memory (CIM) has emerged as a promising solution to address these bottlenecks. This study redefines the traditional computing architecture pyramid by introducing the CIM pyramid, structured around different storage media. CIM architectures are classified into two categories: compute-intensive CIM, which leverages static random-access memory (SRAM) and embedded dynamic random access memory (eDRAM), and memory-intensive CIM, which utilizes DRAM and non-volatile memory (NVM). The research reviews and analyzes recent advancements in both compute-intensive and memory-intensive CIM, highlighting their development trends and challenges. Furthermore, the study suggests a hybrid, heterogeneous architecture that integrates both CIM types with traditional computing systems, aiming to address the diverse computational needs of large models in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+
from $39.99 /Month
  • Starting from 10 chapters or articles per month
  • Access and download chapters and articles from more than 300k books and 2,500 journals
  • Cancel anytime
View plans

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, books and news in related subjects, suggested using machine learning.

References

  1. Gholami A, Yao Z, Kim S, et al. AI and memory wall. IEEE Micro, 2024, 44: 33–39

    Article  Google Scholar 

  2. Jhang C J, Xue C X, Hung J M, et al. Challenges and trends of SRAM-based computing-in-memory for AI edge devices. IEEE Trans Circuits Syst I, 2021, 68: 1773–1786

    Google Scholar 

  3. Su Y, Kim H, Kim B. CIM-Spin: a scalable CMOS annealing processor with digital in-memory spin operators and register spins for combinatorial optimization problems. IEEE J Solid-State Circuits, 2022, 57: 2263–2273

    Article  Google Scholar 

  4. Yue Z, Wang Y, Wang H, et al. CV-CIM: a hybrid domain xor-derived similarity-aware computation-in-memory supporting cost-volume construction. IEEE J Solid-State Circuits, 2025, 60: 719–733

    Article  Google Scholar 

  5. Bankman D, Yang L, Moons B, et al. An always-on 3.8 µJ/86% CIFAR-10 mixed-signal binary CNN processor with All memory on chip in 28-nm CMOS. IEEE J Solid-State Circuits, 2019, 54: 158–172

    Article  Google Scholar 

  6. Biswas A, Chandrakasan A P. Conv-RAM: an energy-efficient SRAM with embedded convolution computation for low-power CNN-based machine learning applications. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2018. 488–490

    Google Scholar 

  7. Zhang J, Wang Z, Verma N. In-memory computation of a machine-learning classifier in a standard 6T SRAM array. IEEE J Solid-State Circuits, 2017, 52: 915–924

    Article  Google Scholar 

  8. Ling Y T, Wang Z W, Yang Y H, et al. An isolated symmetrical 2T2R cell enabling high precision and high density for RRAM-based in-memory computing. Sci China Inf Sci, 2024, 67: 152402

    Article  Google Scholar 

  9. Zhang Z Y, Chen J W, Chen X, et al. From macro to microarchitecture: reviews and trends of SRAM-based compute-in-memory circuits. Sci China Inf Sci, 2023, 66: 200403

    Article  Google Scholar 

  10. Liang B-S. AI computing in large-scale era: pre-trillion-scale neural network models and exa-scale supercomputing. In: Proceedings of International VLSI Symposium on Technology, Systems and Applications (VLSI-TSA/VLSI-DAT), Taiwan, 2023. 1–3

    Google Scholar 

  11. Wolters C, Yang X, Schlichtmann U, et al. Memory is all you need: an overview of compute-in-memory architectures for accelerating large language model inference. 2024. ArXiv:2406.08413

  12. Williams S, Waterman A, Patterson D. Roofline: an insightful visual performance model for multicore architectures. Commun ACM, 2009, 52: 65–76

    Article  Google Scholar 

  13. Kim S, Hooper C, Wattanawong T, et al. Full stack optimization of transformer inference: a survey. 2023. ArXiv:2302.14017

  14. Tu F, Wu Z, Wang Y, et al. TranCIM: full-digital bitline-transpose CIM-based sparse transformer accelerator with pipeline/parallel reconfigurable modes. IEEE J Solid-State Circuits, 2023, 58: 1798–1809

    Article  Google Scholar 

  15. Guo A, Si X, Chen X, et al. A 28nm 64-kb 31.6-TFLOPS/W digital-domain floating-point-computing-unit and double-bit 6T-SRAM computing-in-memory macro for floating-point CNNs. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2023. 128–130

    Google Scholar 

  16. Yoshioka K. 34.5 A 818-4094TOPS/W capacitor-reconfigured CIM macro for unified acceleration of CNNs and transformers. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2024. 574–576

    Google Scholar 

  17. Xie S, Ni C, Sayal A, et al. eDRAM-CIM: compute-in-memory design with reconfigurable embedded-dynamic-memory array realizing adaptive data converters and charge-domain computing. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2021. 248–250

    Google Scholar 

  18. Song J, Tang X, Luo H, et al. A calibration-free 15-level/cell eDRAM computing-in-memory macro with 3T1C current-programmed dynamic-cascoded MLC achieving 233-to-304-TOPS/W 4b MAC. In: Proceedings of IEEE Custom Integrated Circuits Conference (CICC), San Antonio, 2023. 1–2

    Google Scholar 

  19. He Y, Fan S, Li X, et al. A 28 nm 2.4 Mb/mm2 6.9–16.3 TOPS/mm2 eDRAM-LUT-based digital-computing-in-memory macro with in-memory encoding and refreshing. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2024. 578–580

    Google Scholar 

  20. Park S O, Hong S, Sung S J, et al. Phase-change memory via a phase-changeable self-confined nano-filament. Nature, 2024, 628: 293–298

    Article  Google Scholar 

  21. Ikegawa S, Mancoff F B, Janesky J, et al. Magnetoresistive random access memory: present and future. IEEE Trans Electron Devices, 2020, 67: 1407–1419

    Article  Google Scholar 

  22. Kim J H, Kang S, Lee S, et al. Aquabolt-XL: Samsung HBM2-PIM with in-memory processing for ML accelerators and beyond. In: Proceedings of IEEE Hot Chips 33 Symposium (HCS), 2021. 1–26

    Google Scholar 

  23. Kim J H, Ro Y, So J, et al. Samsung PIM/PNM for transformer based AI: energy efficiency on PIM/PNM cluster. In: Proceedings of IEEE Hot Chips 35 Symposium (HCS), 2023. 1–31

    Google Scholar 

  24. Kwon Y, Vladimir V K, Kim N, et al. System architecture and software stack for GDDR6-AiM. In: Proceedings of IEEE Hot Chips 34 Symposium (HCS), 2022. 1–25

    Google Scholar 

  25. Gómez-Luna J, Hajj I E, Fernandez I, et al. Benchmarking memory-centric computing systems: analysis of real processing-in-memory hardware. In: Proceedings of IEEE 12th International Green and Sustainable Computing Conference (IGSC), 2021. 1–7

    Google Scholar 

  26. Lee J H, Zhang H, Lagrange V, et al. SmartSSD: FPGA accelerated near-storage data analytics on SSD. IEEE Comput Arch Lett, 2020, 19: 110–113

    Article  Google Scholar 

  27. An Y, Tang Y, Yi S, et al. StreamPIM: streaming matrix computation in racetrack memory. In: Proceedings of IEEE International Symposium on High-Performance Computer Architecture (HPCA), 2024. 297–311

    Google Scholar 

  28. Sridharan S, Stevens J R, Roy K, et al. X-Former: in-memory acceleration of transformers. IEEE Trans VLSI Syst, 2023, 31: 1223–1233

    Article  Google Scholar 

  29. Li W J, Lyu D X, Wang G, et al. Hardware-oriented algorithms for softmax and layer normalization of large language models. Sci China Inf Sci, 2024, 67: 200404

    Article  Google Scholar 

  30. Liu Y F, Li X Y, Yin S Y. Review of chiplet-based design: system architecture and interconnection. Sci China Inf Sci, 2024, 67: 200401

    Article  Google Scholar 

  31. Choi J, Park J, Kyung K, et al. Unleashing the potential of PIM: accelerating large batched inference of transformer-based generative models. IEEE Comput Arch Lett, 2023, 22: 113–116

    Article  Google Scholar 

  32. Chen L, Chen Y Q, Chu Z F, et al. Large circuit models: opportunities and challenges. Sci China Inf Sci, 2024, 67: 200402

    Article  Google Scholar 

  33. Han S H, Liu S S, Du S C, et al. CMN: a co-designed neural architecture search for efficient computing-in-memory-based mixture-of-experts. Sci China Inf Sci, 2024, 67: 200405

    Article  Google Scholar 

  34. Shin Y, Park J, Cho S, et al. PIMFlow: compiler and runtime support for CNN models on processing-in-memory DRAM. In: Proceedings of the 21st ACM/IEEE International Symposium on Code Generation and Optimization, 2023. 249–262

    Chapter  Google Scholar 

  35. Wu P-C, Su J-W, Chung Y-L, et al. A 28 nm 1 Mb time-domain computing-in-memory 6T-SRAM macro with a 6.6 ns latency, 1241 GOPS and 37.01 TOPS/W for 8b-MAC operations for edge-AI devices. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2022. 1–3

    Google Scholar 

  36. Yang J, Kong Y, Zhang Z, et al. TIMAQ: a time-domain computing-in-memory-based processor using predictable decomposed convolution for arbitrary quantized DNNs. IEEE J Solid-State Circuits, 2021, 56: 3021–3038

    Article  Google Scholar 

  37. Dorrance R, Dasalukunte D, Wang H, et al. An energy-efficient Bayesian neural network accelerator with CiM and a time-interleaved Hadamard digital GRNG using 22-nm FinFET. IEEE J Solid-State Circuits, 2023, 58: 2826–2838

    Article  Google Scholar 

  38. Yang J, Kong Y, Wang Z, et al. Sandwich-RAM: an energy-efficient in-memory BWN architecture with pulse-width modulation. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2019. 394–396

    Google Scholar 

  39. Miyashita D, Kousai S, Suzuki T, et al. A neuromorphic chip optimized for deep learning and CMOS technology with time-domain analog and digital mixed-signal processing. IEEE J Solid-State Circuits, 2017, 52: 2679–2689

    Article  Google Scholar 

  40. Wu P-C, Su J-W, Hong L-Y, et al. A 22 nm 832 Kb hybrid-domain floating-point SRAM in-memory-compute macro with 16.2–70.2 TFLOPS/W for high-accuracy AI-edge devices. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2023. 126–128

    Google Scholar 

  41. Guo A, Xi C, Dong F, et al. A 28-nm 64-kb 31.6-TFLOPS/W digital-domain floating-point-computing-unit and double-bit 6T-SRAM computing-in-memory macro for floating-point CNNs. IEEE J Solid-State Circuits, 2024, 59: 3032–3044

    Article  Google Scholar 

  42. Guo A, Chen X, Dong F, et al. A 22 nm 64 kb lightning-like hybrid computing-in-memory macro with a compressed adder tree and analog-storage quantizers for transformer and CNNs. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2024. 570–572

    Google Scholar 

  43. Jeong S, Oh J, Jeon D. A 28 nm 157TOPS/W 446.9 Kb/mm2 compute-in-memory SRAM macro with analog-digital hybrid computing for deep neural network inference. In: Proceedings of IEEE Custom Integrated Circuits Conference (CICC), Denver, 2024. 1–2

    Google Scholar 

  44. Chiu Y C, Zhang Z, Chen J J, et al. A 4-Kb 1-to-8-bit configurable 6T SRAM-based computation-in-memory unit-macro for CNN-based AI edge processors. IEEE J Solid-State Circuits, 2020, 55: 2790–2801

    Article  Google Scholar 

  45. Ali M, Chakraborty I, Saxena U, et al. A 35.5–127.2 TOPS/W dynamic sparsity-aware reconfigurable-precision compute-in-memory SRAM macro for machine learning. IEEE Solid-State Circuits Lett, 2021, 4: 129–132

    Article  Google Scholar 

  46. Guo R, Yue Z, Si X, et al. A 5.99-to-691.1 TOPS/W tensor-train in-memory-computing processor using bit-level-sparsity-based optimization and variable-precision quantization. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), 2021. 242–244

    Google Scholar 

  47. Kneip A, Lefebvre M, Verecken J, et al. IMPACT: a 1-to-4b 813-TOPS/W 22-nm FD-SOI compute-in-memory CNN accelerator featuring a 4.2-POPS/W 146-TOPS/mm2 CIM-SRAM with multi-bit analog batch-normalization. IEEE J Solid-State Circuits, 2023, 58: 1871–1884

    Article  Google Scholar 

  48. Su J W, Si X, Chou Y C, et al. Two-way transpose multibit 6T SRAM computing-in-memory macro for inference-training AI edge chips. IEEE J Solid-State Circuits, 2022, 57: 609–624

    Article  Google Scholar 

  49. Si X, Tu Y N, Huang W H, et al. A local computing cell and 6T SRAM-based computing-in-memory macro with 8-b MAC operation for edge AI chips. IEEE J Solid-State Circuits, 2021, 56: 2817–2831

    Article  Google Scholar 

  50. Wang H, Liu R, Dorrance R, et al. A charge domain SRAM compute-in-memory macro with C-2C ladder-based 8-bit MAC unit in 22-nm FinFET process for edge inference. IEEE J Solid-State Circuits, 2023, 58: 1037–1050

    Article  Google Scholar 

  51. Chen Z, Yu Z, Jin Q, et al. CAP-RAM: a charge-domain in-memory computing 6T-SRAM for accurate and precision-programmable CNN inference. IEEE J Solid-State Circuits, 2021, 56: 1924–1935

    Article  Google Scholar 

  52. Biswas A, Chandrakasan A P. CONV-SRAM: an energy-efficient SRAM with in-memory dot-product computation for low-power convolutional neural networks. IEEE J Solid-State Circuits, 2019, 54: 217–230

    Article  Google Scholar 

  53. Yang X, Zhu K, Tang X, et al. An in-memory-computing charge-domain ternary CNN classifier. In: Proceedings of IEEE Custom Integrated Circuits Conference (CICC), Austin, 2021. 1–2

    Google Scholar 

  54. Jiang Z, Yin S, Seo J S, et al. C3SRAM: an in-memory-computing SRAM macro based on robust capacitive coupling computing mechanism. IEEE J Solid-State Circuits, 2020, 55: 1888–1897

    Article  Google Scholar 

  55. Zhang Z, Liu Z, Liu F, et al. A 28 nm 16 kb aggregation and combination computing-in-memory macro with dual-level sparsity modulation and sparse-tracking ADCs for GCNs. In: Proceedings of IEEE Custom Integrated Circuits Conference (CICC), Denver, 2024. 1–2

    Google Scholar 

  56. Tu F, Wang Y, Wu Z, et al. ReDCIM: reconfigurable digital computing- in-memory processor with unified FP/INT pipeline for cloud AI acceleration. IEEE J Solid-State Circuits, 2023, 58: 243–255

    Article  Google Scholar 

  57. Chih Y D, Lee P H, Fujiwara H, et al. An 89 TOPS/W and 16.3 TOPS/mm2 all-digital SRAM-based full-precision compute-in-memory macro in 22 nm for machine-learning edge applications. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2021. 252–254

    Google Scholar 

  58. Tu F, Wang Y, Wu Z, et al. TensorCIM: a 28 nm 3.7 nJ/Gather and 8.3 TFLOPS/W FP32 digital-CIM tensor processor for MCM-CIM-based beyond-NN acceleration. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2023. 254–256

    Google Scholar 

  59. Wang J, Wang X, Eckert C, et al. A 28-nm compute SRAM with bit-serial logic/arithmetic operations for programmable in-memory vector computing. IEEE J Solid-State Circuits, 2020, 55: 76–86

    Article  Google Scholar 

  60. Fujiwara H, Mori H, Zhao W-C, et al. A 5-nm 254-TOPS/W 221-TOPS/mm2 fully-digital computing-in-memory macro supporting wide-range dynamic-voltage-frequency scaling and simultaneous MAC and write operations. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2022. 1–3

    Google Scholar 

  61. Xie S, Ni C, Jain P, et al. Gain-cell CIM: leakage and bitline swing aware 2T1C gain-cell eDRAM compute in memory design with bitline precharge DACs and compact Schmitt trigger ADCs. In: Proceedings of IEEE Symposium on VLSI Technology and Circuits, Honolulu, 2022. 112–113

    Google Scholar 

  62. Kim S, Li Z, Um S, et al. DynaPlasia: an eDRAM in-memory-computing-based reconfigurable spatial accelerator with triple-mode cell for dynamic resource switching. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2023. 256–258

    Google Scholar 

  63. Khwa W S, Wu P C, Wu J J, et al. A 16 nm 96 Kb integer/floating-point dual-mode-gain-cell-computing-in-memory macro achieving 73.3–163.3 TOPS/W and 33.2–91.2 TFLOPS/W for AI-edge devices. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2024. 568–570

    Google Scholar 

  64. Chen Z, Chen X, Gu J. A 65 nm 3T dynamic analog RAM-based computing-in-memory macro and CNN accelerator with retention enhancement, adaptive analog sparsity and 44 TOPS/W system energy efficiency. In: Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, 2021. 240–242

    Google Scholar 

  65. Zhou R, Tabrizchi S, Morsali M, et al. P-PIM: a parallel processing-in-DRAM framework enabling row hammer protection. In: Proceedings of Design, Automation & Test in Europe Conference & Exhibition (DATE), 2023. 1–6

    Google Scholar 

  66. Heo J, Shin Y, Choi S, et al. PRIMO: a full-stack processing-in-DRAM emulation framework for machine learning workloads. In: Proceedings of IEEE/ACM International Conference on Computer Aided Design (ICCAD), 2023. 1–9

    Google Scholar 

  67. Wang J, Ge M, Ding B, et al. NicePIM: design space exploration for processing-in-memory DNN accelerators with 3-D stacked-DRAM. IEEE Trans Comput-Aided Des Integr Circuits Syst, 2024, 43: 1456–1469

    Article  Google Scholar 

  68. Li C, Zhou Z, Wang Y, et al. PIM-DL: expanding the applicability of commodity DRAM-PIMs for deep learning via algorithm-system co-optimization. In: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, 2024. 879–896

    Google Scholar 

  69. Kim H, Lee H, Kim J, et al. Cache register sharing structure for channel-level near-memory processing in NAND flash memory. In: Proceedings of the 24th International Symposium on Quality Electronic Design (ISQED), 2023. 1–6

    Google Scholar 

  70. Lee H, Kim M, Min D, et al. 3D-FPIM: an extreme energy-efficient DNN acceleration system using 3D NAND flash-based in-situ PIM unit. In: Proceedings of the 55th IEEE/ACM International Symposium on Microarchitecture (MICRO), 2022. 1359–1376

    Chapter  Google Scholar 

  71. Kang M, Kim H, Shin H, et al. S-FLASH: a NAND flash-based deep neural network accelerator exploiting bit-level sparsity. IEEE Trans Comput, 2021, 71: 1291–1304

    Google Scholar 

  72. Yang T, Li D, Ma F, et al. PASGCN: an ReRAM-based PIM design for GCN with adaptively sparsified graphs. IEEE Trans Comput-Aided Des Integr Circuits Syst, 2023, 42: 150–163

    Article  Google Scholar 

  73. Li B, Wang Y, Chen Y. HitM: high-throughput ReRAM-based PIM for multi-modal neural networks. In: Proceedings of the 39th International Conference on Computer-Aided Design, 2020. 1–7

    Google Scholar 

  74. Jin H, Liu C, Liu H, et al. ReHy: a ReRAM-based digital/analog hybrid PIM architecture for accelerating CNN training. IEEE Trans Parallel Distrib Syst, 2021, 33: 2872–2884

    Google Scholar 

  75. Yang T, Li D, Han Y, et al. PIMGCN: a ReRAM-based PIM design for graph convolutional network acceleration. In: Proceedings of the 58th ACM/IEEE Design Automation Conference (DAC), 2021. 583–588

    Google Scholar 

  76. Liu F, Zhao W, Chen Y, et al. PIM-DH: ReRAM-based processing-in-memory architecture for deep hashing acceleration. In: Proceedings of the 59th ACM/IEEE Design Automation Conference (DAC), 2022. 1087–1092

    Chapter  Google Scholar 

  77. Mamdouh A, Geng H, Niemier M, et al. Shared-PIM: enabling concurrent computation and data flow for faster processing-in-DRAM. 2024. ArXiv:2408.15489

  78. Kim J H, Kang S H, Lee S, et al. Aquabolt-XL HBM2-PIM, LPDDR5-PIM with in-memory processing, and AXDIMM with acceleration buffer. IEEE Micro, 2022, 42: 20–30

    Article  Google Scholar 

  79. Chi P, Li S, Xu C, et al. Prime: a novel processing-in-memory architecture for neural network computation in ReRAM-based main memory. ACM SIGARCH Comput Archit News, 2016, 44: 27–39

    Article  Google Scholar 

  80. Wang Y, Han Y, Zhang L, et al. ProPRAM: exploiting the transparent logic resources in non-volatile memory for near data computing. In: Proceedings of the 52nd IEEE Design Automation Conference (DAC), 2015. 1–6

    Google Scholar 

  81. Chiang H W, Nien C F, Cheng H Y, et al. ReAIM: a ReRAM-based adaptive ising machine for solving combinatorial optimization problems. In: Proceedings of the 51st ACM/IEEE Annual International Symposium on Computer Architecture (ISCA), 2024. 58–72

    Google Scholar 

  82. Li S, Xu C, Zou Q, et al. Pinatubo: a processing-in-memory architecture for bulk bitwise operations in emerging non-volatile memories. In: Proceedings of the 53rd IEEE Design Automation Conference (DAC), 2016. 1–6

    Google Scholar 

  83. Bavikadi S, Sutradhar P R, Ganguly A, et al. UPIM: performance-aware online learning capable processing-in-memory. In: Proceedings of IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021. 1–4

    Google Scholar 

  84. Sharma H, Mandal S K, Doppa J R, et al. SWAP: a server-scale communication-aware chiplet-based manycore PIM accelerator. IEEE Trans Comput-Aided Des Integr Circuits Syst, 2022, 41: 4145–4156

    Article  Google Scholar 

  85. Angizi S, Sun J, Zhang W, et al. PIM-Aligner: a processing-in-MRAM platform for biological sequence alignment. In: Proceedings of Design, Automation & Test in Europe Conference & Exhibition (DATE), 2020. 1265–1270

    Google Scholar 

  86. Shan W W, Cui Y Q, Dai W T, et al. An efficient path delay variability model for wide-voltage-range digital circuits. Sci China Inf Sci, 2023, 66: 129401

    Article  Google Scholar 

  87. Luo X, Zhang C, Geng C B, et al. TSCompiler: efficient compilation framework for dynamic-shape models. Sci China Inf Sci, 2024, 67: 200403

    Article  Google Scholar 

  88. Zhang J Y, Shen J R, Wang Z K, et al. SpikingMiniLM: energy-efficient spiking transformer for natural language understanding. Sci China Inf Sci, 2024, 67: 200406

    Article  Google Scholar 

  89. Xia Z H, Wan R, Chen J N, et al. Reconfigurable spatial-parallel stochastic computing for accelerating sparse convolutional neural networks. Sci China Inf Sci, 2023, 66: 162404

    Article  MathSciNet  Google Scholar 

  90. Sandler M, Howard A, Zhu M, et al. MobileNetV2: inverted residuals and linear bottlenecks. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 4510–4520

    Google Scholar 

  91. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 770–778

    Google Scholar 

  92. Tong W, Liu Y. Recent progress of layered memristors based on two-dimensional MoS2. Sci China Inf Sci, 2023, 66: 160402

    Article  Google Scholar 

  93. Zhao Y, Gao M, Liu F, et al. UM-PIM: DRAM-based PIM with uniform & shared memory space. In: Proceedings of ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA), 2024

    Google Scholar 

  94. Tian B, Li Y, Jiang L, et al. NDPBridge: enabling cross-bank coordination in near-DRAM-bank processing architectures. In: Proceedings of ACM/IEEE 51st International Symposium on Computer Architecture (ISCA), Buenos Aires, 2024. 628–643

    Google Scholar 

  95. Yu Z, Liang S, Ma T, et al. Cambricon-LLM: a chiplet-based hybrid architecture for on-device inference of 70B LLM. In: Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture (MICRO), 2024, 1474–1488

    Chapter  Google Scholar 

  96. Heo G, Lee S, Cho J, et al. NeuPIMs: NPU-PIM heterogeneous acceleration for batched LLM inferencing. In: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating, 2024. 722–737

    Google Scholar 

  97. Li C, Zhou Z, Zheng S, et al. SpecPIM: accelerating speculative inference on PIM-enabled system via architecture-dataflow co-exploration. In: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating, 2024. 950–965

    Google Scholar 

Download references

Acknowledgements

This work was supported by National Key R&D Program of China (Grant Nos. 2020YFA0711900, 2020YF-A0711902, 2022ZD0118901), National Natural Science Foundation of China (Grant Nos. 92264203, 62204036), Key R&D Program of Jiangsu Province (Grant No. BE2023020-1), and Fundamental Research Funds for the Central Universities (Grant No. 2242022k60009).

Author information

Authors and Affiliations

  1. National Application Specific Integrated Circuit Center, Southeast University, Nanjing, 210096, China

    Zhichao Liu, Yanqi Zhang, Zhican Zhang, Yi Yang, Zhaoyang Zhang, Xing Wang, Jinwu Chen, Xin Si & Jun Yang

  2. National Nanjing Starting Line Wearable Microelectronics Technology Co., Ltd., Nanjing, 210032, China

    Xiaomin Li

Authors
  1. Zhichao Liu
    View author publications

    Search author on:PubMed Google Scholar

  2. Yanqi Zhang
    View author publications

    Search author on:PubMed Google Scholar

  3. Zhican Zhang
    View author publications

    Search author on:PubMed Google Scholar

  4. Yi Yang
    View author publications

    Search author on:PubMed Google Scholar

  5. Zhaoyang Zhang
    View author publications

    Search author on:PubMed Google Scholar

  6. Xing Wang
    View author publications

    Search author on:PubMed Google Scholar

  7. Jinwu Chen
    View author publications

    Search author on:PubMed Google Scholar

  8. Xiaomin Li
    View author publications

    Search author on:PubMed Google Scholar

  9. Xin Si
    View author publications

    Search author on:PubMed Google Scholar

  10. Jun Yang
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Xin Si.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Zhang, Y., Zhang, Z. et al. Expansion of the memory pyramid in the era of large models: compute-intensive compute-in-memory and memory-intensive compute-in-memory. Sci. China Inf. Sci. 68, 201401 (2025). https://doi.org/10.1007/s11432-024-4354-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Version of record:

  • DOI: https://doi.org/10.1007/s11432-024-4354-y

Keywords