llms.blog
Storage3 articles

Storage

Articles

  • GPU Cluster Storage in Production: GPUDirect Storage, NVMe-oF, Parallel File Systems, and Checkpointing Throughput

    Training frontier large language models and serving hundred-billion parameter checkpoints places extreme demands on storage subsystems. While compute clusters frequently deploy thousands of GPUs connected via high-bandwidth interconnects like NVLink and InfiniBand, storage architectures often become severe bottlenecks during two critical operational phases: distributed checkpointing and cold-start model weight loading. A standard 70-billion parameter model in BF16 precision generates approximat

    1 min

  • Chinese NAND Flash Maker YMTC Files for .9B Shanghai STAR Market IPO

    The Shanghai Stock Exchange has accepted the listing application for Yangtze Memory Technologies Corp (YMTC) and parent entity CCSH Corporation, setting up a 33 billion yuan ($4.9 billion) initial public offering on China's science and technology-focused STAR Market. The offering marks the third-largest float in the history of the STAR Market, trailing only foundry Semiconductor Manufacturing International Corporation (SMIC) and DRAM manufacturer ChangXin Memory Technologies (CXMT). Offering

    1 min

  • Hierarchical KV Cache Offloading in Production: Multi-Tier Storage Across HBM, DRAM, NVMe, and Remote Pools

    Serving large language models at context lengths of 32,000 to 1 million tokens exposes a hard physical constraint: GPU High-Bandwidth Memory (HBM) capacity. While modern accelerator compute scales efficiently across tensor-parallel and pipeline-parallel topologies, the memory footprint of Key-Value (KV) attention states scales linearly with context length, batch size, number of layers, and hidden dimensions. On an 80GB NVIDIA H100 GPU running Llama 3 70B in FP16 precision, the model weights alo

    1 min