<\/span><\/h2>\nNVMe SSDs offers great benefits for specific AI use cases, like training a machine learning model. Machine learning involves two phases \u2013 training a model based on what is learned from the dataset, and running the model. The most resource-hungry stage of machine learning is the training of a model.<\/p>\n
The modern datasets used for model training can be huge \u2013 for example, MRI scans can reach terabytes each; a learning system may use tens or hundreds of thousands of images. Even if the training itself runs from RAM, the memory should be fed from non-volatile storage, which has to support very high bandwidth. In addition, paging out the old training data and bringing in new data should be done as fast as possible to keep the GPUs from being idle. This necessitates low latency, and the only protocol allowing for both high bandwidth and low latency like this is NVMe.<\/p>\n
However, there are limitations as to how many local NVMe drives can be used. This is often the result of how many PCIe lanes are actually allocated to NVMe drives since the GPUs\/SoCs also require PCIe lanes. Additionally, for checkpoint usage many local NVMe drives need to be synchronized to allow for a usable snapshot.<\/p>\n
<\/span>NVMe Benefits for Artificial Intelligence & Machine Learning<\/strong><\/span><\/h2>\nServer solutions that leverage NVMe SSDs create an ideal storage platform for AI\/ML workloads, especially machine learning for multiple applications. By applying NVMe-based data storage on solid-stage drives, you can:<\/p>\n
\n- Create and Manage Larger Datasets<\/em><\/strong> \u2013 By separating out storage capacity from the compute nodes, datasets for machine learning training can scale up to 1PB. As a result, the dataset grows, bringing additional NVMe storage online, increasing performance and avoiding limitations set by legacy storage controller bottlenecks.<\/li>\n
- Overcome Capacity Limitations of Local SSDs<\/em><\/strong> \u2013 The limited space for SSD media in GPU nodes restricts the ability to manage larger datasets. Conversely, NVMe storage enables NVMe volumes to be dynamically provisioned over high performance Ethernet or InfiniBand networks.<\/li>\n
- Accelerate epoch time of Machine Learning<\/em><\/strong> \u2013 NVMe SSDs eliminates the latency bottlenecks of older storage protocols and unleashes the parallelism inherent to the NVMe protocol by leveraging high performance NVMe-oF by as much as 10x. Every GPU node has direct, parallel access to the media at the lowest possible latency.<\/li>\n
- Improve Utilization of GPUs<\/em><\/strong> \u2013 Having GPUs rest idle due to slow access to data for processing is costly. By offloading storage access to the idle CPUs and delivering storage performance at the speed of local SSDs, NVMe storage ensures that the GPU-nodes are kept busy with fast access to data.<\/li>\n<\/ul>\n