{"id":3781,"date":"2021-06-20T08:00:02","date_gmt":"2021-06-20T15:00:02","guid":{"rendered":"https:\/\/www.www.pogolinux.com\/blog\/?p=3781"},"modified":"2021-09-02T08:06:47","modified_gmt":"2021-09-02T15:06:47","slug":"nvme-over-fabrics-explained","status":"publish","type":"post","link":"https:\/\/www.pogolinux.com\/blog\/nvme-over-fabrics-explained\/","title":{"rendered":"Data Center Strategy: Explanation of NVMe over Fabrics (NVMe-oF)"},"content":{"rendered":"<p>To meet the growing number of data center services and private-cloud applications, IT departments are replacing traditional direct attached storage (DAS) with NVMe flash-based arrays and storage targets that utilize NVMe over Fabrics (NVMe-oF) technologies.<\/p>\n<p><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">Y<\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">ou may be surprised to learn that the <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">NVMe protocol is not limited to connecting flash SSDs <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">by PCIe within a rackmount server or DAS enclosure<\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">: it may also be used as a networking protocol<\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">. (This is to distinguish it from a network, which may restrict the possible connections possible). <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">When used in this context, NVMe-oF enables <\/span><\/span><\/strong><span style=\"font-weight: bold;\">any-to-any connections <\/span><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">of <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">high-performance, low-latency <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">flash arrays, which <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">obliterate DAS storage <\/span><\/span><\/strong><strong><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">with legacy SAS or SATA-based interfaces.<\/span><\/span><\/strong><\/p>\n<p>Should your organization unleash the full performance potential of a flash storage investment with the ability to share flash storage across servers? First, let&#8217;s examine how utilizing NVMe over Fabrics (NVMe-oF) technologies can help flash arrays or storage targets achieve high-performance and low-latency more efficiently.<\/p>\n<p><em>Researching flash-based solutions to <\/em>achieve new levels of performance and efficiency<em>? Here&#8217;s what data center administrators need to know to evaluate <a href=\"https:\/\/www.pogolinux.com\/products\/nvme-storage-servers\" target=\"_blank\" rel=\"noopener noreferrer\">NVMe flash storage solutions<\/a>. Questions? Drop us an email at <a href=\"mailto:sales@www.pogolinux.com\">sales@www.pogolinux.com<\/a><\/em><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_89 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.pogolinux.com\/blog\/nvme-over-fabrics-explained\/#Introducing_NVMe_over_Fabrics_NVMe-oF\" >Introducing NVMe over Fabrics (NVMe-oF)<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.pogolinux.com\/blog\/nvme-over-fabrics-explained\/#When_to_use_NVMe-oF\" >When to use NVMe-oF?<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.pogolinux.com\/blog\/nvme-over-fabrics-explained\/#Choosing_an_all-flash_NVMe_Architecture_Solution\" >Choosing an all-flash NVMe Architecture Solution<\/a><\/li><\/ul><\/nav><\/div>\n<h3 style=\"font-size: 22px; line-height: 175%;\"><span class=\"ez-toc-section\" id=\"Introducing_NVMe_over_Fabrics_NVMe-oF\"><\/span><span style=\"color: #000000;\">Introducing NVMe over Fabrics (NVMe-oF)<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"margin-bottom: 10px;\">Previously, the challenge with using NVMe is that other servers cannot benefit from flash storage. Traditionally, the adoption of NVMe by data center users is often to directly connect an NVMe SSD by PCIe within a rackmount server or DAS enclosure. While this improves the performance of a single enclosed system, this greatly underutilizes NVMe and restricts the possible connections, as the flash device is not accessible by other systems beyond the server to which it is attached.<\/p>\n<p style=\"margin-bottom: 10px;\">With an NVMe \u201cfabric\u201d, any server can access any NVMe SSD without a dedicated PCIe connection. Simply connect the fabric-based JBOF (Just a Bunch of Flash) expansion enclosure to the network switch, to make it accessible among servers on the network. This gives data center users a more efficient way to use flash storage for cloud and mission-critical applications, with network connectivity options that include:<\/p>\n<p style=\"margin-bottom: 10px;\"><strong>Ethernet<\/strong> \u2013 Instead of landlocking NVMe SSDs into a single-enclosed system, NVMe-oF takes the capabilities of NVMe and shares a flash array over an Ethernet fabric to a collection of servers on a network.<\/p>\n<p style=\"margin-bottom: 10px;\"><strong>Fibre Channel<\/strong> \u2013 For environments already invested in fibre channel (FC) infrastructure, most modern switches and IP switches already have updated firmware to support NVMe-oF network protocol. FC is designed for storage and can support both legacy SCSI traffic and NVMe traffic simultaneously. Since most organizations will integrate NVMe-oF into an existing data center, current support of both protocols enables them to make the conversion at a pace comfortable for them.<\/p>\n<p style=\"margin-bottom: 10px;\">Enabled by NVMe over Fabrics (NVMe-oF), IT organizations are able to create a very high performance storage network with latencies that rival direct attached storage (DAS). Now, JBOF DAS can be shared, when needed, among servers.<\/p>\n<h3 style=\"font-size: 22px; line-height: 175%;\"><span class=\"ez-toc-section\" id=\"When_to_use_NVMe-oF\"><\/span><span style=\"color: #000000;\">When to use NVMe-oF?<\/span><span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p style=\"margin-bottom: 10px;\">So, when should you migrate off traditional storage? When IT planners aggressively switch to NVMe-oF is largely a function of need and timing. Data center operators concerned about system performance know that investing in high-performance flash array makes the most sense where the storage target is the current bottleneck.<\/p>\n<p style=\"margin-bottom: 10px;\"><strong> Flash Array<\/strong> \u2013 The first step to NVMe for most organizations is an NVMe Flash Array with traditional networking connections to the storage network. While these systems generate IOPS approaching the millions, the reality is that there are very few workloads that require more than the performance of these systems.<\/p>\n<p style=\"margin-bottom: 10px;\"><strong> Storage Target<\/strong> \u2013 An NVMe-oF storage target can be dynamically shared among workloads \u2013 providing an \u201con-demand\u201d or composable storage resource that provides additional benefits, including flexibility, agility, and greater resource efficiency. By leveraging both NVMe SSDs with NVMe over Fabrics (NVMe-oF), data center operators to create a very high performance storage network with latencies that rival SAS or SATA-based direct attached storage (DAS). Typical workload that benefit from the adoption of an end-to-end architecture are high-read workloads or large block applications.<\/p>\n<p><em>Next, how do you get all the performance potential out of an <a href=\"https:\/\/www.pogolinux.com\/blog\/all-flash-nvme-jbof-direct-attached-storage\/\">all-flash NVMe architecture solution?<\/a><\/em><\/p>\n<h2><span class=\"ez-toc-section\" id=\"Choosing_an_all-flash_NVMe_Architecture_Solution\"><\/span>Choosing an all-flash NVMe Architecture Solution<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">Low latency or high transfer rates are of little benefit if they swamp the target application. <\/span><\/span><span style=\"font-weight: 400;\"><span style=\"font-weight: normal;\">While these systems generate IOPS approaching the millions, the reality is that there are very few workloads that require more than the performance of these systems. <\/span><\/span>However, there is an emerging class of workloads that can take advantage of all the performance and low latency of an end-to-end NVMe system.<\/p>\n<p>If you&#8217;d like to learn more about how to maximize your data center infrastructure by up to 90% while minimizing its footprint, give us a call at <strong>(888) 828-7646<\/strong>, email us at <a href=\"mailto:sales@www.pogolinux.com\">sales@www.pogolinux.com<\/a> or <a href=\"https:\/\/www.pogolinux.com\/blog\/book-meeting\/\">book a time calendar<\/a> to speak. We&#8217;ve helped organizations of all sizes deploy composable solutions for just about every IT budget.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-5603 size-full alignleft\" src=\"https:\/\/www.pogolinux.com\/blog\/wp-content\/uploads\/2020\/12\/blog-article-footer_osnexus-quantastor-object-storage.jpg\" alt=\"\" width=\"725\" height=\"200\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>To meet the growing number of data center services and private-cloud applications, IT departments are replacing traditional direct attached storage (DAS) with NVMe flash-based arrays and storage targets that utilize NVMe over Fabrics (NVMe-oF) technologies. You may be surprised to learn that the NVMe protocol is not limited to connecting flash SSDs by PCIe within&#8230;<\/p>\n","protected":false},"author":6,"featured_media":5536,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[241,1,106],"tags":[],"class_list":["post-3781","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-nvme","category-rackmount-servers","category-western-digital"],"_links":{"self":[{"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/posts\/3781","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/users\/6"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/comments?post=3781"}],"version-history":[{"count":18,"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/posts\/3781\/revisions"}],"predecessor-version":[{"id":5662,"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/posts\/3781\/revisions\/5662"}],"wp:attachment":[{"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/media?parent=3781"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/categories?post=3781"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pogolinux.com\/blog\/wp-json\/wp\/v2\/tags?post=3781"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}