ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet AdapterProduct Guide

Author
Updated
28 Feb 2021
Form Number
LP1351
PDF size
12 pages, 122 KB

Abstract

The ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet Adapter is a high performance 25Gb Ethernet network adapter that offers multiple network offloads including RoCE v2, NVMe over Ethernet and Open vSwitch.

This product guide provides essential presales information to understand the adapter and its key features, specifications, and compatibility. This guide is intended for technical specialists, sales specialists, sales engineers, IT architects, and other IT professionals who want to learn more about the ConnectX-5 EN adapter and consider its use in IT solutions.

Change History

Changes in the February 28, 2021 update:

  • The server support tables are now automatically updated - Server support section

Introduction

The ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet Adapter is a high performance 25Gb Ethernet network adapter that offers multiple network offloads including RoCE v2, NVMe over Ethernet and Open vSwitch.

The following figure shows the ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet Adapter (the standard heat sink has been removed in this photo).

ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet Adapter
Figure 1. ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet Adapter

Did you know?

NVMe storage devices are gaining popularity by offering very fast storage access. The evolving NVMe over Fabric (NVMe-oF) protocol leverages the RDMA connectivity for remote access. ConnectX-5 offers further enhancements by providing NVMe-oF target offloads, enabling very efficient NVMe storage access with no CPU intervention, and thus improving performance and reducing latency.

Part number information

The following table shows the part number for the adapter.

Table 1. Ordering information
Part number Feature code Mellanox
equivalent
Description
4XC7A62574 BEAP MCX512A-ACAT ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 2-port PCIe Ethernet Adapter

The part numbers include the following:

  • One Mellanox adapter
  • Low-profile (2U) and full-height (3U) adapter brackets
  • Documentation

Supported cables and transceivers

This section lists the supported transceivers and cables.

Table 2. Supported transceivers
Part number Feature code Description
10Gb transceivers
46C3447 5053 SFP+ SR Transceiver
49Y4218 0064 QLogic 10Gb SFP+ SR Optical Transceiver
25Gb transceivers
7G17A03537 AV1B Lenovo 25GBase-SR SFP28 Transceiver

The following table lists the supported fiber optic cables and Active Optical Cables.

Table 3. Optical cables
Part number Feature code Description
LC-LC OM3 Fiber Optic Cables (these cables require a 10 GbE SFP+ SR or 25 GbE SFP28 SR transceiver)
00MN499 ASR5 Lenovo 0.5m LC-LC OM3 MMF Cable
00MN502 ASR6 Lenovo 1m LC-LC OM3 MMF Cable
00MN505 ASR7 Lenovo 3m LC-LC OM3 MMF Cable
00MN508 ASR8 Lenovo 5m LC-LC OM3 MMF Cable
00MN511 ASR9 Lenovo 10m LC-LC OM3 MMF Cable
00MN514 ASRA Lenovo 15m LC-LC OM3 MMF Cable
00MN517 ASRB Lenovo 25m LC-LC OM3 MMF Cable
00MN520 ASRC Lenovo 30m LC-LC OM3 MMF Cable
MTP-4xLC OM3 MMF Breakout Cables (these cables require a transceiver)
00FM412 A5UA Lenovo 1m MPO-4xLC OM3 MMF Breakout Cable
00FM413 A5UB Lenovo 3m MPO-4xLC OM3 MMF Breakout Cable
00FM414 A5UC Lenovo 5m MPO-4xLC OM3 MMF Breakout Cable
SFP+ 10Gb Active Optical Cables
00YL634 ATYX Lenovo 1M SFP+ to SFP+ Active Optical Cable
00YL637 ATYY Lenovo 3M SFP+ to SFP+ Active Optical Cable
00YL640 ATYZ Lenovo 5M SFP+ to SFP+ Active Optical Cable
00YL643 ATZ0 Lenovo 7M SFP+ to SFP+ Active Optical Cable
00YL646 ATZ1 Lenovo 15M SFP+ to SFP+ Active Optical Cable
00YL649 ATZ2 Lenovo 20M SFP+ to SFP+ Active Optical Cable
SFP28 25Gb Active Optical Cables
7Z57A03541 AV1F Lenovo 3m 25G SFP28 Active Optical Cable
7Z57A03542 AV1G Lenovo 5m 25G SFP28 Active Optical Cable
7Z57A03543 AV1H Lenovo 10m 25G SFP28 Active Optical Cable
7Z57A03544 AV1J Lenovo 15m 25G SFP28 Active Optical Cable
7Z57A03545 AV1K Lenovo 20m 25G SFP28 Active Optical Cable
QSFP28 100Gb Breakout Active Optical Cables
7Z57A03551 AV1R Lenovo 3m 100G to 4x25G Breakout Active Optical Cable
7Z57A03552 AV1S Lenovo 5m 100G to 4x25G Breakout Active Optical Cable
7Z57A03553 AV1T Lenovo 10m 100G to 4x25G Breakout Active Optical Cable
7Z57A03554 AV1U Lenovo 15m 100G to 4x25G Breakout Active Optical Cable
7Z57A03555 AV1V Lenovo 20m 100G to 4x25G Breakout Active Optical Cable
OM4 LC to LC Cables (these cables require a transceiver)
4Z57A10845 B2P9 Lenovo 0.5m LC-LC OM4 MMF Cable
4Z57A10846 B2PA Lenovo 1m LC-LC OM4 MMF Cable
4Z57A10847 B2PB Lenovo 3m LC-LC OM4 MMF Cable
4Z57A10848 B2PC Lenovo 5m LC-LC OM4 MMF Cable
4Z57A10849 B2PD Lenovo 10m LC-LC OM4 MMF Cable
4Z57A10850 B2PE Lenovo 15m LC-LC OM4 MMF Cable
4Z57A10851 B2PF Lenovo 25m LC-LC OM4 MMF Cable
4Z57A10852 B2PG Lenovo 30m LC-LC OM4 MMF Cable

The following table lists the supported direct-attach copper (DAC) cables.

Table 4. Copper cables
Part number Feature code Description
SFP+ 10Gb Passive DAC Cables
00D6288 A3RG 0.5m Passive DAC SFP+ Cable
90Y9427 A1PH 1m Passive DAC SFP+ Cable
00AY764 A51N 1.5m Passive DAC SFP+ Cable
00AY765 A51P 2m Passive DAC SFP+ Cable
90Y9430 A1PJ 3m Passive DAC SFP+ Cable
90Y9433 A1PK 5m Passive DAC SFP+ Cable
00D6151 A3RH 7m Passive DAC SFP+ Cable
SFP28 25Gb Passive DAC Cables
7Z57A03557 AV1W Lenovo 1m Passive 25G SFP28 DAC Cable
7Z57A03558 AV1X Lenovo 3m Passive 25G SFP28 DAC Cable
7Z57A03559 AV1Y Lenovo 5m Passive 25G SFP28 DAC Cable
QSFP28 100G-to-4x25G Breakout Cables
7Z57A03564 AV22 Lenovo 1m 100G QSFP28 to 4x25G SFP28 Breakout DAC Cable
7Z57A03565 AV23 Lenovo 3m 100G QSFP28 to 4x25G SFP28 Breakout DAC Cable
7Z57A03566 AV24 Lenovo 5m 100G QSFP28 to 4x25G SFP28 Breakout DAC Cable

Features

The Mellanox ConnectX-5 EN 10/25GbE SFP28 Ethernet Adapter offers a number of features that are well suited for workloads, including the following:

Cloud and Web 2.0 Environments

ConnectX-5 adapter cards enable data center administrators to benefit from better server utilization and reduced costs, power usage, and cable complexity, allowing for more virtual appliances, virtual machines and tenants to co-exist on the same hardware.

Supported vSwitch/vRouter offload functions include:

  • Overlay Networks (e.g., VXLAN, NVGRE, MPLS, GENEVE, and NSH) header encapsulation & decapsulation.
  • Stateless offloads of inner packets and packet headers re-write, enabling NAT functionality and more.
  • Flexible and programmable parser and match-action tables, which enable hardware offloads for future protocols.
  • SR-IOV technology, providing dedicated adapter resources, guaranteed isolation and protection for virtual machines (VMs) within the server.
  • Network Function Virtualization (NFV), enabling a VM to be used as a virtual appliance. The full data- path operation offloads, hairpin hardware capability and service chaining enables data to be handled by the virtual appliance, with minimum CPU utilization.

Cloud and Web 2.0 customers developing platforms on Software Defined Network (SDN) environments are leveraging the Virtual-Switching capabilities of the server operating systems to achieve maximum flexibility. Open vSwitch (OvS) is an example of a virtual switch that allows Virtual Machines to communicate with each other and with the outside world. Traditionally residing in the hypervisor where switching is based on twelve-tuple matching onflows, the virtual switch, or virtual router software-based solution, is CPU-intensive. This can negatively affect system performance and prevent the full utilization of available bandwidth.

Mellanox ASAP2 (Accelerated Switching and Packet Processing) technology enables offloading the vSwitch/vRouter by handling the data plane in the NIC hardware, without modifying the control plane. This results in significantly higher vSwitch/vRouter performance without the associated CPU load.

Storage Environments

NVMe storage devices are gaining popularity by offering very fast storage access. The evolving NVMe over Fabric (NVMe-oF) protocol leverages the RDMA connectivity for remote access. ConnectX-5 offers further enhancements by providing NVMe-oF target offloads, enabling very efficient NVMe storage access with no CPU intervention, and thus improving performance and reducing  latency.

The embedded PCIe switch enables customers to build standalone storage or Machine Learning appliances. As with earlier generations of ConnectX adapters, standard block and file access protocols leverage RoCE for high-performance storage access. A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks.

ConnectX-5 enables an innovative storage rack design, Host Chaining, which enables different servers to interconnect without involving the top-of-rack switch. Leveraging Host Chaining, ConnectX-5 lowers the data center’s total cost of ownership by reducing CAPEX (cables, NICs, and switch port expenses). OPEX is also reduced by cutting down on switch port management and overall power usage.

Telecommunications

Telecommunications service providers are moving towards disaggregation, server virtualization, and orchestration as key tenets to modernize their networks. Likewise, they’re also moving towards Network Function Virtualization (NFV), which enables the rapid deployment of new network services. With this move, proprietary dedicated hardware and software, which tend to be static and difficult to scale, are being replaced with virtual machines running on commercial off-the-shelf (COTS) servers.

For telecom service providers, choosing the right networking hardware is critical to achieving a cloud-native NFV solution that is agile, reliable, fast and efficient. Telco service providers typically leverage virtualization and cloud technologies to better achieve agile service delivery and efficient scalability; these technologies require an advanced network infrastructure to support higher rates of packet processing. However, the resultant east-west traffic causes numerous interrupts as I/O traverses from kernel to user space, eats up CPU cycles and decreases packet performance. Particularly sensitive to delays are voice and video applications which often require less than 100ms of latency.

ConnectX-5 adapter cards drive extremely high packet rates, increased throughput and drive higher network efficiency through the following technologies; Open vSwitch Offloads (OvS), OvS over DPDK or ASAP2, Network Overlay Virtualization, SR-IOV, and RDMA. This allows for secure data delivery through higher-performance offloads, reducing CPU resource utilization, and boosting data center infrastructure efficiency. The result is a much more responsive and agile network capable of rapidly deploying network  services.

Technical specifications

Physical adapter:

  • PCIe 3.0 x8 host interface
  • Low profile form factor (142mm length, 69mm height)
  • Two SFP28 cages

Ethernet:

  • 25GbE / 10GbE
  • IEEE 802.3bj, 802.3bm 25 Gigabit Ethernet
  • IEEE 802.3by, Ethernet Consortium 25 Gigabit
  • IEEE 802.3ae 10 Gigabit Ethernet
  • IEEE 802.3ad Link Aggregation
  • IEEE 802.1Q, 802.1P VLAN tags and priority
  • IEEE 802.1Qau Congestion Notification
  • IEEE 802.1Qaz D0.2 ETS
  • IEEE 802.1Qbb D1.0 Priority-based Flow Control
  • IEEE 1588v2
  • Jumbo frame support (9600 byte)

Enhanced Features:

  • Hardware-based reliable transport
  • Collective operations offloads
  • Vector collective operations offloads
  • Mellanox PeerDirect RDMA (aka GPUDirect) communication acceleration
  • 64/66 encoding
  • Extended Reliable Connected transport (XRC)
  • Dynamically Connected Transport (DCT)
  • Enhanced Atomic operations
  • Advanced memory mapping support, allowing user mode registration and remapping of memory (UMR)
  • On demand paging (ODP)
  • MPI Tag Matching
  • Rendezvous protocol offload
  • Out-of-order RDMA supporting Adaptive Routing
  • Burst buffer offload
  • In-Network Memory registration-free RDMA memory access

CPU Offloads:

  • RDMA over Converged Ethernet (RoCE)
  • TCP/UDP/IP stateless offload
  • LSO, LRO, checksum offload
  • RSS (also on encapsulated packet), TSS, HDS, VLAN and MPLS tag insertion/stripping, Receive flow steering
  • Data Plane Development Kit (DPDK) for kernel bypass applications
  • Open VSwitch (OVS) offload using ASAP2
    • Flexible match-action flow tables
    • Tunneling encapsulation/ de-capsulation
  • Intelligent interrupt coalescence
  • Header rewrite supporting hardware offload of NAT router

Storage Offloads:

  • NVMe over Fabric offloads for target machine
  • T10 DIF – Signature handover operation at wire speed, for ingress and egress traffic
  • Storage protocols: SRP, iSER, NFS RDMA, SMB Direct, NVMe-oF

Overlay Networks:

  • RoCE over Overlay Networks
  • Stateless offloads for overlay network tunneling protocols
  • Hardware offload of encapsulation and decapsulation of VXLAN, NVGRE, and GENEVE overlay networks

Hardware-Based I/O Virtualization - Mellanox ASAP²:

  • Single Root IOV
  • Address translation and protection
  • VMware NetQueue support
    • SR-IOV: Up to 512 Virtual Functions
    • SR-IOV: Up to 8 Physical Functions per host
  • Virtualization hierarchies (e.g., NPAR when enabled)
    • Virtualizing Physical Functions on a physical port
    • SR-IOV on every Physical Function
  • Configurable and user-programmable QoS
  • Guaranteed QoS for VMs

Management and Control:

  • NC-SI over MCTP over SMBus and NC-SI over MCTP over PCIe - Baseboard Management Controller interface
  • PLDM for Monitor and Control DSP0248
  • PLDM for Firmware Update DSP0267
  • SDN management interface for managing the eSwitch
  • I2C interface for device control and configuration
  • General Purpose I/O pins
  • SPI interface to Flash
  • JTAG IEEE 1149.1 and IEEE 1149.6

Remote Boot:

  • Remote boot over Ethernet
  • Remote boot over iSCSI
  • Unified Extensible Firmware Interface (UEFI)
  • Pre-execution Environment (PXE)

Server support

The following tables list the ThinkSystem servers that are compatible.

Table 5. Server support (Part 1 of 2)
Part Number Description E 2S Intel V2 AMD Dense V2 4S V2 8S
SE350 (7Z46 / 7D1X)
ST650 V2 (7Z75 / 7Z74)
SR630 V2 (7Z70 / 7Z71)
SR650 V2 (7Z72 / 7Z73)
SR670 V2 (7Z22 / 7Z23)
SR635 (7Y98 / 7Y99)
SR655 (7Y00 / 7Z01)
SR645 (7D2Y / 7D2X)
SR665 (7D2W / 7D2V)
SD630 V2 (7D1K)
SD650 V2 (7D1M)
SD650-N V2 (7D1N)
SN550 V2 (7Z69)
SR850 V2 (7D31 / 7D32)
SR860 V2 (7Z59 / 7Z60)
SR950 (7X11 / 7X12)
4XC7A62574 ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 2-port PCIe Ethernet Adapter N N N N N N N N N N N N N N N N
Table 6. Server support (Part 2 of 2)
Part Number Description 1S Intel 2S Intel V1 Dense V1 4S V1
ST50 (7Y48 / 7Y50)
ST250 (7Y45 / 7Y46)
SR150 (7Y54)
SR250 (7Y51 / 7Y52)
ST550 (7X09 / 7X10)
SR530 (7X07 / 7X08)
SR550 (7X03 / 7X04)
SR570 (7Y02 / 7Y03)
SR590 (7X98 / 7X99)
SR630 (7X01 / 7X02)
SR650 (7X05 / 7X06)
SR670 (7Y36 / 7Y37)
SD530 (7X21)
SD650 (7X58)
SN550 (7X16)
SN850 (7X15)
SR850 (7X18 / 7X19)
SR850P (7D2F / 2D2G)
SR860 (7X69 / 7X70)
4XC7A62574 ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 2-port PCIe Ethernet Adapter N N N N N N N N N Y Y N N N N N N N N

Operating system support

The adapter supports the operating systems listed in the following table.

Tip: This table is automatically generated based on data from Lenovo ServerProven.

Table 7. Operating system support for ThinkSystem Mellanox ConnectX-5 EN 10/25GbE SFP28 2-port PCIe Ethernet Adapter, 4XC7A62574
Operating systems
SR630 (Gen 2)
SR650 (Gen 2)
SR630 (Gen 1)
SR650 (Gen 1)
Microsoft Windows Server 2016 Y Y Y Y
Microsoft Windows Server 2019 Y Y Y Y
Red Hat Enterprise Linux 7.6 Y Y Y Y
Red Hat Enterprise Linux 7.7 Y Y Y Y
Red Hat Enterprise Linux 7.8 Y Y Y Y
Red Hat Enterprise Linux 7.9 Y Y Y Y
Red Hat Enterprise Linux 8.0 Y Y Y Y
Red Hat Enterprise Linux 8.1 Y Y Y Y
Red Hat Enterprise Linux 8.2 Y Y Y Y
Red Hat Enterprise Linux 8.3 Y Y Y Y
SUSE Linux Enterprise Server 12 SP4 Y Y Y Y
SUSE Linux Enterprise Server 12 SP4 with Xen Y Y Y Y
SUSE Linux Enterprise Server 12 SP5 Y Y Y Y
SUSE Linux Enterprise Server 12 SP5 with Xen Y Y Y Y
SUSE Linux Enterprise Server 15 Y Y Y Y
SUSE Linux Enterprise Server 15 SP1 Y Y Y Y
SUSE Linux Enterprise Server 15 SP1 with Xen Y Y Y Y
SUSE Linux Enterprise Server 15 SP2 Y Y Y Y
SUSE Linux Enterprise Server 15 SP2 with Xen Y Y Y Y
SUSE Linux Enterprise Server 15 with Xen Y Y Y Y
VMware vSphere Hypervisor (ESXi) 6.5 N N Y Y
VMware vSphere Hypervisor (ESXi) 6.5 U1 N N Y Y
VMware vSphere Hypervisor (ESXi) 6.5 U2 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 6.5 U3 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 6.7 N N Y Y
VMware vSphere Hypervisor (ESXi) 6.7 U1 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 6.7 U2 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 6.7 U3 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 7.0 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 7.0 U1 Y Y Y Y
VMware vSphere Hypervisor (ESXi) 7.0 U2 Y Y Y Y

Regulatory approvals

  • Safety: CB / cTUVus / CE
  • EMC: CE / FCC / VCCI / ICES / RCM
  • RoHS compliant

Operating environment

Power consumption:

  • Typical power: 9.5W (passive cable)
  • Maximum power: 11.8 W (passive cable), 15.1W (1.5W active cable)

Maximum power through external connectors: 1.5W

Temperature:

  • Operational 0°C to 55°C
  • Non-operational -40°C to 70°C

Humidity: 90% relative humidity

Warranty

One year limited warranty. When installed in a Lenovo server, this adapter assumes the server’s base warranty and any warranty upgrades.

Related product families

Product families related to this document are the following:

Trademarks

Lenovo and the Lenovo logo are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. A current list of Lenovo trademarks is available on the Web at https://www.lenovo.com/us/en/legal/copytrade/.

The following terms are trademarks of Lenovo in the United States, other countries, or both:
Lenovo®
ServerProven®
ThinkSystem

The following terms are trademarks of other companies:

Intel® is a trademark of Intel Corporation or its subsidiaries.

Linux® is the trademark of Linus Torvalds in the U.S. and other countries.

Microsoft®, Windows Server®, and Windows® are trademarks of Microsoft Corporation in the United States, other countries, or both.

Other company, product, or service names may be trademarks or service marks of others.