As prospects migrate to community materials based mostly on Virtual Extensible Local Area Network/Ethernet Virtual Private Network (VXLAN/EVPN) expertise, questions concerning the implications for software efficiency, Quality of Service (QoS) mechanisms, and congestion avoidance typically come up. This weblog submit addresses a number of the widespread areas of confusion and concern, and touches on a number of finest practices for maximizing the worth of utilizing Cisco Nexus 9000 switches for Data Center material deployments by leveraging the obtainable Intelligent Buffering capabilities.
What Is the Intelligent Buffering Capability in Nexus 9000?
Cisco Nexus 9000 sequence switches implement an egress-buffered shared-memory structure, as proven in Figure 1. Each bodily interface has 8 user-configurable output queues that contend for shared buffer capability when congestion happens. A buffer admission algorithm known as Dynamic Buffer Protection (DBP), enabled by default, ensures honest entry to the obtainable buffer amongst any congested queues.
In addition to DBP, two key options – Approximate Fair Drop (AFD) and Dynamic Packet Prioritization (DPP) – assist to hurry preliminary circulation institution, cut back flow-completion time, keep away from congestion buildup, and keep buffer headroom for absorbing microbursts.
AFD makes use of in-built {hardware} capabilities to separate particular person 5-tuple flows into two classes – elephant flows and mouse flows:
- Elephant flows are longer-lived, sustained bandwidth flows that may profit from congestion management indicators reminiscent of Explicit Congestion Notification (ECN) Congestion Experienced (CE) marking, or random discards, that affect the windowing habits of Transmission Control Protocol (TCP) stacks. The TCP windowing mechanism controls the transmission price of TCP periods, backing off the transmission price when ECN CE markings, or un-acknowledged sequence numbers, are noticed (see the “More Information” part for extra particulars).
- Mouse flows are shorter-lived flows which can be unlikely to learn from TCP congestion management mechanisms. These flows include the preliminary TCP 3-way handshake that establishes the session, together with a comparatively small variety of further packets, and are subsequently terminated. By the time any congestion management is signaled for the circulation, the circulation is already full.
As proven in Figure 2, with AFD, elephant flows are additional characterised in line with their relative bandwidth utilization – a high-bandwidth elephant circulation has a better chance of experiencing ECN CE marking, or discards, than a lower-bandwidth elephant circulation. A mouse circulation has a zero chance of being marked or discarded by AFD.
For readers acquainted with the older Weighted Random Early Detect (WRED) mechanism, you may consider AFD as a sort of “bandwidth-aware WRED.” With WRED, any packet (no matter whether or not it’s a part of a mouse circulation or an elephant circulation) is doubtlessly topic to marking or discards. In distinction, with AFD, solely packets belonging to sustained-bandwidth elephant flows could also be marked or discarded – with higher-bandwidth elephants extra prone to be impacted than lower-bandwidth elephants – whereas a mouse circulation is rarely impacted by these mechanisms.
Additionally, AFD marking or discard chance for elephants will increase because the queue turns into extra congested. This habits ensures that TCP stacks again off nicely earlier than all of the obtainable buffer is consumed, avoiding additional congestion and making certain that considerable buffer headroom nonetheless stays to soak up instantaneous bursts of back-to-back packets on beforehand uncongested queues.
DPP, one other hardware-based functionality, promotes the preliminary packets in a newly noticed circulation to a better precedence queue than it could have traversed “naturally.” Take for instance a brand new TCP session institution, consisting of the TCP 3-way handshake. If any of those packets sit in a congested queue, and due to this fact expertise further delay, it could actually materially have an effect on software efficiency.
As proven in Figure 3, as a substitute of enqueuing these packets of their initially assigned queue, the place congestion is doubtlessly extra doubtless, DPP will promote these preliminary packets to a higher-priority queue – a strict precedence (SP) queue, or just a higher-weighted Deficit Weighted Round-Robin (DWRR) queue – which leads to expedited packet supply with a really low likelihood of congestion.
If the circulation continues past a configurable variety of packets, packets are not promoted – subsequent packets within the circulation traverse the initially assigned queue. Meanwhile, different newly noticed flows could be promoted and revel in the good thing about sooner session institution and circulation completion for short-lived flows.
AFD and UDP Traffic
One steadily requested query about AFD is that if it’s acceptable to make use of it with User Datagram Protocol (UDP) site visitors. AFD by itself doesn’t distinguish between totally different protocol sorts, it solely determines if a given 5-tuple circulation is an elephant or not. We typically state that AFD shouldn’t be enabled on queues that carry non-TCP site visitors. That’s an oversimplification, in fact – for instance, a low-bandwidth UDP software would by no means be topic to AFD marking or discards as a result of it could by no means be flagged as an elephant circulation within the first place.
Recall that AFD can both mark site visitors with ECN, or it could actually discard site visitors. With ECN marking, collateral injury to a UDP-enabled software is unlikely. If ECN CE is marked, both the appliance is ECN-aware and would modify its transmission price, or it could ignore the marking utterly. That mentioned, AFD with ECN marking received’t assist a lot with congestion avoidance if the UDP-based software shouldn’t be ECN-aware.
On the opposite hand, in the event you configure AFD in discard mode, sustained-bandwidth UDP functions could undergo efficiency points. UDP doesn’t have any inbuilt congestion-management mechanisms – discarded packets would merely by no means be delivered and wouldn’t be retransmitted, at the least not based mostly on any UDP mechanism. Because AFD is configurable on a per-queue foundation, it’s higher on this case to easily classify site visitors by protocol, and make sure that site visitors from high-bandwidth UDP-based functions at all times makes use of a non-AFD-enabled queue.
What Is a VXLAN/EVPN Fabric?
VXLAN/EVPN is without doubt one of the quickest rising Data Center material applied sciences in latest reminiscence. VXLAN/EVPN consists of two key parts: the data-plane encapsulation, VXLAN; and the control-plane protocol, EVPN.
You can discover considerable particulars and discussions of those applied sciences on cisco.com, in addition to from many different sources. While an in-depth dialogue is outdoors the scope of this weblog submit, when speaking about QOS and congestion administration within the context of a VXLAN/EVPN material, the data-plane encapsulation is the main focus. Figure 4 illustratates the VXLAN data-plane encapsulation, with emphasis on the inside and outer DSCP/ECN fields.
As you may see, VXLAN encapsulates overlay packets in IP/UDP/VXLAN “outer” headers. Both the inside and outer headers comprise the DSCP and ECN fields.
With VXLAN, a Cisco Nexus 9000 swap serving as an ingress VXLAN tunnel endpoint (VTEP) takes a packet originated by an overlay workload, encapsulates it in VXLAN, and forwards it into the material. In the method, the swap copies the inside packet’s DSCP and ECN values to the outer headers when performing encapsulation.
Transit gadgets reminiscent of material spines ahead the packet based mostly on the outer headers to achieve the egress VTEP, which decapsulates the packet and transmits it unencapsulated to the ultimate vacation spot. By default, each the DSCP and ECN fields are copied from the outer IP header into the inside (now decapsulated) IP header.
In the method of traversing the material, overlay site visitors could cross by means of a number of switches, every imposing QOS and queuing insurance policies outlined by the community administrator. These insurance policies may merely be default configurations, or they might include extra advanced insurance policies reminiscent of classifying totally different functions or site visitors sorts, assigning them to distinctive courses, and controlling the scheduling and congestion administration habits for every class.
How Do the Intelligent Buffer Capabilities Work in a VXLAN Fabric?
Given that the VXLAN data-plane is an encapsulation, packets traversing material switches include the unique TCP, UDP, or different protocol packet inside a IP/UDP/VXLAN wrapper. Which results in the query: how do the Intelligent Buffer mechanisms behave with such site visitors?
As mentioned earlier, sustained-bandwidth UDP functions may doubtlessly undergo from efficiency points if traversing an AFD-enabled queue. However, we must always make a really key distinction right here – VXLAN is not a “native” UDP software, however somewhat a UDP-based tunnel encapsulation. While there isn’t any congestion consciousness on the tunnel degree, the unique tunneled packets can carry any sort of software site visitors –TCP, UDP, or nearly another protocol.
Thus, for a TCP-based overlay software, if AFD both marks or discards a VXLAN-encapsulated packet, the unique TCP stack nonetheless receives ECN marked packets or misses a TCP sequence quantity, and these mechanisms will trigger TCP to cut back the transmission price. In different phrases, the unique purpose remains to be achieved – congestion is prevented by inflicting the functions to cut back their price.
Similarly, high-bandwidth UDP-based overlay functions would reply simply as they’d to AFD marking or discards in a non-VXLAN setting. If you have got high-bandwidth UDP-based functions, we suggest classifying based mostly on protocol and making certain these functions get assigned to non-AFD-enabled queues.
As for DPP, whereas TCP-based overlay functions will profit most, particularly for preliminary flow-setup, UDP-based overlay functions can profit as nicely. With DPP, each TCP and UDP short-lived flows are promoted to a better precedence queue, dashing flow-completion time. Therefore, enabling DPP on any queue, even these carrying UDP site visitors, ought to present a constructive impression.
Key Takeaways
VXLAN/EVPN material designs have gained important traction lately, and making certain wonderful software efficiency is paramount. Cisco Nexus 9000 Series switches, with their hardware-based Intelligent Buffering capabilities, make sure that even in an overlay software setting, you may maximize the environment friendly utilization of obtainable buffer, reduce community congestion, velocity flow-establishment and flow-completion instances, and keep away from drops as a consequence of microbursts.
More Information
You can discover extra details about the applied sciences mentioned on this weblog at www.cisco.com:
Share: