<?xmlversion="1.0" encoding="US-ASCII"?> <?xml-model href="rfc7991bis.rnc"?> <!-- <?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?> --> <!-- This third-party XSLT can be enabled for direct transformations in XML processors, including most browsers -->version='1.0' encoding='UTF-8'?> <!DOCTYPE rfc [ <!ENTITY nbsp " "> <!ENTITY zwsp "​"> <!ENTITY nbhy "‑"> <!ENTITY wj "⁠"> ]><?rfc strict="yes" ?> <!-- give errors regarding ID-nits and DTD validation --> <!-- control the table of contents (ToC) --> <?rfc toc="yes"?> <!-- generate a ToC --> <?rfc tocdepth="4"?> <!-- the number of levels of subsections in ToC. default: 3 --> <!-- control references --> <?rfc symrefs="yes"?> <!-- use symbolic references tags, i.e, [RFC2119] instead of [1] --> <?rfc sortrefs="yes" ?> <!-- sort the reference entries alphabetically --> <!-- control vertical white space (using these PIs as follows is recommended by the RFC Editor) --> <?rfc compact="yes" ?> <!-- do not start each main section on a new page --> <?rfc subcompact="no" ?> <!-- keep one blank line between list items --> <!-- end of list of popular I-D processing instructions --><rfccategory="std"xmlns:xi="http://www.w3.org/2001/XInclude" category="std" docName="draft-ietf-bess-evpn-vpws-fxc-12" number="9744" consensus="true" submissionType="IETF" ipr="trust200902" tocInclude="true" tocDepth="4" symRefs="true"sortRefs="true"> <!-- ***** FRONT MATTER ***** --> <front>sortRefs="true" version="3" xml:lang="en" updates="" obsoletes=""> <!--The abbreviated title is used in the page header - it is only necessary if[rfced] Please note that thefulltitleis longer than 39 charactersof the document has been updated as follows. Abbreviations have been expanded per Section 3.6 of RFC 7322 ("RFC Style Guide"). Please review. Original: EVPN VPWS Flexible Cross-Connect Service Current: EVPN Virtual Private Wire Service (VPWS) Flexible Cross-Connect (FXC) Service --> <front> <title abbrev="EVPN VPWSFlexible Cross-Connect">EVPN VPWSFXC Service">EVPN Virtual Private Wire Service (VPWS) Flexible Cross-Connect (FXC) Service</title> <seriesInfoname="Internet-Draft" value="draft-ietf-bess-evpn-vpws-fxc-12"/> <!-- add 'role="editor"' below for the editors if appropriate --> <!-- Another author who claims to be an editor -->name="RFC" value="9744"/> <author fullname="Ali Sajassi" initials="A." surname="Sajassi" role="editor"> <organization>Cisco Systems</organization> <address> <email>sajassi@cisco.com</email> </address> </author> <author fullname="Patrice Brissette" initials="P." surname="Brissette"> <organization>Cisco Systems</organization> <address> <email>pbrisset@cisco.com</email> </address> </author> <author fullname="James Uttaro" initials="J." surname="Uttaro"> <organization>AT&T</organization> <address> <email>uttaro@att.com</email> </address> </author> <author fullname="John Drake" initials="J." surname="Drake"> <organization>Juniper Networks</organization> <address> <email>jdrake@juniper.net</email> </address> </author> <author fullname="Sami Boutros" initials="S." surname="Boutros"> <organization>Ciena</organization> <address> <email>sboutros@ciena.com</email> </address> </author> <author fullname="Jorge Rabadan" initials="J." surname="Rabadan"> <organization>Nokia</organization> <address> <email>jorge.rabadan@nokia.com</email> </address> </author> <dateyear="2024" /> <!-- Meta-data Declarations --> <area>General</area> <workgroup>BESS Working Group</workgroup> <!-- WG name at the upperleft corner of the doc, IETF is fine for individual submissions. If this element is not present, the default is "Network Working Group", which is used by the RFC Editor as a nod to the history of the IETF. -->year="2025" month="February"/> <area>RTG</area> <workgroup>bess</workgroup> <keyword>EVPN</keyword> <keyword>VPWS</keyword> <keyword>Flexible Cross Connect</keyword> <keyword>FXC</keyword> <abstract> <t>This document describes a new EVPNVPWSVirtual Private Wire Service (VPWS) service type specifically for multiplexing multiple attachment circuits across different Ethernet Segments and physical interfaces into a single EVPN VPWS service tunnel and still providing Single-Active and All-Active multi-homing. This new service is referred to as the EVPN VPWSflexible cross-connectFlexible Cross-Connect (FXC) service. This document specifies a solution based on extensions to EVPNVPWS(RFC8214)VPWS (RFC 8214), which in turn is based on extensions to EVPN(RFC7432).(RFC 7432). Therefore, a thorough understanding ofRFC7432RFCs 7432 andRFC82148214 are prerequisites for thisdocument. </t>document.</t> </abstract><note title="Requirements Language"> <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 <xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when, they appear in all capitals, as shown here. </t> </note></front> <middle> <sectiontitle="Introduction"anchor="intro"> <name>Introduction</name> <t><xref target="RFC8214"/> describes a solution to deliverP2Ppoint-to-point (P2P) services using BGP constructs defined in <xref target="RFC7432"/>. <!-- [rfced] We're having trouble understanding "can designate on" in the text below. Should this be updated to "can designate"? Original: [RFC8214] describes a solution to deliver P2P services using BGP constructs defined in [RFC7432]. It delivers this P2P service between a pair of Attachment Circuits (ACs), where an AC can designate on a PE, a port, a VLAN on a port, or a group of VLANs on a port. Perhaps: [RFC8214] describes a solution to deliver P2P services using BGP constructs defined in [RFC7432]. It delivers this P2P service between a pair of Attachment Circuits (ACs), where an AC can designate a PE, a port, a VLAN on a port, or a group of VLANs on a port. --> It delivers this P2P service between a pair of Attachment Circuits (ACs), where an AC can designate on a Provider Edge (PE), a port, a VLAN on a port, or a group of VLANs on a port. It also leverages multi-homing and fast convergence capabilities of <xref target="RFC7432"/> in delivering these VPWS services.Multi&nbhy;homingMulti-homing capabilities include the support of single-active andall&nbhy;activeall-active redundancymodemode, and fast convergence is provided using a "mass withdrawal" message incontrol-planecontrol plane and fast protection switching usingprefix independentprefix-independent convergence indata-planea data plane upon node or link failure <xref target="I-D.ietf-rtgwg-bgp-pic"/>. Furthermore, the use of EVPN BGP constructs eliminates the need for multi-segment pseudowireauto&nbhy;discoveryauto-discovery and signaling if the VPWS service need to span across multipleASesAutonomous Systems (ASes) <xref target="RFC5659"/>.</t> <t>Service providers have a very large number of ACs (in millions) that need to be backhauled across their MPLS/IP network. These ACs may or may not require tag manipulation (e.g., VLAN translation). These service providers want to multiplex a large number of ACs across several physical interfaces spread across one or more PEs (e.g., several Ethernet Segments) onto a single VPWS service tunnel in order to a) reduce the number of EVPN service labels associated with EVPN-VPWS service tunnels and thus the associatedOAM monitoring,Operations, Administration, and Maintenance (OAM) monitoring and b) reduce EVPN BGP signaling (e.g., not to signal each AC as it is the case in <xref target="RFC8214"/>).</t> <t>Service providers want the above functionality without sacrificing any of the capabilities of <xref target="RFC8214"/> includingsingle- activesingle-active and all-activemulti-homing,multi-homing and fast convergence.</t> <t>This document specifies a solution based on extensions to EVPN VPWS(<xref target="RFC8214"/>)<xref target="RFC8214"/> to meet the above requirements. Furthermore, <xref target="RFC8214"/> is itself based on extensions to EVPN(<xref target="RFC7432"/>).<xref target="RFC7432"/>. Therefore, a thorough understanding of <xref target="RFC7432"/> and <xref target="RFC8214"/> are prerequisites for this document. </t> <sectiontitle="Terminology"anchor="terminology"><t> <list style="hanging" hangIndent="3"> <t hangText="AC: ">Attachment Circuit</t> <t hangText="ES: ">Ethernet Segment</t> <t hangText="ESI: ">Ethernet<name>Terminology</name> <dl newline="false" spacing="normal" indent="3"> <dt>AC:</dt> <dd>Attachment Circuit</dd> <dt>ES:</dt> <dd>Ethernet Segment</dd> <dt>ESI:</dt> <dd>Ethernet SegmentIdentififer</t> <t hangText="EVI: ">EVPNIdentifier</dd> <dt>EVI:</dt> <dd>EVPN InstanceIdentifier</t> <t hangText="EVPN:">EthernetIdentifier</dd> <dt>EVPN:</dt> <dd>Ethernet Virtual PrivateNetwork</t> <t hangText="Ethernet A-D:">EthernetNetwork</dd> <!--[rfced] To have a 1:1 matchup between the following abbreviations and their expansions, may we update as follows? Original: Ethernet A-D: Ethernet Auto-Discovery (A-D) per EVI and Ethernet A-D per ESI routes, as defined in [RFC7432] and [RFC8214]. ... PE: Provider Edge device ... VRF: VPN Routing and Forwarding table ... IP-VRF: VPN Routing and Forwarding table, for IP lookup ... MAC-VRF: VPN Routing and Forwarding table, for MAC lookup ... VID-VRF: VPN Routing and Forwarding table, for Normalized VID lookup Perhaps: Ethernet A-D: Ethernet Auto-Discovery (per EVI and per ESI routes, as defined in [RFC7432] and [RFC8214]) ... PE: Provider Edge ... VRF: VPN Routing and Forwarding ... IP-VRF: VPN Routing and Forwarding for IP lookup MAC-VRF: VPN Routing and Forwarding for MAC lookup VID-VRF: VPN Routing and Forwarding for normalized VID lookup --> <dt>Ethernet A-D:</dt> <dd>Ethernet Auto-Discovery per EVI and Ethernet A-D per ESI routes, as defined in <xref target="RFC7432"/> and <xreftarget="RFC8214"/>.</t> <t hangText="FXC: ">Flexibletarget="RFC8214"/>.</dd> <dt>FXC:</dt> <dd>Flexible CrossConnect</t> <t hangText="MAC: ">MediaConnect</dd> <dt>MAC:</dt> <dd>Media AccessControl</t> <t hangText="MPLS:">Multi ProtocolControl</dd> <dt>MPLS:</dt> <dd>Multiprotocol LabelSwitching</t> <t hangText="OAM: ">Operations, Administration and Maintenance</t> <t hangText="PE: ">ProviderSwitching</dd> <dt>OAM:</dt> <dd>Operations, Administration, and Maintenance</dd> <dt>PE:</dt> <dd>Provider Edgedevice</t> <t hangText="VCCV:">Virtual circuit connection verification</t> <t hangText="VID: ">Vlan ID</t> <t hangText="VPWS:">Virtual private wire service</t> <t hangText="VRF: ">VPNdevice</dd> <dt>VCCV:</dt> <dd>Virtual Circuit Connection Verification</dd> <dt>VID:</dt> <dd>VLAN ID</dd> <dt>VPWS:</dt> <dd>Virtual Private Wire Service</dd> <dt>VRF:</dt> <dd>VPN Routing and Forwardingtable</t> <t hangText="IP-VRF: ">VPNtable</dd> <dt>IP-VRF:</dt> <dd>VPN Routing and Forwarding table, for IPlookup</t> <t hangText="MAC-VRF: ">VPNlookup</dd> <dt>MAC-VRF:</dt> <dd>VPN Routing and Forwarding table, for MAClookup</t> <t hangText="VID-VRF: ">VPNlookup</dd> <dt>VID-VRF:</dt> <dd>VPN Routing and Forwarding table, for Normalized VIDlookup</t> <t hangText="VPWSlookup</dd> <dt>VPWS ServiceTunnel:">ItTunnel:</dt> <dd>It is represented by a pair of EVPN service labels associated with a pair of endpoints. Each label is downstream assigned and advertised by the disposition PE through an EthernetAuto-Discovery (A-D)A-D per EVI route. The downstream label identifies the endpoint on the disposition PE. A VPWS service tunnel can be associated with many VPWS service identifiers where each identifier is a normalizedVID.</t> <t hangText="Single-ActiveVID.</dd> <dt>Single-Active RedundancyMode:">WhenMode:</dt> <dd>When a device or a network ismulti&nbhy;homedmulti-homed to two or more PEs and when only a single PE in such redundancy group can forward traffic to/from the multi-homed device or network for a given VLAN, then such multi-homing or redundancy is referred to as"Single-Active".</t> <t hangText="All-Active"Single-Active".</dd> <dt>All-Active RedundancyMode:">WhenMode:</dt> <dd>When a device or a network ismulti&nbhy;homedmulti-homed to two or more PEs and when all PEs in such redundancy group can forward traffic to/from the multi-homed device or network for a given VLAN, then such multi-homing or redundancy is referred to as"All-Active".</t> </list>"All-Active".</dd> </dl> </section> <section> <name>Requirements Language</name> <t>The key words "<bcp14>MUST</bcp14>", "<bcp14>MUST NOT</bcp14>", "<bcp14>REQUIRED</bcp14>", "<bcp14>SHALL</bcp14>", "<bcp14>SHALL NOT</bcp14>", "<bcp14>SHOULD</bcp14>", "<bcp14>SHOULD NOT</bcp14>", "<bcp14>RECOMMENDED</bcp14>", "<bcp14>NOT RECOMMENDED</bcp14>", "<bcp14>MAY</bcp14>", and "<bcp14>OPTIONAL</bcp14>" in this document are to be interpreted as described in BCP 14 <xref target="RFC2119"/> <xref target="RFC8174"/> when, and only when, they appear in all capitals, as shown here. </t> </section> </section> <sectiontitle="Requirements"anchor="requirements"> <name>Requirements</name> <t>Two of the main motivations for service providers seeking a new solution are: 1) to reduce the number of VPWS service tunnels by multiplexing a large number of ACs across different physical interfaces instead of having one VPWS service tunnel perAC,AC and 2) to reduce the signaling of ACs as much as possible. Besides these two requirements, they also want the multi-homing and fast convergence capabilities of <xref target="RFC8214"/>.</t> <t>In <xref target="RFC8214"/>, a PE signals an AC indirectly by first associating that AC to a VPWS service tunnel (e.g., a VPWS service instance) and then signaling the VPWS service tunnel viaaan Ethernet A-D per EVI route with the Ethernet Tag field set to a 24-bit VPWS service instance identifier (which is unique within the EVI) and the ESI field set to a 10-octet identifier of the Ethernet Segment corresponding to that AC.</t> <t>Therefore, a PE device that receives such EVPNroutes,routes can associate the VPWS service tunnel to the remote Ethernet Segment using the ESIfield, andfield. Additionally, when the remote ES fails and the PE receives the "mass withdrawal" message associated with the failed ES per <xref target="RFC7432"/>,ita PE device can quickly update its BGP list of available remote entries to invalidate all VPWS service tunnels sharing the ESI field and achieve fast convergence for multi-homing scenarios. Even if fast convergencewerewas not needed, there would still be a need for signaling each AC failure (via its corresponding VPWS service tunnel) associated with the failedES,ES so that the BGP path list for each of them gets updated accordingly and the packets are sent to a backup PE (in case ofsingle- activesingle-active multi-homing) or to other PEs in the redundancy group (in case of all-active multi-homing). In the absence of updating the BGP path list, the traffic for that VPWS service tunnel will beblack&nbhy;holed.</t> <t> Whenblack-holed.</t> <t>When a single VPWS service tunnel carries multiple ACs across various Ethernet Segments (physical interfaces) without signaling the ACs via EVPN BGP to remote PE devices, those remote PE devices lack the information to associate the received Ethernet Segment with these ACs or with their local ACs. They also lack the association between the VPWS service tunnel (e.g., EVPN service label) and the far-end ACs. This meansthatthat, while the remote PEs can associate their local ACs with the VPWS service tunnel, they cannot make similar associations for the far-endACs. <br/> Consequently,ACs.</t> <t>Consequently, in case of a connectivity failure to the ES, the remote PEs are unable to redirect traffic via another multi-homing PE to that ES. In other words, even if an ES failure is signaled via EVPN to the remote PE devices, they cannot effectively respond because they do not know the relationship between the remote ES, the remote ACs, and the VPWS service tunnel.</t> <t>To address this issue when multiplexing a large number of ACs onto a single VPWS service tunnel, two mechanisms have been developed: one to support VPWS services between two single-homedendpoints,endpoints and another one to support VPWS services where one of the endpoints is multi-homed.</t><t> For<t>For single-homed endpoints, it is acceptable not to signal each AC in BGP because, in the event of a connection failure to the ES, there is no alternative path to that endpoint. However, the implication of not signaling an AC failure is that the traffic destined for the failed AC is sent over the MPLS/IP core and then discarded at the destination PE, thereby potentially wasting networkresources. <br/> Thisresources.</t> <t>This waste of network resources during a connection failure may be transient, as it can be detected and prevented at the application layer in certain cases. <xref target="fxc"/> outlines a solution for such single-homing VPWS services.</t><t> For<t>For VPWS services where one of the endpoints is multi-homed, there are two options: </t><t>1) to signal<ol spacing="normal" type="1"> <li>Signal each AC via BGP, allowing the path list to be updated upon a failure affecting those ACs. This solution is described in <xref target="vlan_sig_fxc"/> and is referred to as theVLAN-signaled"VLAN-signaled flexible cross-connectservice.</t> <t>2) to bundleservice".</li> <li>Bundle several ACs on an ES together per destination endpoint (e.g., ES, MAC-VRF, etc.) and associate such a bundle with a single VPWS service tunnel. This approach is similar to the VLAN-bundle service interface described in <xref target="RFC8214"/>. This solution is described in <xreftarget="fxc_mh"/>.</t>target="fxc_mh"/>.</li> </ol> </section> <sectiontitle="Solution"anchor="solution"><t> This<name>Solution</name> <t>This section specifies how to provide a new VPWS service between two PE devices where a large number of ACs (such as VLANs) that span across multiple Ethernet Segments (physical interfaces) on each PE are multiplexed onto a single P2P EVPN service tunnel. Since the multiplexing involves several physical interfaces, there can be overlapping VLAN IDs (VIDs) across these interfaces. In such cases, theVLAN IDs (VIDs)VIDs must be translated into unique VIDs to prevent collisions. Furthermore, if the number of VLANs being multiplexed onto a single VPWS service tunnel exceeds 4095, then a single tag to double tag translation must be performed. This translation of VIDs into unique VIDs (either single or double) results in a "Normalized VID".</t><t> When<!-- [rfced] We were unable to find exactly "12-bit VPWS service instance identifiers" in [RFC8214]. We did find the following in Section 3 of [RFC8214]: The VPWS service instance identifier value MAY be set to a 24-bit value, and when a 24-bit value is used, it MUST be right-aligned. For a more accurate citation, may we update to something like the following? Current: As stated in [RFC8214], 12-bit and 24-bit VPWS service instance identifiers representing normalized VIDs MUST be right-aligned. Perhaps: 24-bit VPWS service instance identifiers [RFC8214] as well as 12-bit VPWS service instance identifiers representing normalized VIDs MUST be right-aligned. --> <t>When a single normalized VID is used, the lower 12 bits of the Ethernet Tag ID field in EVPN routesMUST<bcp14>MUST</bcp14> be set to that VID. When a double normalized VID is used, the lower 12 bits of the Ethernet Tag ID fieldMUST<bcp14>MUST</bcp14> be set to the inner VID, while the higher 12 bits are set to the outer VID. As stated in <xref target="RFC8214"/>, 12-bit and 24-bit VPWS service instance identifiers representing normalized VIDsMUST<bcp14>MUST</bcp14> be right-aligned.</t><t> Since<t>Since there is only a single EVPN VPWS service tunnel associated with many normalized VIDs (either single or double) across multiple physical interfaces, an MPLS lookup at the disposition PE is no longer sufficient to forward the packet to the correct egress endpoint or interface. Therefore, in addition to an EVPN label lookup corresponding to the VPWS service tunnel, a VID lookup (either single or double) is also required. At the disposition PE, the EVPN label lookup identifies a VID-VRF, and the lookup of the normalizedVID(s)VIDs within that table identifies the appropriate egress endpoint or interface. The tag manipulation (translation from normalizedVID(s)VIDs to the local VID)SHOULD<bcp14>SHOULD</bcp14> be performed either as part of the VID table lookup or at the egress interface itself.</t><t> Since<t>Since the VID lookup (single or double) needs to be performed at the disposition PE, VID normalizationMUST<bcp14>MUST</bcp14> be completed prior to MPLS encapsulation on the ingress PE. This requires that both the imposition and disposition PE devices be capable of VLAN tag manipulation, such as rewriting (single or double),addition,adding, ordeletiondeleting (single or double) at their endpoints (e.g., their ESs, MAC-VRFs, IP-VRFs, etc.). Operators should be informed of potential trade-offs from a performance standpoint, compared to typical pseudowire processing.</t> <sectiontitle="VPWS Service Identifiers"anchor="svc_ids"> <name>VPWS Service Identifiers</name> <t>In <xref target="RFC8214"/>, a unique value identifying the service is signaled in the context of each PE's EVI. The 32-bit Ethernet Tag ID fieldMUST<bcp14>MUST</bcp14> be set to this VPWS service instance identifier value. Translation at an Autonomous System Border Router (ASBR) is needed if re-advertising to another AS affects uniqueness.</t> <t>For FXC, this same Ethernet Tag ID field value is an identifierwhichthat mayrepresent: <list style="symbols"> <t>VLAN-Bundlerepresent:</t> <ul spacing="normal"> <li>VLAN-Bundle Service Interface: a unique value for a group ofVLANs ;</t> <t>VLAN-AwareVLANs</li> <li>VLAN-Aware Bundle ServiceInterface :Interface: a unique value for individualVLANs,VLANs and is considered the same as the normalizedVID.</t> </list> </t>VID</li> </ul> <t>Both the VPWS service instance identifier and normalized VID are carried in the Ethernet Tag ID field of the Ethernet A-D per EVI route. For FXC, in the case of a 12-bitIDID, the VPWS service instance identifier is the same as the single-tag normalized VID and will be the same on both VPWS service endpoints. Similarly in the case of a 24-bit ID, the VPWS service instance identifier is the same as the double-tag normalized VID.</t> </section> <sectiontitle="Default Flexible Xconnect"anchor="fxc"> <name>Default Flexible Xconnect</name> <t>In this mode of operation, many ACs across several Ethernet Segments are multiplexed into a single EVPN VPWS service tunnel represented by a single VPWS service ID. This is the default mode of operation forFXCFXC, and the participating PEs do not need to signal the VLANs (normalized VIDs) in EVPN BGP.</t> <t>Regarding thedata-planedata plane aspects of this solution, the impositionProvider Edge (PE)PE performs VIDnormalizationnormalization, and the disposition PE carries out VID lookup and translation. Both imposition and disposition PE devicesMUST<bcp14>MUST</bcp14> be aware of the VLANs through configuration. There should ideally be a single point-to-point (P2P) EVPN VPWS service tunnel between a pair of PEs for a specific set ofAttachment Circuits (ACs).</t>ACs.</t> <t>As previously mentioned, because the EVPN VPWS service tunnel is employed to multiplex ACs across various Ethernet Segments (ESs) or physical interfaces, the EVPN label alone is not sufficient for accurate forwarding of the received packets over the MPLS/IP network to egress interfaces. Therefore, normalized VID lookup isREQUIRED<bcp14>REQUIRED</bcp14> in the disposition direction to forward packets to their proper egressend-points;endpoints; the EVPN label lookup identifies a VID-VRF, and a subsequent normalized VID lookup in that table identifies the egress interface.</t> <t>In this solution, for each PE, the single-homing ACs represented by their normalized VIDs are associated with a single VPWS service instance within a specific EVI. The generated EVPN route is an Ethernet A-D per EVI route with an ESI of 0,andthe Ethernet Tag field is set tothea VPWS service instance ID, and the MPLS label field is set to a dynamically generated EVPN service label representing the EVPN VPWS service tunnel. This route is sent with a Route Target (RT) that represents the EVI, which can beauto&nbhy;generatedauto-generated from the EVI according to<relref<xref target="RFC8365" section="5.1.2.1"/>. <!--[rfced] To clarify the numbered list, we have updated this sentence in Section 3.2. Please review and ensure that the intended meaning has not changed. Note that we have made a similar update to a sentence in Section 3.3. Original: Additionally, this route is sent with the EVPN Layer-2 Extended Community defined in<relrefSection 3.1 of [RFC8214] with two new flags (outlined in Section 4) that indicate: 1) this VPWS service tunnel is for the default Flexible Cross-Connect, and 2) the normalized VID type (single versus double). Current: Additionally, this route is sent with the EVPN Layer 2 Extended Community defined in Section 3.1 of [RFC8214] with two new flags (outlined in Section 4) that indicate: 1) this VPWS service tunnel for the default Flexible Cross-Connect and 2) the normalized VID type (single versus double). --> Additionally, this route is sent with the EVPN Layer 2 Extended Community defined in <xref target="RFC8214"section="3.1"/>section="3.1" sectionFormat="of"/> with two new flags (outlined in <xref target="bgp_extensions"/>) that indicate: 1) this VPWS service tunnelisfor the default FlexibleCross-Connect,Cross-Connect and 2) the normalized VID type (single versus double). The receiving PE uses these new flags for a consistency check andMAY<bcp14>MAY</bcp14> generate an alarm if it detects inconsistencies, but it will not disrupt the VPWS service.</t> <t>It should be noted that in this mode of operation, a singleEthernet A-DEthernet A-D per EVI route is transmitted upon the configuration of the firstAttachment Circuit (AC)AC with the normalized VID. As additional ACs are configured and associated with this EVPN VPWS service tunnel, the PE does not advertise any additional EVPN BGP routes and onlyassociateslocally associates these ACs with the pre-established VPWS service tunnel.</t> <sectiontitle="Multi-homing"anchor="fxc_mh"> <name>Multi-homing</name> <t>The default FXC mode can also be used for multi-homing. In this mode, a group of normalized VIDs representing ACs on a single Ethernet Segment, all destined to a single endpoint, are multiplexed into a single EVPN VPWS servicetunneltunnel, which is identified by a unique VPWS service ID. When employing the default FXC mode for multi-homing, rather than using a single EVPN VPWS servicetunneltunnel, there may be multiple service tunnels per pair of PEs. Specifically, there is one tunnel for each group of VIDs per pair of PEs, and there can be many such groups between a pair of PEs, resulting in numerous EVPN service tunnels.</t> </section> </section> <sectiontitle="VLAN-Signaled Flexible Xconnect"anchor="vlan_sig_fxc"> <name>VLAN-Signaled Flexible Xconnect</name> <!--[rfced] Please note that a slash (/) can mean "and", "or", or "and/or". We have updated it to "and" in the text below for clarity. Please review and let us know if any further updates are needed. Original: In this mode of operation, similar to the default FXC mode described in Section 3.2, many normalized VIDs representing ACs across several Ethernet Segments/interfaces are multiplexed into a single EVPN VPWS service tunnel. Current: In this mode of operation, similar to the default FXC mode described in Section 3.2, many normalized VIDs representing ACs across several Ethernet Segments and interfaces are multiplexed into a single EVPN VPWS service tunnel. --> <t>In this mode of operation, similar to the default FXC mode described in <xref target="fxc"/>, many normalized VIDs representing ACs across several EthernetSegments/interfacesSegments and interfaces are multiplexed into a single EVPN VPWS service tunnel. However, this single tunnel is represented by multiple VPWS service IDs (one per normalizedVID)VID), and these normalized VIDs are signaled using EVPN BGP.</t> <t>In this solution, on each PE, the multi-homing ACs represented by their normalized VIDs are configured with a single EVI. There is no need to configure a separate VPWS service instance ID in here, as it corresponds to the normalized VID. For each normalized VID on each Ethernet Segment, the PE generates an Ethernet A-D per EVI route where the ESI field represents the ES ID, the Ethernet Tag field is set to the normalized VID, and the MPLS label field is set to a dynamically generated EVPN label representing the P2P EVPN service tunnel. This label is the same for all ACs multiplexed into a single EVPN VPWS service tunnel. This route is sent witha Route Target (RT)an RT representing the EVI. As before, this RT can be auto-generated from the EVI persection <relref<xref target="RFC8365" section="5.1.2.1"/>. Additionally, this route includes the EVPNLayer-2Layer 2 Extended Community defined in<relref<xref target="RFC8214" section="3.1"/> with two new flags (outlined in <xref target="bgp_extensions"/>) that indicate: 1) this VPWS service tunnelisfor VLAN-signaled FlexibleCross-Connect,Cross-Connect and 2) the normalized VID type (single versus double). The receiving PE uses these new flags for a consistency check and may generate an alarm if it detects inconsistency, but it will not disrupt the VPWS service.</t> <t>It should be noted that in this mode of operation, the PE sends a single Ethernet A-D per EVI route for each AC that is configured. Each normalized VID that is configured per ES results in generation of an Ethernet A-D per EVI.</t> <t>This mode of operation enabled automatic cross-checking of normalized VIDs used for Ethernet Virtual Private Line (EVPL) services because these VIDs are signaled in EVPN BGP. For instance, if the same normalized VID is configured on three PE devices (instead of two) for the same EVI, then when a PE receives the second remote Ethernet A-D per EVI route, it generates an error message unless the two Ethernet A-D per EVI routes include the same ESI. Such cross-checking is not feasible in the default FXC mode because the normalized VIDs are not signaled.</t> <sectiontitle="Local Switching"anchor="local_switch"> <name>Local Switching</name> <t>When cross-connection occurs between two ACs belonging to two multi-homed Ethernet Segments on the same set of multi-homing PEs, the forwarding between the two ACs must be performed locally during normal operation (e.g., in absence of a local link failure). This means that traffic between the two ACsMUST<bcp14>MUST</bcp14> be locally switched within the PE.</t> <t>In terms of control plane processing, this means that when the receiving PE processes an Ethernet A-D per EVI route whose ESI is a local ESI, the PE does not modify its forwarding state based on the received route. This approach ensures that local switching takes precedence over forwarding via the MPLS/IP network. This method of prioritizing locally switched traffic aligns with the baseline EVPN principles described in <xref target="RFC7432"/>, where locally switched preference is specified forMAC/&wj;IPMAC/IP routes.</t> <t>In such scenarios, the Ethernet A-D per EVI route should be advertised with the MPLS label either associated with the destinationAttachment CircuitAC or with the destination Ethernet Segment in order to avoid any ambiguity in forwarding. In other words, the MPLS label cannot represent the same VID-VRF outlined in <xref target="vlan_sig_fxc"/>, as the same normalized VID can be reachable via two Ethernet Segments. In the case of using an MPLS label per destination AC, this approach can also be applied to VLAN-based VPWS or VLAN-bundle VPWS services as per <xref target="RFC8214"/>.</t> </section> </section> <sectiontitle="Service Instantiation"anchor="instantiation"> <name>Service Instantiation</name> <t>The V field defined in <xref target="bgp_extensions"/> isOPTIONAL.<bcp14>OPTIONAL</bcp14>. However, if transmitted, its value may indicate an error condition that could lead to operational issues. In such cases, merely notifying the operator of an error is insufficient; the VPWS service tunnel must not be established.</t> <t>If both endpoints of a VPWS tunnel are signaling a matchingNormalisedNormalized VID in the control plane, but one is operating in single-tag mode and the other in double-tag mode, then the signaling of the V-bit facilitates the detection and prevention of this tunnel's instantiation.</t> <t>If single VID normalization is signaled in the Ethernet Tag ID field (12bits)bits), yetdataplanethe data plane is operating based on double tags, the VID normalization applies only to the outer tag. Conversely, if double VID normalization is signaled in the Ethernet Tag ID field (24 bits), VID normalization applies to both the inner and outer tags.</t> </section> </section> <sectiontitle="BGP Extensions"anchor="bgp_extensions"> <name>BGP Extensions</name> <t>Thisdraftdocument uses the EVPNLayer-2Layer 2 attribute extended community as defined in <xref target="RFC8214"/> with two additional flags incorporated into this Extended Community (EC) as detailed below. This EC is sent with Ethernet A-D per EVI route per <xreftarget="solution"/>,target="solution"/> andSHOULD<bcp14>SHOULD</bcp14> be sent for both Single-Active and All-Active redundancymodes. <figure><artwork><![CDATA[modes.</t> <artwork><![CDATA[ +-------------------------------------------+ | Type (0x06) / Sub-type (0x04) (2 octets) | +-------------------------------------------+ | Control Flags (2 octets) | +-------------------------------------------+ | L2 MTU (2 octets) | +-------------------------------------------+ | Reserved (2 octets) | +-------------------------------------------+ 1 1 1 1 1 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | MBZ | V | M | |C|P|B| (MBZ = MUST Be Zero) +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+]]> </artwork></figure> </t>]]></artwork> <t>The following bits in the Control Flags are defined; the remaining bitsMUST<bcp14>MUST</bcp14> be set to zero when sending andMUST<bcp14>MUST</bcp14> be ignored when receiving thiscommunity. <figure><artwork><![CDATA[ Name Meaning --------------------------------------------------------------- B,P,C percommunity.</t> <table> <thead> <tr> <th>Name</th> <th>Meaning</th> </tr> </thead> <tbody> <tr> <td>B,P,C</td> <td>per definition in[RFC8214] M 00<xref target="RFC8214"/></td> </tr> <tr> <td rowspan="3">M</td> <td>00 mode of operation as defined in[RFC8214] 01<xref target="RFC8214"/></td> </tr> <tr> <td>01 VLAN-Signaled FXC10</td> </tr> <tr> <td>10 DefaultFXC V 00FXC</td> </tr> <tr> <td rowspan="3">V</td> <td>00 operating per[RFC8214] 01<xref target="RFC8214"/></td> </tr> <tr> <td>01 single-VIDnormalization 10normalization</td> </tr> <tr> <td>10 double-VIDnormalization ]]> </artwork></figure> </t>normalization</td> </tr> </tbody> </table> <t>The M and V fields areOPTIONAL.<bcp14>OPTIONAL</bcp14>. The M field is ignored at reception for forwarding purposes and is used for error notifications. </t> </section> <sectiontitle="Failure Scenarios"anchor="failure_scenarios"><t>Two<name>Failure Scenarios</name> <t>The two following exampleswill be used as an example toanalyze the failure scenarios.</t> <t>The first scenario is a default Flexible Xconnect withMulti-Homing solutiona multi-homing solution, and it is depicted in <xref target="dflt_fig"/>. In this case, VIDNormalizationnormalization isperformedperformed, and a single Ethernet A-D per EVI route is sent for the bundle of ACs on an ES. That is, PE1 will advertise two Ethernet A-D per EVI routes:theThe first one will identify the ACs on port p1'sESES, and the second one will identify the AC2 in port p2's ES. Similarly, PE2 will advertise two Ethernet A-D per EVIroutes.routes.</t> <figuretitle="Default Flexible Xconnect"anchor="dflt_fig"> <name>Default Flexible Xconnect</name> <artwork><![CDATA[ N.VID 1,2,3 +---------------------+ PE1 | | +---------+ IP/MPLS | +-----+ VID1 p1 | +-----+ | sv.T1 + | CE1 |-------------| FXC |======+ PE3 +-----+ | | /\ | | | | \ +----------+ +--| CE3 | +-----+\ +||---| | sv.T2 \ | | 1/ | | VID3\ / ||---| |=====+ \ | +-----+ | / +-----+ \ // \/ | +-----+ | \ +====| FXC |----+ \ / p2 +---------+ +======| | | 2 +-----+ /\ | | |----------| CE4 | / /\ +---------+ +=====| | | | | / / \p3 | +-----+ sv.T3 / +====| | | +-----+ VIDs1,2 / +----| FXC |=======+ / | | |---+ +-----+ / /\ | | | | / | +-----+ |\3 +-----+ | CE2 |-----||---| | | sv.T4 / | | \ | CE5 | | |-----||---| | |======+ +----------+ +---| | +--VIDs3,4 \/ | +-----+ | | +-----+ p4 +---------+ | PE2 | | N.VID 1,2,3 +-------------------+]]> </artwork></figure> </t>]]></artwork> </figure> <t>The second scenario, depicted in <xref target="vlan_sig_fig"/>, illustrates theVLAN&nbhy;signaledVLAN-signaled FXC mode withMulti-Homing.multi-homing. In thisexample: <list style="symbols"> <t>CE1example:</t> <ul spacing="normal"> <li>CE1 is connected to PE1 and PE2 via (port,VID)=(p1,1) and (p3,3), respectively. CE1's VIDs are normalized tovalue 1value 1 on both PEs, and CE1 is cross-connected to CE3'sVID 1VID 1 at the remoteend.</t>end.</li> <li> <t>CE2 is connected to PE1 and PE2 via ports p2 andp4p4, respectively:<list style="symbols"> <t>The</t> <ul spacing="normal"> <li>The combinations (p2,1) and (p4,3) identify the ACs used to cross-connect CE2 to CE4's VID2,2 and are normalized tovalue 2.</t> <t>Thevalue 2.</li> <li>The combinations (p2,2) and (p4,4) identify the ACs used to cross-connect CE2 to CE5'sVID 3,VID 3 and are normalized tovalue 3.</t> </list> </t> </list>value 3.</li> </ul> </li> </ul> <t> In this scenario, PE1 and PE2 advertise an Ethernet A-D per EVI route for each normalized VID (values 1,22, and 3). <!--[rfced] May we remove "service" after "FXC" in the following sentence? Additionally, please note that we have numbered the items listed to improve readability. Original: However, only two VPWS Service Tunnels are required: VPWS Service Tunnel 1 (sv.T1) between PE1's FXC service and PE3's FXC, and VPWS Service Tunnel 2 (sv.T2) between PE2's FXC and PE3's FXC. Perhaps: However, only two VPWS Service Tunnels are required: 1) VPWS Service Tunnel 1 (sv.T1) between PE1's FXC and PE3's FXC and 2) VPWS Service Tunnel 2 (sv.T2) between PE2's FXC and PE3's FXC. --> However, only two VPWS service tunnels are required: 1) VPWS Service Tunnel 1 (sv.T1) between PE1's FXC service and PE3's FXC and 2) VPWS Service Tunnel 2 (sv.T2) between PE2's FXC and PE3's FXC.</t> <figuretitle="VLAN-Signaled Flexible Xconnect"anchor="vlan_sig_fig"> <name>VLAN-Signaled Flexible Xconnect</name> <artwork><![CDATA[ N.VID 1,2,3 +---------------------+ PE1 | | +---------+ IP/MPLS | +-----+ VID1 p1 | +-----+ | + | CE1 |------------| FXC | | sv.T1 PE3 +-----+ | | /\ | | |=======+ +----------+ +--| CE3 | +-----+\ +||---| | | \ | | 1/ | | VID3\ / ||---| | | \ | +-----+ | / +-----+ \ / /\/ | +-----+ | +=====| FXC |----+ \ / p2 +---------+ | | | | 2 +-----+ /\ | | |----------| CE4 | / /\ +---------+ +======| | | | | / / \p3 | +-----+ | / | | | | +-----+ VIDs1,2 / +----| FXC | / | | |---+ +-----+ / /\ | | |======+ | +-----+ |\3 +-----+ | CE2 |-----||-----| | | sv.T2 | | \ | CE5 | | |-----||-----| | | +----------+ +---| | +-----+ \/ | +-----+ | | +-----+ VIDs3,4 p4 +---------+ | PE2 | | N.VID 1,2,3 +------------------+]]> </artwork></figure> </t>]]></artwork> </figure> <sectiontitle="EVPNanchor="evpws_svc_failure"> <name>EVPN VPWS ServiceFailure" anchor="evpws_svc_failure"> <t>TheFailure</name> <!--[rfced] May we update the following acronyms and their expansions to better reflect the text in RFC 5885? Original: The failure detection of an EVPN VPWS service can be performed via OAM mechanisms such as VCCV-BFD (Bidirectional Forwarding Detection for the Pseudowire Virtual Circuit Connectivity Verification,<xref target="RFC5885"/>)[RFC5885]) and upon such failure detection, the switch over procedure to the backup S-PE is the same as the one described above. Perhaps: The failure detection of an EVPN VPWS service can be performed via OAM mechanisms such as Bidirectional Forwarding Detection (BFD) for the pseudowire Virtual Circuit Connectivity Verification (VCCV) [RFC5885], and upon such failure detection, the switch over procedure to the backup S-PE is the same as the one described above. --> <t>The failure detection of an EVPN VPWS service can be performed via OAM mechanisms such as VCCV-BFD (Bidirectional Forwarding Detection for the Pseudowire Virtual Circuit Connectivity Verification) <xref target="RFC5885"/>, and upon such failure detection, the switchover procedure to the backup Switching Provider Edge (S-PE) is the same as the one described above.</t> </section> <sectiontitle="Attachment Circuit Failure"anchor="ac_failure"> <name>Attachment Circuit Failure</name> <t>In the event of an AC failure, the VLAN-Signaled and default FXC modes exhibit distinctbehaviors: <list style="symbols"> <t>Defaultbehaviors:</t> <ul spacing="normal"> <li>Default FXC (<xref target="dflt_fig"/>):inIn the default mode, a VLAN or AC failure is not signaled. Consequently, in case of an ACfailurefailure, such as VID1 on CE2, there is nothing to prevent PE3 from directing traffic from CE4 to PE1, leading to a potential black hole. Application layerOperations, Administration, and Maintenance (OAM)OAM may be utilized if per-VLAN fault propagation is necessary in thisscenario.</t> <t>VLAN-Signaledscenario.</li> <li>VLAN-Signaled FXC (<xref target="vlan_sig_fig"/>):inIn the case of a VLAN or ACfailurefailure, such as VID1 on CE2, this triggers the withdrawal of the Ethernet A-D per EVI route for the corresponding Normalized VID, specificallyEthernet-Tag 2.Ethernet-Tag 2. Upon receiving the route withdrawal, PE3 will remove PE1 from its outgoing path list for traffic originating fromCE4.</t> </list> </t>CE4.</li> </ul> </section> <sectiontitle="PE Port Failure"anchor="pe_port_failure"> <name>PE Port Failure</name> <t>In the event of a PE port failure, the failure will be signaled, and the other PE will assume forwarding in bothscenarios: <list style="symbols"> <t>Defaultscenarios:</t> <ul spacing="normal"> <li>Default FXC (<xref target="dflt_fig"/>): In the case of a port failure, such as p2, the route forService Tunnel 2Service Tunnel 2 (sv.T2) will be withdrawn. Upon receiving the fault notification, PE3 will remove PE1 from its path list for traffic originating from CE4 andCE5.</t> <t>VLAN-SignaledCE5.</li> <li>VLAN-Signaled FXC (<xref target="vlan_sig_fig"/>): A port failure, such as p2, triggers the withdrawal of the Ethernet A-D per EVI routes for Normalized VIDs 2 and 3, along with the withdrawal of the Ethernet A-D per ES route for p2's ES. Upon receiving the fault notification, PE3 will remove PE1 from its path list for the traffic originating from CE4 andCE5.</t> </list> </t>CE5.</li> </ul> </section> <sectiontitle="PE Node Failure"anchor="pe_node_failure"> <name>PE Node Failure</name> <t>In the case of PE node failure, the operation is similar to the steps described above, albeit that EVPN route withdrawals are performed by theRoute Reflectorroute reflector instead of the PE.</t> </section> </section><section title="Security Considerations"><section> <name>Security Considerations</name> <t>Since this document describes a muxing capabilitywhichthat leverages EVPN-VPWS signaling, no additional functionality beyond the muxing service isaddedadded, and thus no additional security considerations are needed beyond what is already specified in <xref target="RFC8214"/>.</t> </section> <sectionanchor="IANA" title="IANA Considerations">anchor="IANA"> <name>IANA Considerations</name> <t>This documentrequests allocation ofhas allocated bits 8-11 in the "EVPN Layer 2 Attributes Control Flags" registry with names M and V:<figure><artwork><![CDATA[ M Signaling</t> <dl spacing="compact" newline="false"> <dt>M</dt><dd>Signaling mode of operation (bits10-11) V VLAN-ID10-11)</dd> <dt>V</dt><dd>VLAN-ID normalization (bits8-9) ]]></artwork></figure> </t>8-9)</dd> </dl> </section> </middle><!-- *****BACK MATTER ***** --><back> <displayreference target="I-D.ietf-rtgwg-bgp-pic" to="BGP-PIC"/> <references> <name>References</name> <references> <name>Normative References</name> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7432.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8214.xml"/> </references> <references> <name>Informative References</name> <!--References split into informative and normative[I-D.ietf-rtgwg-bgp-pic] IESG State: Expired as of 01/09/25. Note to PE: Recommend changing the cite tag for this I-D to something more symbolic: Current: [I-D.ietf-rtgwg-bgp-pic] Perhaps: [BGP-PIC] --><references title="Normative References"> <?rfc include="reference.RFC.2119.xml"?> <?rfc include="reference.RFC.8174.xml"?> <?rfc include="reference.RFC.7432.xml"?> <?rfc include="reference.RFC.8214.xml"?><reference anchor="I-D.ietf-rtgwg-bgp-pic" target="https://datatracker.ietf.org/doc/html/draft-ietf-rtgwg-bgp-pic-21"> <front> <title>BGP Prefix Independent Convergence</title> <author fullname="Ahmed Bashandy" initials="A." surname="Bashandy" role="editor"> <organization>Cisco Systems</organization> </author> <author fullname="Clarence Filsfils" initials="C." surname="Filsfils"> <organization>Cisco Systems</organization> </author> <author fullname="Prodosh Mohapatra" initials="P." surname="Mohapatra"> <organization>Sproute Networks</organization> </author> <date day="7" month="July" year="2024"/> </front> <seriesInfo name="Internet-Draft" value="draft-ietf-rtgwg-bgp-pic-21"/> </reference> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8365.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5659.xml"/> <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.5885.xml"/> </references><references title="Informative References"> <?rfc include="reference.I-D.draft-ietf-rtgwg-bgp-pic-21.xml"?> <?rfc include="reference.RFC.8365.xml"?> <?rfc include="reference.RFC.5659.xml"?> <?rfc include="reference.RFC.5885.xml"?></references><!-- <section title="Acknowledgments"> </section> --><sectiontitle="Contributors">numbered="false"> <name>Contributors</name> <t>In addition to the authors listed on the front page, the following co-authors have also contributed substantially to thisdocument: </t> <t>Wen Lin<br/>Juniper Networks</t> <t>EMail: wlin@juniper.net</t> <t>Lucdocument:</t> <contact fullname="Wen Lin"> <organization>Juniper Networks</organization> <address> <email>wlin@juniper.net</email> </address> </contact> <contact fullname="Luc AndreBurdet<br/>Cisco</t> <t>EMail: lburdet@cisco.com</t>Burdet"> <organization>Cisco</organization> <address> <email>lburdet@cisco.com</email> </address> </contact> </section> </back> <!-- [rfced] Terminology A) To match usage in RFC 8214, may we update the following terms to the format on the right? single-active > Single-Active all-active > All-Active EVPN VPWS > EVPN-VPWS Ethernet A-D per EVI route > Ethernet A-D per-EVI route Ethernet A-D per ES route > Ethernet A-D per-ES route VLAN-bundle > VLAN bundle B) Throughout the text, the following terminology appears to be used inconsistently. May we update them to the format on the right? Normalized VID > normalized VID VLAN-signaled flexible cross-connect > VLAN-signaled FXC VLAN-signaled Flexible Cross-Connect > VLAN-signaled FXC C) Since RFC 8214 uses "EVPN Layer 2 Attributes Extended Community", should the following terms be update to match? We ask because of the possible addition of "Attributes". EVPN Layer 2 Extended Community (Sections 3.2 and 3.3) EVPN Layer 2 attribute extended community (Section 4) --> <!--[rfced] Abbreviations A) FYI - We have added expansions for the following abbreviations per Section 3.6 of RFC 7322 ("RFC Style Guide"). Please review each expansion in the document carefully to ensure correctness. Autonomous System (AS) Switching Provider Edge (S-PE) B) Both the expansion and the acronym for Ethernet Segment (ES) are used throughout the document. Would you like to update to using the expansion upon first usage and the acronym for the rest of the document for consistency? C) We note that "FXC" appears to be expanded in different ways throughout the document. May we update all these instances to be "Flexible Cross-Connect" for consistency? Flexible Xconnect Flexible Cross Connect Flexible Cross-Connect D) We note that "VCCV" is expanded in two different ways in this document. Please review and let us know which version should be updated for consistency. Virtual Circuit Connectivity Verification Virtual Circuit Connection Verification --> <!-- [rfced] Please review the "Inclusive Language" portion of the online Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language> and let us know if any changes are needed. Updates of this nature typically result in more precise language, which is helpful for readers. For example, please consider whether the following should be updated: black hole block-holed --> </rfc>