4.1. EVPN DF Election for vES

The service carving procedures for vESes are almost the same as the procedures outlined in Section 8.5 of [RFC7432] and in [RFC8584], except that ES is replaced with vES.¶

For the sake of clarity and completeness, the default DF election procedure of [RFC7432] is repeated below with the necessary changes:¶

1. When a PE discovers the vESI or is configured with the vESI associated with its attached vES, it advertises an Ethernet Segment route with the associated ES-Import extended community attribute.¶

2. The PE then starts a timer (default value = 3 seconds) to allow the reception of Ethernet Segment routes from other PE nodes connected to the same vES. This timer value MUST be the same across all PEs connected to the same vES.¶

3. When the timer expires, each PE builds an ordered list of the IP addresses of all the PE nodes connected to the vES (including itself), in increasing numeric value. Each IP address in this list is extracted from the "Originator Router's IP address" field of the advertised Ethernet Segment route. Every PE is then given an ordinal indicating its position in the ordered list, starting with 0 as the ordinal for the PE with the numerically lowest IP address. The ordinals are used to determine which PE node will be the DF for a given EVPN instance on the vES using the following rule: Assuming a redundancy group of N PE nodes, the PE with ordinal i is the DF for an EVPN instance with an associated Ethernet Tag value of V when (V mod N) = i. It should be noted that using the "Originator Router's IP address" field in the Ethernet Segment route to get the PE IP address needed for the ordered list allows a CE to be multi-homed across different ASes, if such need ever arises.¶

4. The PE that is elected as a DF for a given EVPN instance will unblock traffic for that EVPN instance. Note that the DF PE unblocks all traffic in both ingress and egress directions for Single-Active vESes and unblocks multi-destination in the egress direction for All-Active multi-homed vESes. All non-DF PEs block all traffic in both ingress and egress directions for Single-Active vESes and block multi-destination traffic in the egress direction for All-Active vESes.¶

In case of an EVC failure, the affected PE withdraws its corresponding Ethernet Segment route if there are no more EVCs associated to the vES in the PE. This will re-trigger the DF election procedure on all the PEs in the redundancy group. For PE node failure, or upon PE commissioning or decommissioning, the PEs re-trigger the DF election procedure across all affected vESes. In case of a Single-Active, when a service moves from one PE in the redundancy group to another PE because of DF re-election, the PE (which ends up being the elected DF for the service) MUST trigger a MAC address flush notification towards the associated vES if the multi-homing device is a bridge or the multi-homing network is an Ethernet bridged network.¶

For LSP-based and PW-based vES, the non-DF PE SHOULD signal PW-status 'standby' to the Aggregation PE (e.g., AG1 and AG2 in Figure 2), and a new DF PE MAY send a Label Distribution Protocol (LDP) MAC withdraw message as a MAC address flush notification. It should be noted that the PW-status is signaled for the scenarios where there is a one-to-one mapping between EVI (EVPN instance) and the PW.¶

4.2. Grouping and Route Coloring for vES

Physical ports (e.g., ENNI) that aggregate many EVCs are 'colored' to enable the grouping schemes described below.¶

By default, the MAC address of the corresponding port (e.g., ENNI) is used to represent the 'color' of the port, and the EVPN Router's MAC Extended Community defined in [RFC9135] is used to signal this color.¶

The difference between coloring mechanisms for EVPN and PBB-EVPN is that the extended community is advertised with the Ethernet A-D per ES route for EVPN, whereas the extended community is advertised with the B-MAC route for PBB-EVPN.¶

The subsequent sections detailing Grouping of Ethernet A-D per ES and Grouping of B-MAC addresses will be essential for addressing port failure handling, as discussed in Sections 5.3, 5.4, and 5.5.¶

5.1. EVC Failure Handling for Single-Active vES in EVPN

In [RFC7432], when a DF PE connected to a Single-Active multi-homed Ethernet Segment loses connectivity to the segment, due to link or port failure, it signals the remote PEs to invalidate all MAC addresses associated with that Ethernet Segment. This is done by means of a mass-withdraw message, by withdrawing the Ethernet A-D per ES route. It should be noted that for dual-homing use cases where there is only a single backup path, MAC invalidating can be avoided by the remote PEs as they can update their next hop associated with the affected MAC entries to the backup path per the procedure described in Section 8.2 of [RFC7432].¶

In case of an EVC failure that impacts a single vES, this same EVPN procedure is used. In this case, the mass withdraw is conveyed by withdrawing the Ethernet A-D per vES route carrying the vESI representing the failed EVC. Upon receiving this message, the remote PEs perform the same procedures outlined in Section 8.2 of [RFC7432].¶

5.2. EVC Failure Handling for a Single-Active vES in PBB-EVPN

In [RFC7432], when a PE connected to a Single-Active Ethernet Segment loses connectivity to the segment, due to link or port failure, it signals the remote PE to flush all C-MAC addresses associated with that Ethernet Segment. This is done by updating the advertised B-MAC route's MAC Mobility Extended Community.¶

In case of an EVC failure that impacts a single vES, if the above PBB-EVPN procedure is used, it results in excessive C-MAC flushing because a single physical port can support a large number of EVCs (and their associated vESes); therefore, updating the advertised B-MAC corresponding to the physical port, with MAC Mobility Extended Community, will result in flushing C-MAC addresses not just for the impacted EVC but for all other EVCs on that port.¶

To reduce the scope of C-MAC flushing to only the impacted service instances (the service instance(s) impacted by the EVC failure), the PBB-EVPN C-MAC flushing needs to be adapted on a per-service-instance basis (i.e., per I-SID). [RFC9541] introduces a B-MAC/I-SID route where the existing PBB-EVPN B-MAC route is modified to carry an I-SID in the "Ethernet Tag ID" field instead of NULL value. To the receiving PE, this field indicates flushing all C-MAC addresses associated with that I-SID for that B-MAC. This C-MAC flushing mechanism per I-SID SHOULD be used in case of an EVC failure impacting a vES. Since an EVC typically maps to a single broadcast domain and thus a single service instance, the affected PE only needs to advertise a single B-MAC/I-SID route. However, if the failed EVC carries multiple VLANs each with its own broadcast domain, then the affected PE needs to advertise multiple B-MAC/I-SID routes -- one for each VLAN (broadcast domain) -- i.e., one for each I-SID. Each B‑MAC/I-SID route basically instructs the remote PEs to perform flushing for C-MACs corresponding to the advertised B-MAC only for the advertised I-SID.¶

The C-MAC flushing based on a B-MAC/I-SID route works fine when there are only a few VLANs (e.g., I-SIDs) per EVC. However, if the number of I-SIDs associated with a failed EVC is large, then it is RECOMMENDED to assign a B-MAC per vES, and upon EVC failure, the affected PE simply withdraws this B-MAC message to other PEs.¶

5.3. Port Failure Handling for Single-Active vESes in EVPN

When many EVCs are aggregated via a single physical port on a PE, where each EVC corresponds to a vES, then the port failure impacts all the associated EVCs and their corresponding vESes. If the number of EVCs corresponding to the Single-Active vESes for that physical port is in the thousands, then thousands of service instances are impacted. Therefore, the propagation of failure in BGP needs to address all these impacted service instances. In order to achieve this, the following extensions are added to the baseline EVPN mechanism:¶

1. The PE MAY color each Ethernet A-D per ES route for a given vES, as described in Section 4.2.1. The PE SHOULD use the MAC physical port by default. The receiving PEs take note of this color and create a list of vESes for this color.¶

2. The PE MAY advertise a special Grouping Ethernet A-D per ES route for that color, which represents all the vESes associated with the port.¶

3. Upon a port failure (e.g., an ENNI failure), the PE MAY send a mass-withdraw message by withdrawing the Grouping Ethernet A-D per ES route.¶

4. When this message is received, the remote PE MAY detect the special vES mass-withdraw message by identifying the Grouping Ethernet A-D per ES route. The remote PEs MAY then access the list created in (1) of the vESes for the specified color and locally initiate MAC address invalidating procedures for each of the vESes in the list.¶

In scenarios where a logical ENNI is used, the above procedure equally applies. The logical ENNI is represented by a Grouping Ethernet A-D per ES where the Type 3 ESI and the 6 bytes used in the ENNI's ESI MAC address field are used as a color for the vESes as described above and in Section 4.2.1.¶

5.4. Port Failure Handling for Single-Active vESes in PBB-EVPN

When many EVCs are aggregated via a single physical port on a PE, where each EVC corresponds to a vES, then the port failure impacts all the associated EVCs and their corresponding vESes. If the number of EVCs corresponding to the Single-Active vESes for that physical port is in the thousands, then thousands of service instances (I-SIDs) are impacted. In such failure scenarios, the following two MAC flushing mechanisms per [RFC7623] can be performed.¶

1. If the MAC address of the physical port is used for PBB encapsulation as B-MAC SA, then upon the port failure, the PE MUST use the EVPN MAC route withdrawal message to signal the flush.¶

2. If the PE's shared MAC address is used for PBB encapsulation as B-MAC SA, then upon the port failure, the PE MUST re-advertise this MAC route with the MAC Mobility Extended Community to signal the flush.¶

The first method is recommended because it reduces the scope of flushing the most.¶

As noted above, the advertisement of the extended community along with the B-MAC route for coloring purposes is optional and only recommended when there are many vESes per physical port and each vES is associated with a very large number of service instances (i.e., a large number of I-SIDs).¶

If there are large numbers of service instances (i.e., I-SIDs) associated with each EVC, and if there is a B-MAC assigned per vES as recommended in the above section, then in order to handle port failure efficiently, the following extensions are added to the baseline PBB-EVPN mechanism:¶

1. Each vES MAY be colored with a MAC address representing the physical port like the coloring mechanism for EVPN. In other words, each B-MAC representing a vES is advertised with the 'color' of the physical port per Section 4.2.2. The receiving PEs take note of this color being advertised along with the B-MAC route, and for each such color, they create a list of vESes associated with this color.¶

2. The PE MAY advertise a special Grouping B-MAC route for that color (consisting of a port MAC address by default), which represents all the vESes associated with the port.¶

3. Upon a port failure (e.g., ENNI failure), the PE MAY send a mass-withdraw message by withdrawing the Grouping B-MAC route.¶

4. When this message is received, the remote PE MAY detect the special vES mass-withdraw message by identifying the Grouping B-MAC route. The remote PEs MAY then access the list created in (1) above for the specified color and flush all C-MACs associated with the failed physical port.¶

5.5. Fast Convergence in EVPN and PBB-EVPN

As described above, when many EVCs are aggregated via a physical port on a PE, and where each EVC corresponds to a vES, the port failure impacts all the associated EVCs and their corresponding vESes. Two actions must be taken as the result of such a port failure:¶

• For EVPN, initiate the mass-withdraw procedure for all vESes associated with the failed port to invalidate MACs and for PBB-EVPN to flush all C-MACs associated with the failed port across all vESes and the impacted I-SIDs¶

• Use DF election for all impacted vESes associated with the failed port¶

Section 5.3 already describes how to perform a mass withdraw for all affected vESes and invalidate MACs using a single BGP withdrawal of the Grouping Ethernet A-D per ES route. Section 5.4 describes how to only flush C-MAC addresses associated with the failed physical port (e.g., optimum C-MAC flushing) as well as, optionally, the withdrawal of a Grouping B-MAC route.¶

This section describes how to perform DF election in the most optimal way, e.g., by triggering DF election for all impacted vESes (which can be very large) among the participating PEs via a single BGP message as opposed to sending a large number of BGP messages (one per vES). This section assumes that the MAC flushing mechanism described in Section 5.4 and route coloring are used.¶

                  +-----+
       +----+     |     |       +---+
       | CE1|AC1--0=====0--ENNI1|   |  +-------+
       |    |AC2--0     |       |PE1|--|       |
       +----+     |\  ==0--ENNI2|   |  |       |
                  | \/  |       +---+  |       |
                  | /\  |              |IP/MPLS|
       +----+     |/  \ |       +---+  |Network|   +---+  +---+
       | CE2|AC4--0    =0--ENNI3|   |  |       |---|PE4|--|CE4|
       |    |AC4--0=====0--ENNI3|PE2|--|       |   +---+  +---+
       +----+     | ====0--ENNI3|   |  |       |
                  |/    |       +---+  |       |
                  0     |              |       |
       +----+    /|     |       +---+  |       |
       | CE3|AC5- |     |       |PE3|--|       |
       |    |AC6--0=====0--ENNI4|   |  +-------+
       +----+     |     |       +---+
                  +-----+

As discussed in Section 4.2, it is highly desirable to have a mass-withdraw mechanism similar to the one in [RFC7432]. Although such an optimization is desirable, it is OPTIONAL. If the optimization is implemented, the following procedures are used:¶

1. When a vES is configured, the PE advertises the Ethernet Segment route for this vES with a color that corresponds to the associated physical port.¶

2. All receiving PEs within the redundancy group record this color and compile a list of vESes associated with it.¶

3. Additionally, the PE advertises a Grouping Ethernet A-D per ES for EVPN, and a Grouping B-MAC for PBB-EVPN, which corresponds to the color and vES grouping.¶

4. In the event of a port failure, such as an ENNI failure, the PE withdraws the previously advertised Grouping Ethernet A-D per ES or Grouping B-MAC associated with the failed port. The PE should prioritize sending these Grouping route withdrawal messages over the withdrawal of individual vES routes affected by the failure. For instance, as depicted in Figure 4, when the physical port associated with ENNI3 fails on PE2, it withdraws the previously advertised Grouping Ethernet A-D per ES route. Upon receiving this withdrawal message, other multi-homing PEs (such as PE1 and PE3) recognize that the vESes associated with CE1 and CE3 are impacted, based on the associated color, and thus initiate the DF election procedure for these vESes. Furthermore, upon receiving this withdrawal message, remote PEs (such as PE4) initiate the failover procedure for the vESes associated with CE1 and CE3 and switch to the other PE for each vES redundancy group.¶

5. On reception of Grouping Ethernet A-D per ES or Grouping B-MAC route withdrawal, other PEs in the redundancy group initiate DF election procedures across all their affected vESes.¶

6. The PE with the physical port failure (ENNI failure) sends a vES route withdrawal for every impacted vES. Upon receiving these messages, the other PEs clear up their BGP tables. It should be noted that the vES route withdrawal messages are not used for executing DF election procedures by the receiving PEs when Grouping Ethernet A-D per ES or Grouping B-MAC withdrawal has been previously received.¶