FabricPath basics

Ever thought we will have alternate to STP? If not, then you must realize that now we do have  an option,The Fabric path.As of now, it works on Cisco Nexus series of switches.

So what is fabric Path? Here i am going to explain the basics level of Fabric Path in one shot. I am sure, you would not even need to read it twice to understand the fabric Path. Lets get started...

So what is the need of FabricPath?

As, i said its kind of alternate to STP but its a lot more than that. As we all know that STP blocks the redundant links in a network. We can somehow use those redundant links using vPC, VSS, port-channel etc but still there are restrictions
-like you cant have port-channel between links going to multiple hardware without vpc/vss.
-vpc/vss also supports upto 2 hardware in peer.
-what if i want to create random topologies or just kind of plug n play topology without any hassle of STP considerations?
-what if i want to use equal cost multipathing over layer 2 links like we do in routing?

Thats where Fabric path comes in and resolve all such issues...You will see a lot more benefits when you will go through its implementation...I will start with config first and then will explain the technology and trust me, you will enjoy!!


So, how do we configure two classical ethernet switch together...

SWitch(config)#interface ethernet1/1
SWitch(config)#switchport mode trunk

So, cisco kept FabricPath configuration also very similar, You just have to just replace mode "Trunk" with mode "fabricpath"
Nexus(config)# int e1/1
N7K-1(config-if)# switchport mode fabricpath


FabricPath runs over Nexus series of switches only, so what if a Nexus switch is connected to a classical Ethernet switch.hmm..so, to make it easy, few terminology were introduced to identify whether its a FabricPath Edge(which connects to classical ethernet) switch or end host, or its a Spine switch(which connects to another FP switch). The port which connects FabricPath to FabricPath switch is called FabricPath Core port.

when any vlan traffic has to be forwarded through FabricPath links, it has be specified as fabricPath vlan on all FabricPath enabled switches..

Nexus(config)#vlan 50
Nexus(config-vlan)# mode fabricpath

IMportant to note that Only FabricPath-mode VLANs are carried over FabricPath interfaces.

End host connects to a switch in normal access mode and when that data is transferred from one switch to another, that goes from edge to Fabric Path core ports.
Now, take a minute to understand this basic diagram:

Now, to understand how a frame is forwarded from one source to destination,think of normal classical ethernet frame flow. Lets say PC1 is connected on switchA and PC2 is connected on Switch D, so in a classical ethernet environment, it will just check from which port destination MAC address is being learnt and frame will be forwarded to that port and this process will continue upto destination switch. But in FabricPath, when a switch receives a Ethernet Frame on a VLAN which is fabricpath vlan, the switch will not check from which port the destination MAC is reachable, instead it will check on which Edge switch destination is connected (we will see how). Yes, so switch A will come to know that it has to forward frame to SWITCH D and it will try to find a best path to route that packet to SWITCH D. Can you believe it. A switch will forward frame based on best path to a destination switch. So, if we talk about a normal routing protocol, you need a IP address of the destination router to route the packet to that device. Similarly, in FabricPath, a new terminology was introduced which is called "Switch ID". Every switch will have a unique switch ID and source switch will forward frame to destination SwitchID based on best path available. The intermediate switch do not need to know the destination MAC address of the PC, they will just forward that frame towards destination Edge SWITCH ID.

So, now the next question comes into your mind that how it will calculate best path at layer 2? dont we need some kind of routing to let everyone know that which switch ID exists where? Yes, valid questions..

To solve this, FabricPath uses link-state "IS-IS" protocol in background to build its control plane. Using IS-IS, it builds its SWitch ID table, and from that table, it can identify the best path to reach to destination switch ID and above that, if there are multiple equal cost path, it can load balance also. I hope this makes sense now.


Now, lets take a look how MAC Learning works in FabricPath environment:

In classical ethernet, when a frame is received on a switch port, switch may or may not know the Destination but it will learn the source MAC address unconditionally,right? but in case of FabricPath, switch will not learn the MAC address until Destination is already known. that means if a destination MAC address exists in the mac table, then only source will be learnt. The idea is to keep the entries in MAC table for only those devices which are actively communicating. I mean, in a traditional ethernet, MAC entries are learnt whether there is any communication going on or not,  even a end host graciously announcing its ARP or due to any other reason and every switch who is part of that vlan will have those mac entries. This will keep on filling mac table. but in case of fabricPath, switch will not learn MAC address until destination is known. This is called "Conversational MAC Learning". (one more new term  ;-) )

Important to note that this difference is not Valid for Entire switch, but its valid for whether the fame is received on FabricPath Core PORT.
If a frame is Ingress from Edge port, the traditional source mac address is learnt but if the frame is Ingress on FabricPath Core Port (mean coming from another FP switch), then the switch will not learn source MAC address until the destination is already in mac table (conversational learning)

Example: (diagram above)

PC1 wants to send a frame to PC2 (both are in same vlan).First time communication, so it will send ARP request. As frame is ingress on Edge port, switch will learn the source mac address and its port(traditional MAC learning). Destination is broadcast (ffff.ffff.ffff)
SwitchA has multiple paths to all switches. As this is a broadcast frame, switch has to make sure it reaches all switches without creating a loop. So, to achieve the same, FabricPath introduces a concept of multi-destination tree. Means, switch has multiple path to all switches and if it has to chose one path, then create a loop free tree (Yes, at this point, it sounds like spanning tree but confusion will be cleared when you read it to advance level).
so,as of now, lets say Switch A could create below trees)

Now lets say switch A, selects one of the tree (the green one) and forwards the packet out on FabricPath port. Obviously it has to encapsulate the frame into FabricPath header. (Read here for the complete encapsulation method).

SwtichB will receive the frame, It is also part of green tree, so it will forward the frame on all FP port except, it was received on.

SwitchD receive the frame. It knows, it is an edge switch because it has end machine connected, means it has access-port which is part of fabricPath vlan. So, it decapsulates the packet and see that the frame is broadcase for vlan10. so, it will forward the frame in all ports in vlan10 but IT WILL NOT LEARN SOURCE MAC ADDRESS, because the frame was ingress on FP port and destination (ffff.ffff.ffff) is not known, so source will not be learnt.

Now, the ARP response from PC2 will be unicast, SWitch D will learn the source MAC because frame is ingress on CE port, frame will be forwarded on one of the tree, now when Switch A receives the frame, it knows the Destination MAC which it learnt from ARP request when PC1 sent the first packet. So, for a ARP reply packet from Pc2 to PC1, now destination is known to SWitchA, so it will learn the source MAC address but what it will not create an entry that source mac (PC2 mac) is being learnt via port E1/1, but instead it will learn tht PC1 MAC adress is learnt via SwitchD. (That is conversationl MAC learning). Next frame destined for PC2, will be forwarded by SwitchA based on the entry that it created that says "PC2 is reachable via SwitchD". so all intermediate switch will forward that frame to switch D, instead of learning mac address for every single host.


I hope the basic of FabricPath is clear from this post. The unanswered question like below will explain in next simplified post.
1)How did SwitchA select which tree to use and how frame is forwarded within a tree?
2)what if there are multiple path and want to do load balancing in fabricPath?
2)How FabricPath encapsulation works?
3)Configuration and verification of FabricPath.
4)Switch ID allocation?
5)how switch ID table is created and frame is forwarded at each switch.
6)Post in comments if you have any question so far.

Thank you!!