Autonomous Systems and AS Numbers

ipSpace.net » Documents » Using BGP in a Data Center Leaf-and-Spine Fabric » Autonomous Systems and AS Numbers

BGP autonomous system design and AS numbering within a data center fabric depend on two major parameters:

Do you plan to use BGP as the only routing protocol within the fabric, or do you have to introduce BGP to support EVPN or MPLS/VPN;
How complex is the EBGP versus IBGP configuration on equipment you plan to use in your data center.

IBGP or EBGP?

If you decided to use OSPF or IS-IS as the IGP as the fabric routing protocol, and need BGP to carry endpoint reachability information (EVPN addresses/routes or MPLS/VPN routes), use IBGP to simplify the network design and device configuration.

In larger fabrics that use BGP as the internal routing protocol for scalability reasons use EBGP. IBGP was designed to run in combination with an IGP, and while it’s possible to build an IBGP-only fabric, you’d be tweaking numerous BGP parameters to make it a round peg fit into a star-shaped hole.

BGP on Spine Switches

If you use OSPF or IS-IS as the fabric routing protocol, and use IBGP between leaf switches to transport EVPN or MPLS/VPN prefixes, you don’t need to run BGP on the spine switches unless you use the spine switches as IBGP route reflectors.

If you decide to carry global IP prefixes in BGP then it’s easier to implement BGP everywhere design (with BGP running on spine switches) than using BGP-free MPLS core design. Also, if you’re using BGP to carry global IP prefixes for scalability reasons, then a design using BGP as the sole fabric routing protocol might be simpler than a combination of BGP and IGP.

When running EBGP as the sole routing protocol in a data center fabric use the same AS number on all spine switches to prevent path hunting – prefixes advertised between spine switches through leaf switches.

To learn more about path hunting, watch the Using BGP in Leaf-and-Spine Fabrics videos from the Leaf-and-Spine Fabrics webinar or Designing and Building Data Center Fabrics online course

You could also use traditional BGP policy tools like AS-path filters to prevent leaf switches from becoming transit switches, but it’s much easier to solve this problem with good AS numbering scheme.

AS Numbers on Leaf Switches

When using EBGP in your data center fabric use a different AS number on every leaf switch. This design:

Eliminates BGP tweaks like allowas-in
Makes troubleshooting easier due to easy attribution of prefixes to leaf switches.

You might be forced to use the same AS number on all leaf switches when working with equipment from vendors that never took the effort to simplify BGP configuration for data center fabrics use case – many BGP implementations still require remote IP address and remote AS number in BGP neighbor configuration and cannot support dynamic EBGP neighbors that use a variety of AS numbers.

AS Numbers for MLAG Clusters

Layer-3-only fabric designs need special provisions for multi-homed servers, regardless of whether the servers use link aggregation (LAG) to connect to the network or active/standby uplinks. Leaf switches to which the redundantly-connected servers are attached must share a VLAN and an IP subnet and must be connected with a direct link to exchange layer-2 intra-subnet traffic.

IP Subnets Shared Across ToR Switches

Layer-2 fabrics using EVPN or layer-2 routing technologies (like TRILL or SPB) don’t have the same limitations from the traffic forwarding perspective, but might still need a direct inter-leaf link to support MLAG functionality.

Traditional BGP designers might be inclined to use the same BGP AS number on all members of an MLAG cluster (or a leaf pair sharing a VLAN) as these switches advertise the same IP prefix into the data center fabric.

Shared versus Unique AS Numbers

Large-scale data center fabric designers prefer to deviate from traditional BGP design guidelines and use a unique BGP AS number on every leaf switch to:

make route advertisement attribution easier – the AS path in a BGP update uniquely identifies the leaf switch originating the advertisement;
simplify automation scripts – leaf switch BGP configuration is consistent regardless of whether the leaf switch provides redundant or non-redundant server connectivity.

More information

Explore related webinars, podcasts an blog posts;
Leaf-and-Spine Fabric Architectures webinar gives you plenty of Data Center BGP design guidelines based on practical experience;
A good data center networking engineer or architect understands the networking technologies and design guidelines. A great networking engineer or architect understands the needs and limitations of the whole stack of technologies and architectures, including compute, storage, and network services. You'll get that in the Building Next-Generation Data Center online course.

Continue

Next: BGP in EVPN-Based Data Center Fabrics