Why use a full iBGP Mesh?

In this blog I will be showing you a use case where a full iBGP mesh is required and why is it recommended BGP best practice.

INTRODUCTIONPROBLEMSOLUTION

I will be working on the following network diagram:

Here, I have router R13 running eBGP with route R14 and, R18 running eBGP with R16. Within AS 65001, I have OSPF configured as the IGP; on top of OSPF, I am then running an iBGP (internal) session between R13 and R18 routers. Since synchronisation is disabled by default, I have left it as is.

At last, during this lab, I will be focusing on network 89.89.89.89 configured on the Loopback0 interface connected to R16 router – the red arrow is showing how router R14 is learning this route:

  1. R16 advertises the route to R18 over the eBGP session
  2. R18 router, is not redistributing the route into OSPF; the route is rather advertised to R13 router, over the iBGP session
  3. Further, R13 advertises the route to R14
On bullet point 2 above, the recall that BGP, by design, will automatically advertise routes learnt from eBGP peers, to its iBGP peers –  hence, no network command is actually needed here; all you need is a peering session.

Secondly, notice that the iBGP session between R13 and R18 is established over a logical topology – i.e., there is no direct physical link between these two routers. The peering session is a TCP point to point session – at the IP Layer however, everything happens through router R3.

So routing has been configured and my routers should know how to route to 89.89.89.89, right? Let’s confirm:

You can clearly see below that, we don’t have reachability point-to-point since we cannot reach beyond 10.0.1.0. You probably noticed the ICMP error code identified as “!H” – this stands for “ICMP Host Unreachable” – which tells us that the router at 10.0.1.0 doesn’t have a route to network 89.89.89.89.

Also notice that R14 will use 10.0.0.0 (R13) as the next hop to reach the 89.89.89.89 network. If we check the routing table on R13 …

So, router R13 knows how to route traffic back (to 10.0.0.0) and also knows about network 89.89.89.89, via next-hop of 18.18.18.18 (this is fine because the route is actually learnt via BGP and the BGP peer, is indeed 18.18.18.18).

Notice however, that 18.18.18.18 is not part of a directly connected network; so the router will have do a recursive lookup to find out how to route to this next-hop. From the routing table, we can see that the next hop will be 10.0.1.0 (R3).

So let’s check the routing table at the next hop router (R3):

The initial traceroute was initiated on R14; by default, the source IP address in the packet will  be set to the IP address as set on the exit interface – in our case this is 10.0.0.1 – we can see in the routing table that the network 10.0.0.0 is not missing! So the router should be able to route packets back!

But, where is the route to 89.89.89.89 ?? It seems like we found where the problem is: router R3 is dropping the packet since it never got to learn this route; the route exists in BGP only and this route is not known to the IGP process running on either routers.

A BGP domain (AS 65001) should not advertise a route to another BGP domain (AS 65002) unless it knows how to fully route to it – as we can see here, traffic to 89.89.89.89 is being black-holed within AS 65001.

This is a case where BGP and the IGP are out of sync!

So we have two possible scenarios so far:

  1. We leave BGP sync off – in which case we are black-holing traffic inside AS 65001
  2. We turn BGP sync on and AS65002 would still not know how to route to 89.89.89.89

So how do we achieve full reachability for 89.89.89.89 route?

  1. R3 learns how to route to 89.89.89.89 via IGP – this implies redistributing the BGP routes into the IGP (OSPF, in our case). However, this is not a recommended best practice and it should be avoided.

2. A much better solution is to have all routers learn routes via iBGP and keep BGP synchronisation disabled.

Creating two additional iBGP sessions – R3-R13 and R3-R18 respectively – would give us a full iBGP mesh. By doing so, as router R18 learns of route 89.89.89.89, it would share it with both R3 and R13 – so all routers would know how to route to this particular network. Router R13 would then advertise the route out, since synchronisation would have been disabled.

Other options include:

  • Using BGP route reflectors to achieve full BGP reachability
  • Create an IP GRE tunnel between R18 and R13 routes and run iBGP over the tunnel
  • Add physical link between R13 and R18 routers and run iBGP over the link

Good BGP design promotes a full iBGP mesh over point-to-point L3 links (including virtual links) and BGP synchronisation should always be disabled.


 

Thank you,

Rafael A. Couto Cabral • LinkedIn Profile
Cisco​ | F5 | VMware Certified • PRINCE2 Practitioner

Originally posted 2019-04-23 09:00:32.

Related Post

Leave a Reply