This post represents the solution and explanation for quiz-24.
Quiz #24 opens the discussion about a scenario in which traffic is black-holed when a certain link fails. Let's summarize the quiz:
- company ABC runs OSPF internally, in all 3 buildings
- internet access is provided via 2 Border Routers (BR-B and BR-C) each connected to a separate ISP
- each BR receives a default route from its directly connected ISP (via eBGP)
- each BR is configured with
default-information originate always(the network administrator considered that the Border Routers / BRs will always be an exit point for the internet traffic sourced from the internal networks/buildings
- buildings A and D are single-connected to only 1 Border Router (BR-B and BR-C, respectively)
In the above configuration, it was noticed that when ISP-1 fails,
sh ip ospf data... Type-5 AS External Link States Link ID ADV Router Age Seq# Checksum Tag 0.0.0.0 192.168.12.2 664 0x80000003 0x00CA6E 1 0.0.0.0 192.168.15.1 1114 0x80000002 0x00BD7A 1CORE-A# CORE-A# sh ip route ospf... O*E2 0.0.0.0/0 [110/1] via 192.168.15.1, 00:51:22, FastEthernet0/0CORE-A#
The network engineer expected that when ISP-1 fails, internet connectivity from Building-A would not be impacted as it would go from CORE-A to BR-B, then BR-C and to Internet via ISP-2.
Unfortunately the network admin was wrong ... simple troubleshooting shows that if/when ISP-1 fails, traffic from Building-A is black-holed on BR-B.
Let's see what happens in details (again, when ISP-1 fails):
- as shown above, CORE-A still has both Type 5 LSAs for 0.0.0.0 (injected by BR-B and BR-C) and installs a default route via BR-B (all fine here)
- due to the
alwayskeyword, BR-B generates a Type-5 LSA (external) for default 0.0.0.0
- BR-B receives also the Type-5 External LSA for 0.0.0.0 originated by the other border router BR-C
checking the routing table, we see that
BR-B does not have any default routeeven though it does have both Type-5 LSAs in the OSPF database:
BR-B# *Mar 1 00:02:58.203: %BGP-5-ADJCHANGE:
neighbor 220.127.116.11 DownInterface flap BR-B# BR-B#sh ip route 0.0.0.0 % Network not in tableBR-B# BR-B# BR-B# sh ip ospf database... Type-5 AS External Link States Link ID ADV Router Age Seq# Checksum Tag 0.0.0.0 192.168.12.2 293 0x80000001 0x00CE6C 1 0.0.0.0 192.168.15.1 294 0x80000001 0x00BF79 1 BR-B#
there were people suggesting (in the comments of the quiz) that BR-B dis-regards the 0.0.0.0 Type-5 LSA received from the other router BR-C because of higher metric (including cost to the FA/Forwarding Address)... but I do not agree with such an explanation: first, BR-B cannot consider its own generated LSA for SPF calculation (see RFC 1583, Section 16.4 "Calculating AS external routes, point 2) => so it cannot compare BR-C's LSA against something that is already out of the calculation.
Let's see these LSAs in detail:
sh ip ospf database externalOSPF Router with ID (192.168.15.1) (Process ID 1) Type-5 AS External Link States LS age: 1329 Options: (No TOS-capability, DC) LS Type: AS External Link Link State ID: 0.0.0.0 (External Network Number ) Advertising Router: 192.168.12.2LS Seq Number: 80000001 Checksum: 0xCE6C Length: 36 Network Mask: /0 Metric Type: 2 (Larger than any link state path) TOS: 0 Metric: 1 Forward Address: 0.0.0.0 External Route Tag: 1 LS age: 1330 Options: (No TOS-capability, DC) LS Type: AS External Link Link State ID: 0.0.0.0 (External Network Number ) Advertising Router: 192.168.15.1LS Seq Number: 80000001 Checksum: 0xBF79 Length: 36 Network Mask: /0 Metric Type: 2 (Larger than any link state path) TOS: 0 Metric: 1 Forward Address: 0.0.0.0 External Route Tag: 1 BR-B#
Note that none of these LSAs have the "Routing Bit Set" (a first sign that something is "fishy" with the LSA received from BR-C)
performing an SPF debug will reveal the reason why the BR-C's 0.0.0.0 Type-5 LSA is not considered for the SPF calculation (which explains the missing "Routing Bit Set") => explains why BR-B does not have a default route via BR-C:
debug ip ospf spfOSPF spf events debugging is on OSPF spf intra events debugging is on OSPF spf inter events debugging is on OSPF spf external events debugging is on BR-B# ... OSPF: Start partial processing Type 5 External LSA 0.0.0.0, mask 0.0.0.0, adv 192.168.12.2, age 1, seq 0x80000003, metric 1, metric-type 2 BR-B# ... OSPF: We originate default always. Don't install default from others ... OSPF: delete lsa id 0.0.0.0, type 5, adv rtr 192.168.12.2 from delete list
- due to the
1. Remove ALWAYS
The simplest solution is to remove the
always keyword. After doing so, the SPF debugs shows that the Type-5 LSA 0.0.0.0 from BR-C (192.168.12.2) is accepted into the SPF calculations and a default route via BR-C is installed:
*Mar 1 01:00:40.447: OSPF: Start partial processing Type 5 External LSA 0.0.0.0, mask 0.0.0.0, adv 192.168.12.2, age 1583, seq 0x80000002, metric 1, metric-type 2 *Mar 1 01:00:40.447: Add better path to LSA ID 0.0.0.0, gateway 192.168.12.2, dist 1 *Mar 1 01:00:40.447: Add path: next-hop 192.168.12.2, interface FastEthernet0/0 *Mar 1 01:00:40.447: network update dest_addr 0.0.0.0 mask 0.0.0.0 gateway 192.168.12.2 *Mar 1 01:00:40.447: Add External Route to 0.0.0.0\. Metric: 1, Next Hop: 192.168.12.2 *Mar 1 01:00:40.451: OSPF: insert route list LS ID 0.0.0.0, type 5, adv rtr 192.168.12.2 BR-B# BR-B#
sh ip route ospf O*E2 0.0.0.0/0 [110/1] via 192.168.12.2, 00:01:50, FastEthernet0/0BR-B#
Of course, without the "always" keyword and with the link to ISP-1 down, BR-B does not generate a 0.0.0.0 LSA, so the only one in the OSPF database is the one from neighbor BR-C (note the "Routing Bit Set" on this LSA):
BR-B#sh ip ospf database external OSPF Router with ID (192.168.15.1) (Process ID 1) Type-5 AS External Link States
Routing Bit Set on this LSALS age: 1708 Options: (No TOS-capability, DC) LS Type: AS External Link Link State ID: 0.0.0.0 (External Network Number ) Advertising Router: 192.168.12.2LS Seq Number: 80000002 Checksum: 0xCC6D Length: 36 Network Mask: /0 Metric Type: 2 (Larger than any link state path) TOS: 0 Metric: 1 Forward Address: 0.0.0.0 External Route Tag: 1
In my opinion, as a matter of best practice, this is the recommended solution because the injection of the default route should be conditional in most scenarios (use a
default-information originate without always or, even better, together with a route-map).
2. iBGP between BRs
Another solution would be to form an iBGP peering between BR-B and BR-C and keep the
This way, in case the ISP-1 fails, BR-B will install the default via iBGP from BR-C and connectivity in Building-A will not be impacted.
BR-C(config-router)#router bgp 65001 BR-C(config-router)#
neigh 192.168.12.1 remote-as 65001BR-C(config-router)# neigh 192.168.12.1 next-hop-self
Thanks again for all your comments in the quiz !
Subscribe to this blog to get more interesting quizzes and detailed solutions.