This post represents the solution and explanation for quiz-24.

Quiz Review

Quiz #24 opens the discussion about a scenario in which traffic is black-holed when a certain link fails. Let's summarize the quiz:

default-information-originate-always "Side Effects of ALWAYS keyword"

  • company ABC runs OSPF internally, in all 3 buildings
  • internet access is provided via 2 Border Routers (BR-B and BR-C) each connected to a separate ISP
  • each BR receives a default route from its directly connected ISP (via eBGP)
  • each BR is configured with default-information originate always (the network administrator considered that the Border Routers / BRs will always be an exit point for the internet traffic sourced from the internal networks/buildings
  • buildings A and D are single-connected to only 1 Border Router (BR-B and BR-C, respectively)

In the above configuration, it was noticed that when ISP-1 fails, CORE-A (and all users in Building-A) cannot reach the internet any more, even though CORE-A still has a default route: 

CORE-A#sh ip ospf data
...
                Type-5 AS External Link States

Link ID         ADV Router      Age         Seq#       Checksum Tag
0.0.0.0         192.168.12.2    664         0x80000003 0x00CA6E 1
0.0.0.0         192.168.15.1    1114        0x80000002 0x00BD7A 1
CORE-A#
CORE-A#sh ip route ospf
...
O*E2 0.0.0.0/0 [110/1] via 192.168.15.1, 00:51:22, FastEthernet0/0
CORE-A#

The network engineer expected that when ISP-1 fails, internet connectivity from Building-A would not be impacted as it would go from CORE-A to BR-B, then BR-C and to Internet via ISP-2.

Problem description

Unfortunately the network admin was wrong ... simple troubleshooting shows that if/when ISP-1 fails, traffic from Building-A is black-holed on BR-B.
Let's see what happens in details (again, when ISP-1 fails):

  • on CORE-A:

    • as shown above, CORE-A still has both Type 5 LSAs for 0.0.0.0 (injected by BR-B and BR-C) and installs a default route via BR-B (all fine here)

  • on BR-B:

    • due to the always keyword, BR-B generates a Type-5 LSA (external) for default 0.0.0.0
    • BR-B receives also the Type-5 External LSA for 0.0.0.0 originated by the other border router BR-C

    • checking the routing table, we see that BR-B does not have any default route even though it does have both Type-5 LSAs in the OSPF database:

      BR-B#
*Mar  1 00:02:58.203: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Down Interface flap
BR-B#
BR-B#sh ip route 0.0.0.0
% Network not in table
BR-B#
BR-B#
BR-B#sh ip ospf database
...
                Type-5 AS External Link States
 
Link ID         ADV Router      Age         Seq#       Checksum Tag
0.0.0.0         192.168.12.2    293         0x80000001 0x00CE6C 1
0.0.0.0         192.168.15.1    294         0x80000001 0x00BF79 1
BR-B#

    • there were people suggesting (in the comments of the quiz) that BR-B dis-regards the 0.0.0.0 Type-5 LSA received from the other router BR-C because of higher metric (including cost to the FA/Forwarding Address)... but I do not agree with such an explanation: first, BR-B cannot consider its own generated LSA for SPF calculation (see RFC 1583, Section 16.4 "Calculating AS external routes, point 2) => so it cannot compare BR-C's LSA against something that is already out of the calculation.
      Let's see these LSAs in detail:

      BR-B#sh ip ospf database external
 
            OSPF Router with ID (192.168.15.1) (Process ID 1)
 
                Type-5 AS External Link States
 
  LS age: 1329
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 0.0.0.0 (External Network Number )
  Advertising Router: 192.168.12.2
  LS Seq Number: 80000001
  Checksum: 0xCE6C
  Length: 36
  Network Mask: /0
        Metric Type: 2 (Larger than any link state path)
        TOS: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 1
 
  LS age: 1330
  Options: (No TOS-capability, DC)
  LS Type: AS External Link
  Link State ID: 0.0.0.0 (External Network Number )
  Advertising Router: 192.168.15.1
  LS Seq Number: 80000001
  Checksum: 0xBF79
  Length: 36
  Network Mask: /0
        Metric Type: 2 (Larger than any link state path)
        TOS: 0
        Metric: 1
        Forward Address: 0.0.0.0
        External Route Tag: 1
 
BR-B#

      Note that none of these LSAs have the "Routing Bit Set" (a first sign that something is "fishy" with the LSA received from BR-C)

    • performing an SPF debug will reveal the reason why the BR-C's 0.0.0.0 Type-5 LSA is not considered for the SPF calculation (which explains the missing "Routing Bit Set") => explains why BR-B does not have a default route via BR-C:

      BR-B#debug ip ospf spf
OSPF spf events debugging is on
OSPF spf intra events debugging is on
OSPF spf inter events debugging is on
OSPF spf external events debugging is on
BR-B#
... OSPF: Start partial processing Type 5 External LSA 0.0.0.0, mask 0.0.0.0, 
adv 192.168.12.2, age 1, seq 0x80000003, metric 1, metric-type 2
BR-B#
... OSPF: We originate default always. Don't install default from others
... OSPF: delete lsa id 0.0.0.0, type 5, adv rtr 192.168.12.2 from delete list

Conclusion:
Any OSPF router that originates a default with always keyword will never accept other 0.0.0.0 Type-5 LSA from another neighbor !




Quiz Solutions

1. Remove ALWAYS

The simplest solution is to remove the always keyword. After doing so, the SPF debugs shows that the Type-5 LSA 0.0.0.0 from BR-C (192.168.12.2) is accepted into the SPF calculations and a default route via BR-C is installed:

*Mar  1 01:00:40.447: OSPF: Start partial processing Type 5 External LSA 0.0.0.0, mask 0.0.0.0, 
adv 192.168.12.2, age 1583, seq 0x80000002, metric 1, metric-type 2
*Mar  1 01:00:40.447:    Add better path to LSA ID 0.0.0.0, gateway 192.168.12.2, dist 1
*Mar  1 01:00:40.447:    Add path: next-hop 192.168.12.2, interface FastEthernet0/0
*Mar  1 01:00:40.447:  network update dest_addr 0.0.0.0 mask 0.0.0.0 gateway 192.168.12.2
*Mar  1 01:00:40.447:    Add External Route to 0.0.0.0\. Metric: 1, Next Hop: 192.168.12.2
*Mar  1 01:00:40.451: OSPF: insert route list LS ID 0.0.0.0, type 5, adv rtr 192.168.12.2
BR-B#
BR-B#sh ip route ospf
O*E2 0.0.0.0/0 [110/1] via 192.168.12.2, 00:01:50, FastEthernet0/0
BR-B#

Of course, without the "always" keyword and with the link to ISP-1 down, BR-B does not generate a 0.0.0.0 LSA, so the only one in the OSPF database is the one from neighbor BR-C (note the "Routing Bit Set" on this LSA):

BR-B#sh ip ospf database external

    OSPF Router with ID (192.168.15.1) (Process ID 1)

        Type-5 AS External Link States

Routing Bit Set on this LSA
LS age: 1708
Options: (No TOS-capability, DC)
LS Type: AS External Link
Link State ID: 0.0.0.0 (External Network Number )
Advertising Router: 192.168.12.2
LS Seq Number: 80000002
Checksum: 0xCC6D
Length: 36
Network Mask: /0
      Metric Type: 2 (Larger than any link state path)
      TOS: 0
      Metric: 1
      Forward Address: 0.0.0.0
      External Route Tag: 1

In my opinion, as a matter of best practice, this is the recommended solution because the injection of the default route should be conditional in most scenarios (use a default-information originate without always or, even better, together with a route-map).

2. iBGP between BRs

Another solution would be to form an iBGP peering between BR-B and BR-C and keep the always keyword.
This way, in case the ISP-1 fails, BR-B will install the default via iBGP from BR-C and connectivity in Building-A will not be impacted. 

BR-C(config-router)#router bgp 65001
BR-C(config-router)#neigh 192.168.12.1 remote-as 65001
BR-C(config-router)#neigh 192.168.12.1 next-hop-self


Thanks again for all your comments in the quiz !
Subscribe to this blog to get more interesting quizzes and detailed solutions.