OSPF on CE-PE links

Here it comes a pretty long post that summarizes the characteristics of OSPF protocol when using it on a CE-PE link. For those without too much free time (myself included), you can take a fast look at the drawing above that serves as a fast summary / review.
If you want to find out more details, then go ahead and read this post till the end (more cool drawings will come ☺ ).
UPDATE: A special post was dedicated to Understanding the Down Bit in OSPF on CE-PE links.

Some basic comments/features

Let's start with reviewing some basic comments/features:

• MP-BGP is used to transport the routing info accross the MPLS backbone
• all internal routes (LSA 1, LSA 2, LSA 3) are carried as inter-area summary Type-3 LSAs
• PE routers appear as ABR (OSPF Area Border Routers) for the devices in customer OSPF domains
• there are NO OSPF ADJACENCIES or flooding across the MPLS cloud (except when sham-links are used)
• MP-BGP cloud can be seen as a "superbackbone" / "super area 0" that gives the following advantages:
  • we may have non-zero areas at different VPN sites without the need of an area 0
  • we may have area 0 at different sites together with non-zero areas attached to the superbackbone
• OSPF information is carried across the MPLS VPN cloud using BGP extended communities

BGP extended communities

The BGP extended communities used to carry OSPF information over the MPLS cloud (via MP-BGP) are:

• router-id = router ID of the PE in the relevant VRF instance of OSPF
• domain-id
  • by default, equal to the OSPF process number
  • manually configured via command "domain-id"
  • if the domain ID of the route does not match the domain ID on the receiving PE the route is translated to the external OSPF route (LSA Type 5) with metric-type E2
• OSPF route-type
• MED (metric)
  • routes sent across the MP-BGP cloud do not increment their metric
  • MED can be manually manipulated to influence path selection

Loop Prevention Mechanisms

The addition of the super-backbone area also introduces the possibility of routing loops. In order to prevent them, several basic loop prevention rules apply:

1. "DOWN" Bit

The DOWN Bit is used to prevent loops in multi-homed sites and it is automatically set in all summary LSA 3 (not in LSA 5) when routes are redistributed from MP-BGP (back) into OSPF.
When a prefix with DOWN bit set is received on an interface which is configured with VRF, that LSA is dropped - The correct sentence is: that LSA is not considered during the SPF calculations!

This feature has an undesirable effect when using VRF-lite in the customer cloud:

The solutions to this problem:
A) configure the command "capability vrf-lite" on the customer router(s), but this may not be supported on all IOS:

router ospf 1 vrf VPN_A
 capability vrf-lite

B) configure different domain-IDs as this will force all redistributed routes to become external (LSA 5, thus bypassing the DN bit check)

UPDATE: There is another post that describes in better detail the usage of the Down bit: OSPF on CE-PE Links and Understanding the Down Bit

2. Route Tagging

The DOWN Bit helps preventing routing loops when Summary Type-3 LSAs are redistributed, but not when external routes are announced (the DN bit is not set on LSA Type 5/7). Instead, routes redistributed via a particular PE will carry the OSPF route tag which, by default, is the BGP AS number.
This tag is preserved when the external route is propagated across the entire OSPF domain (including redistribution into another OSPF domains).
The Loop Prevention mechanisms works this way: when another PE receives this route and it sees that its own local AS number matches the AS number in the tag, it will ignore this LSA.
Here is the configuration for it:

router ospf 1 vrf VPN_A
 domain-tag 777

or

router ospf 1 vrf VPN_A
 redistribute bgp 100 subnets tag 777

Sham-Link

Situation/problem description: when there is a backup link between sites (called a backdoor), this link will always be used for inter-site traffic because intra-area routes (LSAs received via the backdoor) are preferred to the inter-area (LSA 3 received from PE).
The solution to this problem is the usage of OSPF sham-links = special tunnel similar to virtual-links between PE routers and configured in the same area as the PEs.

Sham-links have the following characteristics:

• OSPF adjacency established via MPLS cloud
• routes in the OSPF database are now seen as intra-area (even though they are received via the super-backbone)
• the information across the sham-link is ONLY used for SPF calculations - the actual forwarding is being done based on the info learned via MP-BGP

One last note about sham-links: sham endpoints IP addresses should be advertised into the VRF by means other than OSPF (commonly via BGP) - known before creating the sham-links.

I hope you did not fell asleep reading this long post. At least the drawings helped you do a faster "reading".