Routing Protocols

Tutorial: Troubleshooting Split Horizon Issues, Part 1

Thanks for visiting! If you're new here, you may want to subscribe to my RSS feed. This blog posts regular tutorials, news, and study tips about networking, especially about Cisco CCIE related topics. Go ahead, subscribe to the feed! You can also receive updates from this blog via email. Thanks for visiting!

I was reading through Troubleshooting IP Routing Protocols by Cisco Press and came across an example of when to disable Split Horizon on an interface. This particular example involved secondary interfaces, and RIP. After reading about it I tried to go through in my mind all the circumstances in which one might need to disable or troubleshoot split horizon. I labbed up a couple of these scenarios in Dynamips and came up with the following scenarios: Using Distance Vector Routing protocol with Hub and Spoke Frame Relay, Using Distance Vector Routing protocols with Ethernet topologies, Using a Distance Vector Routing protocol with secondary addresses and Redistribution issues with Split Horizon. It is these scenarios that we will be having a look at in the next couple of articles.

What is Split Horizon?
You might remember from CCNA days that split horizon is a technique employed in Distance Vector routing protocols to stop the “counting to infinity” problem. The basic premise is this: it is never useful to send the same update back to a router who just sent it to you. Or as I like to remember it “never tell the same joke to someone who just told it to you”. So for example if a router recieves an update for network 3.0.0.0 in its f0/0 interface, there is no point in sending that same update for 3.0.0.0 out f0/0. Thats where you learnt it from, why do the routers out there need to know about it…they told you!

Lets explore this a bit futher. We will be using the following topology for these scenarios.

You can download the dynagen .net file for this here.

Scenario 1: Using Distance Vector Routing Protocols with Hub and Spoke Frame Relay

The first scenario we will look at is using a distance vector routing protocol with Hub and Spoke Frame Relay. We will only be using R1, R2 and R3 in the topology above for this scenario. First lets set up R1, R2, and R3 with the following configuration:

R1:

hostname R1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.0
!
interface FastEthernet0/0
 ip address 192.168.145.1 255.255.255.0
 duplex auto
 speed auto
!
interface Serial1/0
 no ip address
 encapsulation frame-relay
 no frame-relay inverse-arp
 ip address 192.168.123.1 255.255.255.0
 frame-relay map ip 192.168.123.2 102 broadcast
 frame-relay map ip 192.168.123.3 103 broadcast
!
router rip
 version 2
 passive-interface Loopback0
 network 192.168.123.0
 no auto-summary
!

R2:

hostname R2
!
interface Loopback0
 ip address 2.2.2.2 255.255.255.0
!
interface Serial1/0
 ip address 192.168.123.2 255.255.255.0
 encapsulation frame-relay
 frame-relay map ip 192.168.123.1 201 broadcast
 frame-relay map ip 192.168.123.3 201
 no frame-relay inverse-arp
!
router rip
 version 2
 passive-interface Loopback0
 network 192.168.123.0
 no auto-summary
!

R3:

hostname R3
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.0
!
interface Serial1/0
 ip address 192.168.123.3 255.255.255.0
 encapsulation frame-relay
 frame-relay map ip 192.168.123.1 301 broadcast
 frame-relay map ip 192.168.123.2 301
 no frame-relay inverse-arp
!
router rip
 version 2
 passive-interface Loopback0
 network 192.168.123.0
 no auto-summary
!

Here we have set up a basic Frame Relay hub and spoke topology with inverse arp disabled. R1 is the Hub with R2 and R3 the spokes. RIP has been activated across the frame relay cloud, although nothing has been advertised across the frame relay link.

Let advertise R3’s loopback accross the link.

R3:

R3#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R3(config)#router rip
R3(config-router)#network 3.0.0.0
R3(config-router)#exit
R3(config)#exit

Lets check the routing table on R2.

R2#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.123.0/24 is directly connected, Serial1/0
     2.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0
     3.0.0.0/24 is subnetted, 1 subnets
R       3.3.3.0 [120/2] via 192.168.123.3, 00:00:01, Serial1/0

Hang on a sec! Why is the 3.3.3.0 network showing up in the routing table on R3? Shouldn’t split horizon stop R1 advertising back out its S1/0 interface to R2? I didn’t use a no ip split-horizon command on R1’s S1/0 to disable split horizon yet, so what gives?

Let’s have a look at whats happening on R1:

R1#sh ip int s1/0
Serial1/0 is up, line protocol is up
  Internet address is 192.168.123.1/24
  Broadcast address is 255.255.255.255
  Address determined by setup command
  MTU is 1500 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Multicast reserved groups joined: 224.0.0.9
  Outgoing access list is not set
  Inbound  access list is not set
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is disabled
  ICMP redirects are always sent
  ICMP unreachables are always sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP fast switching on the same interface is enabled
  IP Flow switching is disabled
  IP CEF switching is enabled
  IP CEF Fast switching turbo vector
  IP multicast fast switching is enabled
  IP multicast distributed fast switching is disabled
  IP route-cache flags are Fast, CEF
  Router Discovery is disabled
  IP output packet accounting is disabled
  IP access violation accounting is disabled
  TCP/IP header compression is disabled
  RTP/IP header compression is disabled
  Policy routing is disabled
  Network address translation is disabled
  BGP Policy Mapping is disabled
  WCCP Redirect outbound is disabled
  WCCP Redirect inbound is disabled
  WCCP Redirect exclude is disabled

You can see above that Split horizon has been disabled by default. When we configure Frame Relay using the physical interface and not sub interfaces, Cisco IOS actually disables split horizon for you. It worked automagically!

Normally what should happen is that R1 receives the 3.3.3.0 network in its s1/0. With split horizon enabled it would not advertise that route out to R2 because it would have to exit S1/0: the interface that it was recieved on! A distance vector routing protocol like RIP assumes that it can broadcast. R1 makes the assumption that R2 would have already got that update so there is no need to send it back out there. With frame-relay being a non broadcast technology, this is an invalid assumption. Cisco disables split horizon by default if you configure frame-relay hub and spoke using the physical interface.

Lets see what happens though when we reconfigure R1 to use a multipoint sub-interface:

R1:

hostname R1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.0
!
interface FastEthernet0/0
 ip address 192.168.145.1 255.255.255.0
 duplex auto
 speed auto
!
interface Serial1/0
 no ip address
 encapsulation frame-relay
 no frame-relay inverse-arp
!
interface Serial1/0.123 multipoint
 ip address 192.168.123.1 255.255.255.0
 frame-relay map ip 192.168.123.2 102 broadcast
 frame-relay map ip 192.168.123.3 103 broadcast
!
router rip
 version 2
 passive-interface Loopback0
 network 192.168.123.0
 no auto-summary
!

Let's have a look at the split horizon status on R1:

R1#sh ip int s1/0.123
Serial1/0 is up, line protocol is up
  Internet address is 192.168.123.1/24
  Broadcast address is 255.255.255.255
  Address determined by setup command
  MTU is 1500 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Multicast reserved groups joined: 224.0.0.9
  Outgoing access list is not set
  Inbound  access list is not set
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is enabled
  ICMP redirects are always sent
  ICMP unreachables are always sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP fast switching on the same interface is enabled
  IP Flow switching is disabled
  IP CEF switching is enabled
  IP CEF Fast switching turbo vector
  IP multicast fast switching is enabled
  IP multicast distributed fast switching is disabled
  IP route-cache flags are Fast, CEF
  Router Discovery is disabled
  IP output packet accounting is disabled
  IP access violation accounting is disabled
  TCP/IP header compression is disabled
  RTP/IP header compression is disabled
  Policy routing is disabled
  Network address translation is disabled
  BGP Policy Mapping is disabled
  WCCP Redirect outbound is disabled
  WCCP Redirect inbound is disabled
  WCCP Redirect exclude is disabled

You can see that on a multipoint sub-interface split horizon is enabled by default. Let’s check R2’s routing table now:

R2#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.123.0/24 is directly connected, Serial1/0
     2.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0
     3.0.0.0/24 is subnetted, 1 subnets
R       3.3.3.0 [120/2] via 192.168.123.3, 00:00:52, Serial1/0

Looks like RIP’s hold down timer hasn’t expired yet. Let’s wait a little while and try again:

R2#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.123.0/24 is directly connected, Serial1/0
     2.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0
R2#

Bingo! You can see there that R2 has lost its route to 3.3.3.0/24. This is because R1 is no longer advertising this out its S1/0 interface. Lets turn on debugging on R1 and have a look at this:

R1#debug ip rip
RIP protocol debugging is on
R1#
R1#
*Mar  1 00:33:55.391: RIP: received v2 update from 192.168.123.2 on Serial1/0.123
*Mar  1 00:33:55.391:      192.168.123.0/24 via 0.0.0.0 in 1 hops
R1#
*Mar  1 00:33:58.647: RIP: received v2 update from 192.168.123.3 on Serial1/0.123
*Mar  1 00:33:58.647:      3.3.3.0/24 via 0.0.0.0 in 1 hops
*Mar  1 00:33:58.651:      192.168.123.0/24 via 0.0.0.0 in 1 hops
R1#
*Mar  1 00:34:17.547: RIP: sending v2 update to 224.0.0.9 via Serial1/0.123 (192.168.123.1)
*Mar  1 00:34:17.551: RIP: build update entries - suppressing null update

You can see in the above output that we recieved the update from R3 in s1/0.123. You can also see that when its time to send an update R1 reports “suppressing null update”. According to R1 there is nothing to send out! It received its update on S1/0, and i can’t send it out s1/0 because split horizon is enabled!

Lets disable split horizon on R1 and see if it fixes our problem.

R1#conf t
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#int s1/0.123
R1(config-subif)#no ip spli
R1(config-subif)#no ip split-horizon
R1(config-subif)#end

R1#
*Mar  1 00:42:21.535: RIP: received v2 update from 192.168.123.3 on Serial1/0.123
*Mar  1 00:42:21.539:      3.3.3.0/24 via 0.0.0.0 in 1 hops
*Mar  1 00:42:21.543:      192.168.123.0/24 via 0.0.0.0 in 1 hops
R1#
*Mar  1 00:42:39.771: RIP: sending v2 update to 224.0.0.9 via Serial1/0.123 (192.168.123.1)
*Mar  1 00:42:39.775: RIP: build update entries
*Mar  1 00:42:39.775:   3.3.3.0/24 via 192.168.123.3, metric 2, tag 0
*Mar  1 00:42:39.779:   192.168.123.0/24 via 0.0.0.0, metric 1, tag 0

Looks like we are sending the update out s1/0.123 again! Lets check if this route shows up on R2:

R2#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.123.0/24 is directly connected, Serial1/0
     2.0.0.0/24 is subnetted, 1 subnets
C       2.2.2.0 is directly connected, Loopback0
     3.0.0.0/24 is subnetted, 1 subnets
R       3.3.3.0 [120/2] via 192.168.123.3, 00:00:02, Serial1/0

Awesome! R2 now has its route out to the 3.3.3.0/24 subnet. So if you are using distance vector routing protocols like rip, igrp and eigrp on a frame-relay hub and spoke topology, remember to disable split horizon if you are using a multipoint subinterface on the hub. For eigrp we would use the no ip split-horizon eigrp command to disable split horizon. For rip we would use the no ip split-horizon command to disable split horizon.

In our next article we will be exploring some of the issues with split horizon and ethernet topologies.

HTH! Now back to labs!

Summary:

• Split horizon is a mechanism used in distance vector routing protocols to stop the counting to infinity problem
• On a physical serial interface split horizon is disabled by default
• On a serial sub-interface split horizon is enabled by default.
• Use the no ip slit-horizon command under an interface to disable split-horizon when using RIP.
• Use the no ip split-horizon eigrp command under an interface to disable split-horizon when using EIGRP.

Resources:

• Dynagen Split Horizon Topology
• R1 Split Horizon Config
• R2 Split Horizon Config
• R3 Split Horizon Config

Related posts

• Tutorial: Troubleshooting Split Horizon Issues, Part 2
• Tutorial: OSPF Network Types and Frame Relay Part 2
• Tutorial: OSPF Network Types and Frame Relay Part 1
• Troubleshooting IP Routing Protocols Video Part 2

© 2007 Arden Packeer. All rights reserved. The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway. This material is not sponsored or endorsed by Cisco Systems, Inc. Cisco, Cisco Systems, CCIE and the CCIE Logo are trademarks of Cisco Systems, Inc. and its affiliates