ICND2 200-101

ICND2 200-101

Troubleshooting Multiarea OSPF Issues

Let me be honest - it is not so important for you to memorize this flowchart. It's not, it's not at all. But we're going to go through a flow here together. Just to describe all necessary steps that we follow depending on the type of circumstances or situation that might arise in our Open Shortest Path First, or OSPF, environment.

Troubleshooting Multiarea OSPF

We have to have layer 3 connectivity to our adjacent devices for anything to work. Any routing protocol needs good layer 3. So I would be making sure that my interfaces are up/up, that they're addressed. And of course, included in this, as we have to agree about the subnet mask. OSPF will bark at you, if you disagree about the subnet mask being the common link between two adjacent routers. So show ip interface brief doesn't give us that. We might, in fact, want to look at that as well. So I would be looking at branch, I would also be looking at HQ.

Troubleshooting Multiarea OSPF Example

Troubleshooting OSPF neighbor issues

Now why might it be necessary for us to ping, to make sure we have connectivity? Well here we're in a point-to-point scenario. So we'll see our interfaces up/up, as up/up will be connected. We'll ping to make sure that we have connectivity to that Internet Protocol, or IP, address. We had to be in the same subnet or else we won't form neighbor adjacencies.

Branch#ping 192.168.1.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/10 ms

But what about a multi access environment? Are we going to be directly connected to that other router? Not necessarily. We've seen examples where we're directly connected over that Ethernet link. But what if we had a topology with four routers and we had switches in there providing the connectivity that we need? So we're no longer directly connected, we'd still be up/up. But do we officially have Layer 3 connectivity? So don't just assume you have Layer 3 connectivity. Execute a ping to make sure you have connectivity to those devices you will be forming a neighbor adjacency with.

Now this is not the most glorious command. But I'll tell you, it's one of the most examination useful commands, especially if you were to go into a troubleshooting oriented course.

Branch#show ip ospf interface
Serial0/0/0 is up, line protocol is up
  Internet Address 192.168.1.1/30, Area 0, Attached via Network Statement
  Process ID 1, Router ID 192.168.1.1, Network Type POINT_TO_POINT, Cost: 64
...

show ip ospf interface and also show ip eigrp interface are both the best way to make sure that we're enabled for the interfaces that we expect. So we do this and if it comes up in the listing, that interface is enabled for OSPF. And so if we were challenged with forming a neighborship adjacency, we can do this and say, "Hey, am I locally configured?", or I'm not even running OSPF on that link because, maybe, my wildcard mask was something other than what I needed. My wildcard mask did not include that IP address that could've been a cause for this. We can also see the area, we'll cover that a little bit later. But of course the area has to match.

Troubleshooting neighbor adjacency involves making sure that both routers are configured properly. So we've just verified branch. Let's use the exact same command to verify HQ. So are my interfaces enabled for OSPF, same command, or up/up on Serial0/0/0. There is our IP address. And are we in the proper area? We are.

HQ#sh ip ospf interface
Serial0/0/0 is up, line protocol is up
  Internet Address 192.168.1.2/30, Area 0
  Process ID 1, Router ID 192.168.1.2, Network Type POINT_TO_POINT, Cost: 64
...

Here is a challenge for you. What if you couldn't go to HQ? What's a way that I could watch neighborship formation or potentially neighborship not being formed and observing that real-time? What would you be thinking about if you couldn't go to HQ or you're stuck on branch and really dealing with this? Because sometimes you don't have the luxury of telnetting or secure shelling into that other router in a timely way.

We could debug our OSPF packet and see if we are successfully sending "hello" packets and receiving "hello" packets. Yeah, and I might also do a debug IP OSPF adjacencies, where adjacencies is ADJ. So you could do a debug in the real world too. And that would be a pretty useful thing to do if you weren't able to do this command show ip ospf interface on HQ.

This probably isn't the best way to view your area if you've already done show ip ospf interface. Just look at the interface output, and it's got the area right in it. And actually, that's, in fact, going to be more telling because you could look at the show ip protocols output and you could be misled by the wildcard mask. Or if you've done the interface level command, it's going to come up as an interface here and could get confusing. So you know the lesson here is, we have to match the areas on all common OSPF routers in a broadcast domain. If we're running OSPF between five routers in a broadcast domain or here two, they better agree about the area.

Branch#show ip protocols
Routing Protocol is "ospf 1"
...
  Routing for Networks:
    10.1.1.0 0.0.0.255 area 1
    192.168.1.0 0.0.0.3 area 0
...

So in this case, branch area 0 and that was for interface serial 0/0/0. It's good, let's verify HQ now.

HQ#show ip protocols
Routing Protocol is "ospf 1"
  Routing for Networks:
    172.16.1.0 0.0.0.255 area 0
    192.168.1.0 0.0.0.3 area 0
...

HQ's interface Serial0/0/0, also in area 0. All right, so we are still not neighbors and we've verified all these different parameters. What else can we verify?

Well are any of the interfaces passive? Passive interface serial 0/0/0 on either the branch router or HQ router? Where can we verify them if an interface is passive? show ip protocols will tell us. I don't see any information here on branch router indicating that serial 0/0/0 is passive.

Let's go check HQ then. Oops, problem identified, problem identified and you're going to quickly resolve this one. You got to remove that passive interface command.

HQ#sh ip protocols
Routing Protocol is "ospf 1"
  Routing for Networks:
    172.16.1.0 0.0.0.255 area 0
    192.168.1.0 0.0.0.3 area 0
  Passive Interface(s):
    Serial0/0/0
...

So go into OSPF configuration mode, because that's where it's done. It's not in interface level command. No passive interface serial0/0/0, boom. Quickly, we've got to get one of those messages that says, "Loading to full."

*Jan 7 22:50:41.072: %OSPF-5-ADJCHG: Process 1, Nbr 192.168.1.1 on Serial0/0/0 from LOADING to FULL, Loading Done

We're good, we've got our neighborship. So you look at this and you got to understand, there is one of those things that can break your adjacency just like we warned you before.

Troubleshoot OSPF routing table issues

So now we're neighbors. But we have another problem. We can't reach a specific network. Is HQ advertising that particular network? We can check this with show ip protocols - there is a row Routing for Networks.

HQ#show ip protocols
  Outgoing update filter list for all interfaces is not set
  Incoming update filter list for all interfaces is not set
  Router ID 192.168.1.2
  Number of areas in this router is 1. 1 normal 0 stub 0 nssa
  Maximum path: 4
  Routing for Networks:
    192.168.1.0 0.0.0.3 area 0

Do you see anything missing? I see that we're currently only routing for networks 192.168.1.0. So HQ wasn't configured properly to turn OSPF on that interface that is part of the 172.16.1.0/24 network.

This is kind of fun. Let's fix something that isn't broken. We can get there, we don't know why we can get there. That's kind of the problem. We are running OSPF. I can get to this network but I don't see it as an OSPF route. Well we could have probably figured that out by looking at the routing table. But if you want to get details about a route, you can do this command - show ip route and then you can put, in fact, the network. You can get some really powerful information. Things like routing tags show up here too. If it's like a redistributed route, you'd see tags and lots and lots of goodies. So you want to get hyper detailed information about a network as a routing entry, this is the place to go.

Branch#show ip route 172.16.1.0
Routing entry for 172.16.1.0/24
  Known via "eigrp 1", distance 90, metric 2195456, type internal
  Redistributing via eigrp 1
  Last update from 192.168.1.2 on Serial1/0, 00:01:02 ago
  Routing Descriptor Blocks:
  * 192.168.1.2, from 192.168.1.2, 00:01:02 ago, via Serial0/0/0
      Route metric is 2195456, traffic share count is 1
      Total delay is 21000 microseconds, minimum bandwidth is 1544 Kbit
      Reliability 255/255, minimum MTU 1500 bytes
      Loading 1/255, Hops 1

So we are to set, "Oh! right, I've got EIGRP running on this network also." And EIGRP route will trump OSPF because it has the superior administrative distance of 90 versus 110. So this would be an interesting thing to resolve. I'm not going to tell you the right way to resolve this one because somehow we have EIGRP running also and we have to be very cautious at this point, because usually you don't want two different sources telling us the same information. But at least we've figured out why OSPF is not the reason why we can get there.

Troubleshoot OSPF path selection

In this troubleshooting example, we have traffic crossing both the Gigabit link as well the serial link at the same time.

Multiarea OSPF with Multipaths

How do we know this? Let's look at our routing table, show ip route ospf.

Branch#show ip route ospf
      172.16.0.0/24 is subnetted, 1 subnets
O        172.16.1.0 [110/2] via 209.165.201.2, 00:00:00, GigabitEthernet0/1
                    [110/2] via 192.168.1.2, 00:00:00, Serial0/0/0

We can see that both paths are being utilized to get to 172.16.1.0 from branch. But wait, what is the default speed of a Gigabit Ethernet link? That's 1000, it's just 1000 megabits per second. What's the default for serial? It's 1.544 megabits per second. That has a cost of 64 by default. All right, so we have a cost of 1 by default and a cost of 64 by default. They're not the same, but we can see here in our routing table that the cost is listed as two to get to the destination network. So right there bells should be going off! Somebody's modified cost somewhere. So this proves the point - not all load-balancing is good, right? We have that pinhole congestion.

Branch#show ip ospf interface
GigabitEthernet0/1 is up, line protocol is up
  Internet Address 209.165.201.1/27, Area 0, Attached via Network Statement
  Process ID 1, Router ID 192.168.1.1, Network Type BROADCAST, Cost: 1
...
Serial0/0/0 is up, line protocol is up
  Internet Address 192.168.1.1/30, Area 0, Attached via Network Statement
  Process ID 1, Router ID 192.168.1.1, Network Type POINT_TO_POINT, Cost: 1
...

The gigabit link is basically slowed down effectively to the speed of the serial link and we just cause some massive quality problems for our traffic because we've probably got a bottleneck between the two places now and we're dropping just loads of traffic. That's probably what we would experience. Just massive bottlenecks, which are exhibited by dropped traffic because we get what is called "tail drop". Packets come in, router doesn't have room for it in memory buffers, because it's still waiting to send it down the gigabit link, it dies. So sometimes it's better to just go across one link.