There are many problems that could occur on a wired network. In this video, you’ll learn about some of the most common problems and how to troubleshoot them.
<< Previous Video: Command Line Tools Next: Wireless Network Troubleshooting >>
On any network connection, whether you’re using fiber or copper, the signal will begin to degrade as it goes farther and farther in distance. We call this signal loss attenuation. And if we have too much attenuation on a wire or fiber, we won’t be able to hear that signal when it reaches the other side.
On any of your networks, you’ll have to consider attenuation when you begin engineering and troubleshooting these networks. If these are copper cables, you’ll have electrical signals that will attenuate as they move through the cable. If it’s a fiber connection, the same thing happens with light. And if these are radio waves on a wireless network, you’ll have attenuation as you move farther away from the access point.
One way to numerically quantify the signal strength or loss of a signal is to use decibels. A decibel is literally 1/10 of a bell. And if you see that abbreviation of decibel, it’s dB. And the B is capitalized in honor of Alexander Graham Bell. Decibels are measured logarithmically.
So if you were to measure twice the amount of signal across a line, you would say that it had increased 3 decibels. If you increase the signal 10 times, the difference is 10 dB. If you increase the signal by 100 times, then the difference is 20 dB. And if the signal increases by 1,000, you would say that the increase was 30 decibels.
One of the most obvious symptoms associated with the dB loss is no signal at all, which means you’re going to have no connectivity from a particular device. In those particular cases it’s relatively easy to troubleshoot because you know there’s no signal coming through the line.
But sometimes the problem is intermittent. Sometimes you have just enough to get the link up and running and maybe you can send some traffic across that network link. But you may find at other times that the network traffic is either not performing as you would expect or it’s not able to communicate at all.
If you are finding there’s poor performance, you should look at the statistics associated with that network interface card. And if you see a number of CRC errors or other types of errors on that connection, it may be related to a loss of signal. This is where it would be very useful to have a TDR or an OTDR. You can connect these advanced testing instruments to provide you with very detailed information of exactly how much signal you’re able to put through a particular medium.
Many of the applications we use are very sensitive to latency. Latency is the difference between the request and the response. You can see a long latency may cause problems with your application. There’s going to be a little bit of latency across the network because it takes time for the electrical signals to go down the wire and the electrical signals to come back the other direction.
So there will be a bit of delay. It’s when that delay becomes excessive that you start to have problems with your applications. So you may want to use some measurement tools to be able to understand the exact latency that you’re seeing for this application. It’s very common to use packet captures and a protocol analyzer so that you can get very specific timestamps of when you send traffic across the network and how long it takes to receive a response to those requests.
On today’s networks, we have a lot of real-time applications. We’re streaming real-time video. We have voice over IP communication. And all of this communication requires that there be a constant flow of traffic through the network. We expect these applications to send and receive traffic at regular intervals. If there happens to be delays between those intervals, you have jitter.
It’s these real-time applications that are so sensitive to any excessive jitter. That’s because, if you’re on a voice over IP phone call, you don’t have time to have information retransmitted back to you. If you don’t receive the information in time, then it’s dropped and you can’t ever rewind the conversation to begin again.
Jitter measurements are the time that we’ll see between frames. So you should see a regular interval between each of the frames that are going across the network as time proceeds. If you start to have excessive jitter between these frames, then you’ll have choppy phone calls and you’ll lose frames on your video.
It can be challenging to troubleshoot excessive jitter on your network. The first thing you might want to look at is how much bandwidth is available. If you’re using an excessive amount of bandwidth on your internet connection, then it will be challenging to receive real-time information at regular intervals.
Your switches and routers can also contribute to this jitter. We want to be sure that none of those devices are queuing up information or have excessive congestion. And we want to be sure that those devices aren’t dropping any frames as they’re coming through the network. And in many environments, you’re applying quality of service to the applications so that, if somebody is performing a very large file transfer, it won’t affect any of your real-time communication.
Crosstalk is one signal that’s going across one pair of wires is affecting the signal on another pair of wires. This leaking of information from one wire to the other causes interference. And it may affect the overall performance of a connection. One good way to measure how much crosstalk you’re having on a particular pair of wires is with a time-domain reflectometer. You may want to plug that in and get specific readings of crosstalk.
One of these readings may be Near End Crosstalk, or NEXT. This is how much crosstalk is occurring as the signal is at the near end. It’s the side that is transmitting that signal. You can also understand how much crosstalk you’re seeing as it goes through the network on the other side by measuring at the destination. That’s Far End Crosstalk, or FEXT. This means that you can look at the near end crosstalk to see how much crosstalk is introduced when the signal is at its strongest. And then you can look at the far end crosstalk to see how much crosstalk was introduced as the signal went through the cable.
There’s always going to be a little bit of crosstalk in a copper connection. But if you have excessive crosstalk values, you may want to look more at the cable. One good place to start is the crimp that you added to the end of the cable. You want to make sure that you maintain the twists as they’re going into the RJ45 connector. And if there are other types of connections along the way, patch panels, for example, you want to be sure you’re maintaining the twists on those as well.
You also might want to consider using a different cable. You could use a shielded cable that is shielding between the different pairs. Or you might want to try a category 6A cable, which increases the cable diameter. And that means that you’ll have a larger distance between those pairs.
This is why it’s very common after you’ve installed a new cable infrastructure to always go through and perform an analysis of each connection. That way, you’ll be able to find any problems with crosstalk or any other cabling issues before you plug in any other devices.
EMI is Electromagnetic Interference. Our cables are not indestructible, and we want to be sure that we’re handling them properly and we’re running them along areas where the EMI will be minimized. If you are running some new cables, you want to be sure not to twist them during the installation and minimize the amount of pulling or stretching that you would do to any set of cables.
You also want to be sure that you don’t have any sharp bends. Each cable will document the maximum bend radius allowed. You want to be sure not to extend over that bend radius. And of course, you don’t want to use staples or any type of cable ties that might crimp the wires inside of those cables.
You’ll find electromagnetic interference anywhere there’s a power source. So if you’re running your cables near electrical outlets or near fluorescent lights, you’ll find there may be an excessive amount of EMI on your ethernet network. One good way to test for EMI is to use a time-domain reflectometer and see exactly how much signal and how much noise happens to appear on that link.
A short circuit is two wires that are touching each other. So if someone has bent an ethernet cable, you may find that there is a short inside of that bend. An open is when the cable has been broken completely. There’s no signal that’s going to be able to make it through an open circuit. With an open circuit, there’s obviously no communication that can occur across that open. But if it is a bent cable that’s constantly moving, you may find there’s intermittent connectivity with that short circuit.
Because these opens and shorts may be inside the cable itself, it’s very difficult to find exactly where that may be. You may just find that moving the cable a certain way causes the intermittent connection. These are also very difficult to repair. It’s usually easier and less expensive to simply run a new cable and use that one instead of the one that has the short circuit.
This is another example of where the TDR can help you find exactly the location where this particular open or short happens to be. By simply connecting to the wire, the TDR can tell you exactly how many feet away from the TDR this particular open or short happens to be.
If you’re someone who crimps your own RJ45 connectors onto your ethernet cables, then you know it’s very easy to switch cables around and have the incorrect pin-outs on these wires. When you plug in the wire, you may find that you’re not able to go at the speeds you were expecting. Or there may be no communication at all across the wire.
It also can be difficult to visually inspect the wire to see if you really did punch things down in the correct order. That’s why it may be useful to get a cable tester like this one where you can simply plug in two sides of the tester to see exactly the pin-outs between one side and the other. And if you’re someone who hasn’t run a lot of cable or performed a lot of network termination, you may want to bring in an expert who can efficiently install your network infrastructure.
The pin-outs that we use for our ethernet networks are an international standard. They come from the EIA/TIA-568-B standard. This specifies eight conductor 100-ohm balanced twisted-pair cabling. And you’ll find that there are two popular ways of performing pin-outs on an ethernet cable. There’s the T568A and the T568B. You may even see on different punch blocks and connectors that there may be different colors that will tell you, these colors are for A, and these colors are for B.
These standards will show that most people will use 568A for horizontal cabling– that would be cabling that’s on the same floor. But many organizations have used 568B. And as long as you’ve chosen one or the other and you stay consistent, you’re not going to have a problem. If you do punch down one side of the cable with the 568A standard and the other side of the cable with 568B, you’re not going to have a straight-through cable.
Here’s visually what these two standards look like. You can see the 568A has white and green, green, white and orange, blue, white and blue, orange, white and brown, and brown as numbers 1 through 8. The only differences between the A and B are on pins 1 and 2, and 3 and 6. Notice that pins 4 and 5, and 7 and 8, are exactly the same between these two standards.
Sometimes you can look at the bottom of an ethernet connector to see exactly what standard is being used. And you would number these pins as 1 through 8 from left to right. Notice on this one we have orange as the first two. You’ve got a green on pin 3 and a green on pin 6. That means that this particular connector is using the 568B standard.
If you look at the statistics for your ethernet interface, you notice a lot of physical errors or CRC errors. You may want to check and make sure that you’re using the right kind of cable. One way to tell is to simply look at the outside of the cable. And usually, there will be information printed onto the sheath itself. You might also want to get a TDR and run your own tests on this cable and make sure the specifications of what you’re seeing on your TDR match what you’re seeing on the outside of the cable.
Here’s a larger picture of the outside of this cable from my studio. You can see this one is rated for 550 megahertz, which automatically makes me think that this is at least a category 6 cable. And if we look at the outside, it does say that it’s ETL well verified, TIA/EIA-568-B be standard. It is a UTP, or Unshielded Twisted-Pair cable. And it is a category 6-rated cable.
Some cables even have length markers on them. So you can look at one cable that’s going into a wall and then look at the cable that’s coming out of the wall on the other side and see what the differences are between those lengths. In those cases, you wouldn’t have to measure manually or get a TDR. You can simply look at the cable itself to determine how long it is.
If you’re having problems with the cable or fiber and you think there’s an issue at the physical layer, you’ll see errors appear on the interface that’s connected to that cable. For example, you can look at your interface and see if there are any frame check sequence errors. You can see if there are any oversized packets or late collisions. And those might give you an idea that there is something happening with that physical layer.
You might also look at the ethernet adapter configurations on both sides of the connection and make sure the speed and the duplex match. If you’re not communicating at all, you might also want to check the VLAN and make sure that both devices are configured to be on the same VLAN. And then it’s very common to send traffic back and forth between those devices to see if any of these physical-level errors are going to increase as more traffic is sent over the connection.
If you’re using transceivers in your network switches, you may want to check and make sure that those transceivers are matching the fiber or the connection that you’re plugging in to them. For example, if this is a fiber transceiver, then the transceiver needs to match the wavelength of the fiber.
So if you have 850-nanometer fiber, you need an 850-nanometer transceiver. You also want to check across the entire length that you’re using the correct transceivers and the correct optic fiber. If you don’t use the correct fiber or the correct transceivers, you’ll see signal loss, dropped frames, missing frames, or other physical-layer problems.
This is an easy mistake to make because transceivers all look very similar to each other. They have exactly the same format, the same connectors on the end. But if you look closely at these two transceivers, you’ll notice one is designed for a 1,310-nanometer connection. And the other is designed for 850.
When you’re running cable for a business, there are many different places where you might run into problems with the wire map. And it’s very easy to reverse the transmit and the receives at the ends of the cable when you’re putting on the RJ45 connectors. And you could also make a wiring mistake at the punchdown block itself.
The reversing of transmit and receive are easy problems to catch. You can easily see the wire map on a cable tester. And sometimes you can visually see it by examining the two ends of a cable or looking at the punchdown block itself. And some ethernet adapters will automatically adjust if they see that transmit and receive is reversed across a connection so that you’re still able to communicate, even though the wire map isn’t exactly the way it should be.
If you plug in a connection and you get no connectivity at all, you may want to look at the wire map and see if there was a reversal. If your ethernet adapter supports auto-MDIX, you may want to enable that and see if you’re able to get a signal across the wire then. If you have identified a reversal, it’s probably on the punchdown block or at the end device. So you may want to start with the patch panel and then work out from there to see if you can find where the reversal might be.
The copper cables that we use are pretty rugged. And when we run cables through a wall or over a ceiling, we don’t usually have a problem with those cables. But very often, we have cables that are used for patches from the wall that are plugging into printers and other devices. And sometimes they can be stepped on. Or a cable can be pushed between a wall and a table. And then you have shorts and opens and problems communicating.
If you do run into a cable that looks like it’s been stepped on or bent, you may want to check the physical layer and make sure that the cable is working as expected. You might also want to look at the device the cable is connecting to. Because if that cable has been pulled, it might have bent pins inside of the ethernet adapter. Sometimes it’s easy to simply replace a patch cable and see if that resolves the problem. But other times, you may need a TDR to see if there’s a short, open, or some other kind of damage inside of that cable.
Whenever someone says, the network is slow, what they’re really saying is that any one of these many devices that are plugged into this network may be having some type of problem somewhere inside of them. There’s never just one single performance metric that you need to examine. You need to look at every single step along the entire path.
This means you may need to examine the I/O bus of a server or the CPU speed. Or you may need to look at the access to a switch or examine what the router performance might be to really get an understanding of exactly the performance of the traffic going across the network.
This means that you may find yourself examining a lot of different metrics on a lot of different devices to really get an understanding of the overall health of the network. This means you’ll need to look at statistics in the server, in the routers, the switches, the networks, and the workstations.
It also helps if you know what the normal type of communication should be. If you know that there shouldn’t be any more than 100 milliseconds of difference of your database communication and you notice that suddenly you’re getting 500 to 600 milliseconds of delay, then you know the problem is somewhere with that particular communication. And by resolving that bottleneck, you’ll find that the overall performance of the application is improved.
If you have problems with the configurations of the interfaces used on your network, you may see symptoms such as poor throughput. This would be constant poor throughput through any connection that you’re using on your network. Or you may find some devices have no connectivity at all. You’re not able to see any link lights appear on the ethernet adapter. Or there may be a link light but no activity light on the ethernet adapter. All of these situations might be easily resolved by checking the interface configuration of the ethernet adapters.
On many networks, the ethernet cards are configured to automatically configure themselves when they connect to the network. They’ll determine what’s on the other end, and they’ll make sure that the configurations match on both sides. But this doesn’t work 100% of the time. So you may find that some network administrators prefer setting all of their connections up manually so they know that both sides are going to match.
The first thing you’ll want to check for is a link light. That means you at least have connectivity between your device and the switch. If there’s no light, then there is no connection. So there might be a cabling problem or an interface configuration issue. The speed of the ethernet connection also needs to match on both sides. If you have 100 megabits on one side, it needs to be 100 megabits on the other. If there’s a mismatch in those speeds, you’ll have no connectivity across the network.
A more challenging problem to troubleshoot is when the duplex is mismatched between two devices. This is when one side is configured as half duplex and the other side is configured as full duplex. You’ll have connectivity between the devices, but you’ll notice the throughput is slower than you would expect. You’ll also see an increase in the late collision counter on these devices, which might give you an indication that there is a duplex mismatch.
When you’re configuring the interfaces on your switch, you’re assigning each interface with a VLAN. And if you happen to put the wrong VLAN in for an interface, you may run into some problems. For example, a device may have a link light, which shows that it sees the switch on the other end, but it’s not able to surf the internet or connect to any other devices.
Or you may see an IP address that’s assigned via DHCP, but it’s not for the subnet that you thought you should be on. And if you configure the IP address manually, you still aren’t able to connect to the devices on the network. The best way to check for a VLAN configuration is on the switch itself. So you would SSH or connect to the switch and see what the VLAN setting is for the interface that’s connected to that device.
VLAN 1 is usually the default for a switch. But many organizations will have many different VLANs. So you may need to check your documentation to see exactly what VLAN that device should be a member of. Here’s the VLAN settings on my switch. I have everybody on this particular switch configured for VLAN 1. But if any of these interfaces showed on a different VLAN, I would need to check my documentation and make sure that’s exactly the VLAN for that device.
And lastly, let’s revisit a speed and duplex setting for an ethernet connection. We know that for ethernet we can choose between 10 megabit, 100 megabit, and 1,000 megabit, or 1 gigabit, usually on our devices. Or we could tell the device to automatically configure itself with the appropriate speed for that connection.
You might also see options for duplex. You can configure an interface for half duplex, for full duplex, or again, you can tell the interface to automatically configure the duplex based on what it’s connecting to. In many environments, the switch is configured to automatically negotiate the speed when a device is connected to it. But if there’s a problem with the wiring of the cable or the end device is manually configured for 100 megabit, you may find that it’s not the fast gigabit connection that you were expecting.
And duplex, of course, is also automatically negotiated between the switch and the device that’s connecting. And again, it needs to match on both sides of that connection. If there’s a mismatch, there’ll be a significant slowdown. Often, we’ll perform a bandwidth test to see exactly the throughput we’re getting through that connection. And that might indicate that we have a duplex mismatch.
You might also want to look at the ethernet statistics on that adapter and see if late collisions happens to be increasing. That may be another indication there is a duplex mismatch. So check the ethernet connections on both sides to see what both the speed and the duplex are set to.