July Linux Labbing – Part 1

I had a few home lab projects I wanted to accomplish this weekend, and they required me to learn a few things in Linux which I hadn’t done before:

  • Create an NFS share on one device
    • Required downloading and configuring the NFS services
  • Mount the previously-created NFS share on another device
  • Mount an external USB hard drive
  • Modify fstab so that this mounting / mapping would take place automatically at every startup
  • Synchronize the contents of one folder into another
  • Display the size of a directory structure, recursively
  • Display CPU details of a device

I plan to make another entry soon covering these steps….

Advertisement

You Can Do a ‘show CDP neighbor’ on End Hosts — Even ESXi

If you’re from the Cisco world, no doubt you make regular use of the ‘sh cdp neighbor’ command to learn about how your network devices are physically connected – especially when learning about a new network with questionable documentation or physical labeling (though that never happens, right?).

Most network folks are also quite familiar with correlating mac address tables with ARP tables to DNS entries to find out exactly where end hosts are attached.  But what if you have to approach the problem from another perspective?  Said another way, how would someone at an end-host view the CDP data from an attached Cisco switch?

(I realize that many shops prefer to disable CDP on host-facing ports – especially those connected to equipment under the control of another entity.  For the sake of this post, we’re going to suppose that CDP is enabled on the ports under discussion.)

We need to deconstruct CDP a bit (there is no magic, right?).  CDP (as well as VTP) messages are sent in frames with an L2 multicast destination address of : 01-00-0c-cc-cc-cc.  Depending on the options in your capture tool, you may be able to filter by the LLC protocol ID of CDP, which is 0x2000.

Knowing this, we just need to run a capture filtering for frames with that destination address.  This can be done in various ways depending on your OS and the capture tool being used – below are a few examples.

Linux (running on a Raspberry Pi here)
tcpdump -nn -v -i eth0 -s 1500 -c 1 ‘ether host 01:00:0c:cc:cc:cc’
or
tcpdump -nn -v -i eth0 -s 1500 -c 1 ‘ether[20:2] == 0x2000’

cdp-info-from-a-raspberry-pi-using-dst-mac

Wireshark (running on a Windows system in this example)
Start capture on the relevant interface, then enter ‘cdp’ in the filter field

wireshark-cdp-example

CDP4Win
This is a handy tool for those who may not be comfortable with tcpdump or Wireshark, such as field techs who just need to quickly find port info.

  • cdp-4-win-example

ESXi
12/26/2016 edit:
While the method I presented below is valid, it’s a bit low-level.  It turns out that there is native support in ESXi for viewing Cisco CDP info, as explained in the following VMware KB article: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007069

Original Entry Follows…
This gets a little trickier, as there isn’t an easy way to display the output of a capture at the ESXi CLI (or I just haven’t figured it out – will freely admit that I am a relative beginner in the ways of VMware).

  • SSH to the ESXi host (you may need to enable ssh for the host if it isn’t already enabled)
  • To save output to a file (in pcap format):
    • [root@localhost:/tmp] pktcap-uw –uplink vmnic0 –mac 01:00:0c:cc:cc:cc -o /tmp/cdptrace.pcap
      • Then transfer the file to whatever system / tool you like to use to read pcaps
  • To display output on screen (will be in hex) use the following command:
    • [root@localhost:/tmp] pktcap-uw –uplink vmnic0 –mac 01:00:0c:cc:cc:cc
      0x0000: 0100 0ccc cccc 0019 aab0 af02 01ce aaaa
      0x0010: 0300 000c 2000 02b4 1d99 0001 001b 5377
      0x0020: 6974 6368 6572 6f6f 2e6e 6963 6b6f 6c75
      0x0030: 732e 636f 6d00 0500 fc43 6973 636f 2049
      0x0040: 4f53 2053 6f66 7477 6172 652c 2043 3335
      0x0050: 3630 2053 6f66 7477 6172 6520 2843 3335
      0x0060: 3630 2d49 5053 4552 5649 4345 534b 392d
      0x0070: 4d29 2c20 5665 7273 696f 6e20 3132 2e32
      0x0080: 2835 3529 5345 3130 2c20 5245 4c45 4153
      0x0090: 4520 534f 4654 5741 5245 2028 6663 3229
      0x00a0: 0a54 6563 686e 6963 616c 2053 7570 706f
      0x00b0: 7274 3a20 6874 7470 3a2f 2f77 7777 2e63
      0x00c0: 6973 636f 2e63 6f6d 2f74 6563 6873 7570
      0x00d0: 706f 7274 0a43 6f70 7972 6967 6874 2028
      0x00e0: 6329 2031 3938 362d 3230 3135 2062 7920
      0x00f0: 4369 7363 6f20 5379 7374 656d 732c 2049
      0x0100: 6e63 2e0a 436f 6d70 696c 6564 2057 6564
      0x0110: 2031 312d 4665 622d 3135 2031 313a 3334
      0x0120: 2062 7920 7072 6f64 5f72 656c 5f74 6561
      0x0130: 6d00 0600 1863 6973 636f 2057 532d 4333
      0x0140: 3536 3047 2d34 3850 5300 0200 1100 0000
      0x0150: 0101 01cc 0004 c0a8 0102 0003 0016 4769
      0x0160: 6761 6269 7445 7468 6572 6e65 7430 2f32
      0x0170: 0004 0008 0000 0028 0008 0024 0000 0c01
      0x0180: 1200 0000 00ff ffff ff01 0221 ff00 0000
      0x0190: 0000 0000 19aa b0af 00ff 0000 0009 000a
      0x01a0: 6c6f 6f70 6572 000a 0006 0001 000b 0005
      0x01b0: 0100 1200 0500 0013 0005 0000 1600 1100
      0x01c0: 0000 0101 01cc 0004 c0a8 0102 001a 0010
      0x01d0: 0000 0001 0000 0000 ffff ffff
  • Converting Hex to ASCII:
  • hex-to-ascii-cdp-conversion
  • If someone knows how to cleanly display the output of the capture at the ESXi CLI just like can be done with tcpdump, please share!

Palo Alto Firewall
This example only pulls data from the FW mgmt interface, but a similar procedure is possible from data plane ports…though it goes without saying that care should be used when taking packet captures in any production environment.

# let capture run for about 30 seconds
tcpdump snaplen 1500 filter ‘ether[20:2] == 0x2000’

# view pcap on screen
view-pcap verbose++ yes mgmt-pcap mgmt.pcap
# or alternately, tftp it somewhere

That’s about it!  I hope this comes in handy for folks.

Background
Shout-out to Derek Parsons, one of my old colleagues, who had the idea to run the tcpdump command from Redhat Linux devices while we were doing a network migration. Saved us a ton of time!

CDP4Win – found this tool during a quick Google search when some field support colleagues said that a Fluke test device was way too expensive and they needed a cheap, easy way to pull CDP info using only a laptop.

Palo Alto method – referenced Palo Alto support documentation for packet captures and just plugged in the filters that worked in Linux.

ESXi CLI method – figured it out by referencing VMware documentation and combining the methods that had already worked on other platforms.

References:
https://en.wikipedia.org/wiki/Cisco_Discovery_Protocol
https://wiki.wireshark.org/CDP
https://sourceforge.net/projects/cdp4win/files/cdp4win/
https://www.wireshark.org/docs/dfref/c/cdp.html
https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2051814

Troubleshooting WiFi @ Windows CLI w/ Netsh Command – Part 2

In the previous post, I introduced the netsh wlan command and shared a sample batch file that can be used to run the command with the ‘show all’ options and direct the output to a text file.

In this post I’ll go through some of the more useful bits of info the command can provide.

So why would a network engineer care about something boring like the wireless NIC driver version running on a windows laptop?  Well, it turns out that the racks & racks of super expensive gear we work on every day exist for a reason beyond our own amusement.

Few people in a modern IT organization would dispute the need to install OS patches regularly to stay ahead of reported bugs and vulnerabilities & to occasionally enable new features.  Yet when dealing with drivers for client hardware, many people take the ‘if it’s not broke don’t fix it’ attitude.

Many critical aspects of wireless network performance depend on the client, including roaming logic and band selection.  Updated drivers & release notes can be found at the website of the NIC vendor.

netsh-wlan-show-drivers

With the wireless chipset on my laptop, I noticed that in an earlier version of the driver, only 802.11 a/b/g was supported.  An update was required to unlock 802.11n support.

Other bits of info produced by this handy command (specifically ‘netsh wlan show interfaces’) include details on the network to which the system is currently connected.  In the example below, we can see that the laptop is connected to the gogoinflight SSID via the 5GHz flavor of 802.11n.

netsh-wlan-show-interfaces

 

In a professional wireless deployment, a given SSID is provided by multiple APs & radios.  In the output of the ‘netsh wlan networks mode=bssid’ command, we see that gogoinflight is visible on 6 unique radios – 3 in the 2.4 GHz band and 3 in the 5 GHz band.

netsh-wlan-show-networks

 

Several more variants of this command are possible, each yielding different info, as shown by the use of the ? option.

netsh-wlan-show-questionPlay around with this command on your own to get a sense of the useful info it can provide.

Troubleshooting WiFi @ Windows CLI w/ Netsh Command – Part 1

Our end users may complain about “slow WiFi”, but as network engineers we know that an interconnected system of devices, links, and applications all work together to provide wireless data services.  From applications running on end hosts to device drivers to wireless radios to APs to cabling to access, distro, and core switches, to authentication systems, certificate authorities, DHCP & DNS servers, WAN accelerators, routers, backhaul services, to content providers — and of course the wireless medium itself, it can be tricky to pinpoint the source of user pain.

In this post, I’d like to focus on a handy diagnostic tool available at the Windows CLI: the ‘netsh wlan’ commands.  I’ll cover some of the options I’ve found most useful, but by all means explore more for yourself using the ? help option.

Before we delve into some details, I’d like to share a way to get a variety of data with one quick command: ‘netsh wlan show all’. Depending on your environment, the amount of output may exceed the command prompt window scroll buffer.  For this reason, it’s useful to pipe output directly to a text file, as shown in the example below:

echo off
set SAVESTAMP=%DATE:/=-%@%TIME::=-%
set SAVESTAMP=%SAVESTAMP: =%
netsh wlan show all > %USERPROFILE%\desktop\WiFiDiag_%SAVESTAMP%.txt
route print >> %USERPROFILE%\desktop\WiFiDiag_%SAVESTAMP%.txt

When troubleshooting client issues, one handy option is to drop these commands in a .bat file and send it directly to an end user.  They can run the bat file and email back the results.  Note that some corporate email systems flag .bat files as malicious attachments and remove them from emails.  You can usually get around this by simply giving the file a .txt extension.  Give the user a few simple bullets explaining how to save, rename, and run the file.

Credit for this idea goes to several of my colleagues at a previous gig (Adept).  I made a couple changes to the file naming logic per my own preferences, but that’s about it.

If all goes as planned, the command should plop a tidy, labeled text file right on your desktop.

wifidiag-filename

In Part 2, we’ll look at some of the output.

PlayStation 3 & High Data Utilization When Idle

Have folks noticed their PS3 or other game console using a lot of data or otherwise behaving weirdly on the network when it was, as far as you knew, idle?

I powered on my PS3 to charge my phone, and about 24 hours later, this was a summary of its usage.  It was just sitting at the opening screen where game and movie promos roll by.  Not downloading any updates, etc.


I did some close monitoring for a few minutes this evening, and saw a few of the sites that were reached out to:

Has anyone analyzed PS3 network performance before?
Also, I got all this data from my new SOPHOS UTM Virtual Firewall – I love it!!  Hope to do a few posts just on it.

Gear Overview

After a typical arduous week of work, I don’t have the mental energy to compose engaging prose on some interesting technical topic.  However, in order to meet my new goal to just publish something regularly, quality-be-darned, I thought I’d share a list of some of the gear I use regularly.  Here’s hoping it will help build blogging momentum and lead to many future posts!

This list isn’t exhaustive and doesn’t include most of what I touch in my day job.  However, it’s what allows me to learn, try out new configs, stream Netflix & Hulu, etc.

  • Cisco 871 Integrated Services Router
  • Cisco Catalyst 3560G-48PS Switch
  • 2 x Cisco 1142N Lightweight Access Points
  • Intel NUC with 5th Generation Intel Core i7-5557U processor
    • 16 GB RAM, expandable to 32 GB
    • 2 x 500 GB Samsung EVO Solid-State Drives
  • MediaSonic USB 3.0 drive enclosure with 4 x 2 TB HDDs
  • vmWare ESXi 6.0
img_3860

Intel NUC ESXi Host & MediaSonic Drive Enclosure

Cisco Virtual Wireless Controller Running AireOS 8.0.135vwlc-login

Cisco Prime Infrastructure 3.1 Virtual Applianceprime-3_1


Some work gear: standard boring corporate laptop with far-less-boring AirPcapNx (kudos to colleague Mike for hooking me up with the AirPcap)

img_3850

AirPcap Nx w/ Laptop

There are No Perfect Circumstances

I’ve had many ideas for potential blog posts over the last year, but some obstacle always came up: maybe the idea needed to be polished a bit more, or I had to get more work done, or just completely lacked motivation.  As with many worthwhile pursuits in life, though, I realized that perfect circumstances rarely exist.  Sometimes you have to force yourself to get started and do something.  Who knows – those efforts might just pay off.

CWNA Unlocked & a New Gig

A lot has happened since my last (first) post: CWNA cert obtained, job end, job search, new job (return to the old).

The CWNA is by far the best certification experience I’ve had.  From the well-written study guide by David D. Coleman & David A. Westcott to the exam itself, the materials were mentally engaging and seemed worth the significant time commitment required.  Rather than require you to burn a lot of mental cycles remembering CLI commands or GUI clicks, the CWNA guide teaches the actual underlying technologies that make WiFi work – from the physics around wave propagation to modulation & coding schemes to authentication methods & everything in between.  The book is written in such a way that you feel the authors are personally explaining the material, even putting in humor at times.  The material was not light, yet reading the book felt more like a conversation with a knowledgeable colleague than trudging through a technical manual.  After reading the book (and re-reading many sections!), reading the glossary, going through the end-of-chapter knowledge-checks, and lots of flash card making and reviewing, I took the exam in mid-January 2016 and passed.  Always a great feeling!

Along with this up came a bit of a down…my job as a Wireless Support Engineer would come to an end on January 31, 2016, as my employer lost the contract to provide wireless services to their main client.  It was an enjoyable, if brief, experience.  On the plus side, I had the opportunity to work with some awesome folks who taught me some handy skills, particularly about the Cisco WLC CLI.  My previous shop had been GUI-only, whereas this group heavily emphasized the CLI.  I also learned a new way to implement FlexConnect in a large enterprise, techniques for better managing the finite spectrum available to us (RF Profiles are da bomb!), and sooo much client troubleshooting.  This gig really drove home the importance of never thinking you’re all that in some area, as there’s always more to learn.  Great group and I hope I get the chance to work with some of them again in the future.

With another gig done, it was time for a scheduled septoplasty & turbinate reduction surgery…not going into the details here, but suffice to say it wasn’t very fun.  On the bright side though, my wife and I were sick of winter, and with a little time on our hands decided to spend a few weeks in the Caribbean – one week in Punta Cana, DR, and a week in Aruba.  This was the first time I’d had this much time off from work since college, and was definitely a welcome break.

With March 2016 came a new gig with new challenges and opportunities…more to come…

 

CWNA Certification Trek Begins…

Well here goes…I’ve started on the journey towards becoming a Certified Wireless Network Administrator (CWNA).

I’ve been working with enterprise wireless networks in some form or another for the last 5 years, and obtained my CCNA-Wireless last year.  While the CCNA-Wireless curriculum certainly provided good wireless theory, there was a lot of memorization of Cisco-specific tasks.  Some test questions even asked about which tabs you would click on in a Cisco wireless controller GUI to find certain options!

As I’ve started to follow and interact with folks who are active in the wireless networking community, I’ve found that a true wireless professional needs to have a solid understanding of the underlying wireless theory, including solutions from multiple vendors.  Along those lines, many people recommend the vendor-neutral CWNA certification, along with other certs from the CWNP organization.

With this blog, I hope to be able to share some of the things I’m learning with others in the community, and at the same time motiviate myself to keep learning!