MPLS Basics

Posted on August 19, 2009 by Jeff

MPLS Study Notes Terminology

Penultimate Hop Popping

RFC 3031

Multiprotcol
Transport diff payloads
-Ethernet, FRLY, ATM,IPv4, IPv6

Layer 3 MPLS VPN (RFC2547) will be the focus of the 4.0 R&S Exam.

MPLS_Terms

Label Switching

  • Labels used to route(switch)traffic instead of normal IPv4/IPv5 route.
  • Similar to how FRLY DLCI or ATM VPI/VCI works….
  • Labels are locally significant between each adjacent neighbor.
  • MPLS TE(Traffic Engineering), fast convergence…
  • SP no longer has to hold 300k routes on every router in their network.

How MPLS Works

  • Layer2 or Layer3 traffic enters SP network.
  • MPLS label is added to incoming traffic at SP edge(PE) or (LSR).
  • SP core switches traffic towards exit point using MPLS label
  • MPLS label is removed as traffic exits SP network
  • Traffic is “tunneled” from SP entry to SP exit since SP core does not inspect payload.

How Labels Work

Each label represents a unique IP prefix

  • Label to IP binding is called a Forwarding Equivalence Class(FEC)

Labels are dynamically advertised using a Label Distribution Protocol. LDP and TDP use IS-IS or OSPF.

  • Tag Distribution Protocol(TDP)
  • >>  Legacy and Cisco Proprietary
  • Label Distribution Protocol(LDP)…Most common(TCP based)
  • >>  Open Standard
  • BGP
  • >>  Multiprotocol Extensions…MP-BGP or MBGP
  • Resource Reservation Protocol(RSVP)
  • >>  Used for MPLS TE

We are going to look at LDP and BGP for the 4.0 Exam.

How Switching Works
Label + Prefix binding forms Label Forwarding Information Base (LFIB)
When traffic is received, LFIB is consulted in order to perform one of 3 operations

  • Label push(ADD)
    Add a label to an incoming packet
    AKA label imposition
  • Label swap(Change)
    Replace the label on an incoming packet
  • Label pop(Remove)
    Remove the label from an outgoing packet
    AKA label disposition

MPLS Terms
Customer Edge (CE) Router

  • CPE equip that typically does not run MPLS
  • Can be layer 2 only or layer 3 aware

Provider Edge (PE) Router

  • Receives traffic from Customer Edge(CE) devices, adds MPLS label(push), and forwards into core
  • Receives traffic from core, removes MPLS labels(pop), and forwards towards Customer Edge(CE) devices.

Provider (P) Router

  • Connects only to PE’s and other P routers
  • Switches traffic between interfaces based on MPLS labels(swap)
Posted in MPLS, Networking, Routing | Tagged , , , , | Leave a comment

High CPU on a Catalyst switch running IOS

Posted on July 30, 2009 by Jeff

This is the troubleshooting process you can take to solve High CPU problems in your network.  The root cause is always something different, but the steps are mostly the same.

High CPU since Market open….

C6500#show ver

System image file is “disk0:s72033-advipservicesk9_wan-mz.122-33.SXH.bin”

=-=

Below we see 77% Total CPU with 33% from interrupt traffic, and ~ 40% from IP Input.

C6500#show proc cpu | exc 0.00

CPU utilization for five seconds: 77%/33%; one minute: 75%; five minutes: 77%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

5    58144468   3708230      15679  0.87%  0.30%  0.29%   0 Check heaps

140     1850756  45849063         40  0.07%  0.11%  0.13%   0 CDP Protocol

146   106922548 674564539        158 40.09% 41.10% 41.95%   0 IP Input

168       15712     17711        887  0.23%  0.03%  0.22%   1 SSH Process

335   136091476 995454760        136  1.11%  0.51%  0.45%   0 Port manager per

374    18563992 190871473         97  0.15%  0.29%  0.28%   0 IGMP Input

376    11477444 194064823         59  0.15%  0.19%  0.18%   0 PIM Process

377      114620 192409350          0  0.15%  0.06%  0.06%   0 Mwheel Process

C6500#

=-=

Next we cleared the counters and then look at the vlan interfaces to see who has the most input queue drops.  Vlan 10, and vlan 200 seem to be getting hit the hardest.

C6500#show int | inc is up|drop

Vlan10 is up, line protocol is up

Input queue: 16/75/416/416 (size/max/drops/flushes); Total output drops: 0

Vlan200 is up, line protocol is up

Input queue: 4/75/565/565 (size/max/drops/flushes); Total output drops: 0

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

Loopback0 is up, line protocol is up

Next, we dump the buffers to see what kind of traffic is hitting the buffers of vlan 10, and 200.  We see that it is all multicast traffic.

C6500#show buffers input-interface vlan 10 packet | inc source:

source: 10.5.1.54, destination: 239.248.10.134, id: 0x0000, ttl: 15,

source: 10.5.1.54, destination: 239.248.10.134, id: 0x0000, ttl: 15,

source: 10.5.1.78, destination: 239.248.10.54, id: 0x0000, ttl: 15,

source: 10.5.1.78, destination: 239.248.10.55, id: 0x0000, ttl: 15,

source: 10.5.1.54, destination: 239.248.10.132, id: 0x0000, ttl: 15,

source: 10.5.1.78, destination: 239.248.10.55, id: 0x0000, ttl: 15,

=-=

C6500#show buffers input-interface vlan 200 packet | inc source:

source: 10.5.200.103, destination: 239.248.10.175, id: 0x0000, ttl: 15,

source: 10.5.200.108, destination: 239.248.10.145, id: 0x0000, ttl: 15,

source: 10.5.200.112, destination: 239.248.10.224, id: 0x0000, ttl: 15,

source: 10.5.200.103, destination: 239.248.10.175, id: 0x0000, ttl: 15,

source: 10.5.200.108, destination: 239.248.10.146, id: 0x0000, ttl: 15,

source: 10.5.200.113, destination: 239.248.10.94, id: 0x0000, ttl: 15,

=-=

So, we focus on 1 multicast stream to see why it would be getting punted to the CPU for processing.  We look at the mroute table and see many of the multicast routes in “Registering, Partial-SC”.  This indicates that the DR is trying to register to the rendezvous point(RP), but process is not completing.

C6500#show ip mroute 239.248.10.134

IP Multicast Routing Table

Flags: D – Dense, S – Sparse, B – Bidir Group, s – SSM Group, C – Connected,

L – Local, P – Pruned, R – RP-bit set, F – Register flag,

T – SPT-bit set, J – Join SPT, M – MSDP created entry,

X – Proxy Join Timer Running, A – Candidate for MSDP Advertisement,

U – URD, I – Received Source Specific Host Report,

Z – Multicast Tunnel, z – MDT-data group sender,

Y – Joined MDT-data group, y – Sending to MDT-data group

V – RD & Vector, v – Vector

Outgoing interface flags: H – Hardware switched, A – Assert winner

Timers: Uptime/Expires

Interface state: Interface, Next-Hop or VCD, State/Mode

(*, 239.248.10.134), 02:06:47/stopped, RP 10.7.240.240, flags: SJCF

Incoming interface: Vlan6, RPF nbr 10.5.20.5, Partial-SC

Outgoing interface list:

Vlan10, Forward/Sparse, 01:44:18/00:02:48, H

(10.5.1.54, 239.248.10.134), 01:55:47/00:02:59, flags: PFT

Incoming interface: Vlan10, RPF nbr 0.0.0.0, Registering, Partial-SC

Outgoing interface list: Null

C6500#

=-=

So we look at the RP information and see several static RP statements.

C6500#show run | inc ip pim rp

ip pim rp-address 10.7.240.240 <—may not be needed

ip pim rp-address 198.140.52.4 AAAA

ip pim rp-address 198.140.52.3 BBBB

ip pim rp-address 198.140.52.1 CCCC

ip pim rp-address 198.140.52.2 DDDD

ip pim rp-address 198.140.33.5 EEEE

ip pim rp-address 198.140.33.2 FFFF

=-=

We set up a tempory rate-limit for the partial-SC packets hitting the cpu to only allow 10 per second(non-intrusive).  With the Rate-limiter in place, the CPU is now in the 10-20% range, with is inline with the 72hour historical average.  Customer will look into removing the invalid RP config.

C6500 (config)#mls rate-limit multicast ipv4 partial 10

C6500 (config)#do show proc cpu | exc 0.00

CPU utilization for five seconds: 13%/7%; one minute: 67%; five minutes: 71%

PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process

146   107426044 674852622        159  3.67% 36.28% 38.19%   0 IP Input

168       20684     22373        924  0.15%  0.14%  0.35%   1 SSH Process

335   136098248 995490856        136  1.43%  0.50%  0.51%   0 Port manager per

374    18567412 190884074         97  0.23%  0.26%  0.28%   0 IGMP Input

386    41108136 139317998        295  0.07%  0.13%  0.12%   0 SNMP ENGINE

C6500 (config)#

Hope this helps!

Posted in Switching | Tagged , , | 5 Comments

Beaufort, NC

Posted on July 27, 2009 by Jeff

This past week my wife and  I took a short 4 day vacation to the NC coast to a little town called Beaufort.  We had a great time.  Beaufort is a wonderful place to vacation whether you want to just relax, do something active, or just eat great food.

Pecan Tree Inn Bed & Breakfast

Pecan Tree Inn

Pecan Tree Inn

Located in the heart of the Beaufort Historic District and just a few steps from the scenic yacht harbor, the Pecan Tree Inn is known throughout the Southeast as the place to stay when visiting the Carolina coast and southern Outer Banks.

We enjoyed our stay here and would definitely come back.  The Pecan Tree Inn sits on a lot deeded to the Franklin Masonic Lodge in 1866 by Rebecca Piggott.  The rooms were very spacious and adequate.  Breakfast was light continental style, muffins, hard-boiled eggs, fruit, etc..

Restuarants

There are quite a few restaurants in Beaufort worth visiting.  The seafood is always fresh and most are overlooking the harbor and the occasional wild horses on the beach on the other side of the harbor.

Our favorites:

Spouter Inn Restaurant and Bakery

Spouter Inn

Spouter Inn

The Spouter Inn had the best Crab Dip I have ever tasted.  We also had fresh broiled shrimp seasoned with the traditional Old Bay Seasoning.  Dinner on the deck overlooking the water at sunset is hard to beat.  Dinner was so good, we came back for lunch the next day.

Beaufort Grocery Co

Beaufort Grocery Co.

Beaufort Grocery Co.

A little pricier, but well worth it.  This was another place where we came back the next day for lunch.  Can’t go wrong with shrimp, scallops, clams, over pasta.  The Pecan Pie was great.

Things to do:

Fort Macon

Great place for all the history buffs.  The entire Fort has been restored over the years.  There are numerous displays and recreations of what life was like in the fort during war times.  Lots of interesting history here.  Many artifacts, and very interesting architecture as well.

IMG00211IMG00202IMG00201

Shackleford Banks Wild Horses

Take a hike on the Shackleford banks and see the oldest documented wild horses in country.

Indoors

If you need a place to cool off from the summer heat or if the weather turns for the worse, check out the NC Aquarium at Pine Knoll Shore, 20 minutes away or the NC Maritime Museum right on the waterfront in Beaufort.  Enjoy!

Posted in Just for fun | Tagged , , , | Leave a comment

Marathon Training Week 9

Posted on July 25, 2009 by Jeff

So I am in my ninth week of training for the Marine Corps Marathon. I just hopped on the scale and was pleasantly surprised to see that I have dropped nearly 10 pounds!!  Woohoo!  I have had little to no changes in diet so far.

scale

I started out at 188 LBS on week 1.  So you CAN lose significant weight just exercising 4 times a week.  I have not made any changes to my diet.  For the next phase of my training I will need to make some diet changes and quit eating the larger portions when we go out to eat.  My goal is to get back down to a “playing weight” of 170 pounds for the Marathon in October.

Posted in Running | Tagged , , | Leave a comment

So you want to Build your own CCIE Lab Rack

Posted on July 13, 2009 by Jeff

Building your own CCIE Lab Rack can be a daunting task if you’re not sure how to get started. Hopefully this will provide you a blueprint of what you will need and how to set up correctly so you can spend more time doing Labs and less time trying to figure out how to build the Lab Rack.

Hardware needed for CCIE Lab Rack
2 x 3550 Catalyst Switch $400
2 x 3560 Catalyst Switch $1300
3 x 2501 Router $75 (BB’s)
1 x 2511 Router $100 – Terminal Server
5 x 2611 Router  $100 (or Cisco 2611XM $150, or Cisco 3640’s $150)
2 x 4500 Router $50 (1 is your FRLY Switch, )
——————————
~$4500

1. Ebay is your friend. If you don’t have an account, definitely get one.
2. Set up a PayPal account.

Once you pass your CCIE Lab you can sell everything that you previously bought on Ebay.  Think of this like a college loan.

18 x Crossover Cables (Interswitch connections)
12 x Straight-Through Cables (Switch to Router connections)

CablesToGo for your Serial cables if you can’t find some for free.
12 Serial Cables
2 Octal Cables

1 or 2 WIC module for each router depending:
Cisco WIC-1T ($40)
Cisco WIC-2T ($50)
CENTRECOM 210T TWISTED PAIR TRANSCEIVER($10 or free)

Cisco 2600 Series Memory Upgrade

You will need 64 MB of DRAM and 16 MB of system Flash on Cisco 2611’s.  Cisco 2611XM’s can be upgraded to 256MB of DRAM.

Posted in How To, Networking, Routing, Switching | Tagged , , | 8 Comments

Setting up a TCL(pronounced as “tickle”) script for CCIE Lab troubleshooting

Posted on July 7, 2009 by Jeff

TCL shell is available on Cisco IOS release 12.3(2)T and later.  So, you can run this on your routers but NOT on your switches.  This is a great tool, especially during Lab/simulation.  You will want to verify connectivity on your Lab network constantly.

So open up Notepad and copy/paste the ip address of every device/interface that you would like to ping. Alt + highlight with your mouse will allow you to copy in columns instead of the rows&columns.

The syntax for the TCL script is as follows:

R1#tclsh

R1(tcl)#foreach address {

1.1.2.166
1.1.2.167
1.1.2.164
1.1.2.165
1.1.2.170
1.1.2.171
1.1.2.168
1.1.2.169
1.1.2.174

} {ping $address}

You can copy/paste the entire output from your notepad into your telnet session. Hit <enter> after the last curly bracket “}” and the script is fired off very fast.

Here is Screencast of the TCL script in action.

Hope this helps!

Posted in Networking, Routing | Tagged | 1 Comment

Keeping a Training Log for your Marathon Training

Posted on July 6, 2009 by Jeff

I’ve been using Training logs for as long as I can remember.  They are a great way to keep you motivated, measure your progress, or just to stay organized. There’s really no correct way to set up a training log, as it really just depends on how much information you want to track.  Some people are hard-core and like to track, nutrition, sleep, heart rate, weight, and pretty much anything you can think of.  I like to track my distances and my heart rate, but usually don’t worry about the rest.

When I was training for the Florida Ironman in 2000 it was important for me to track how much of swimming/biking/running I was doing each day.  I would simply do the following for each workout day.

S – Swimming

B – Biking

R – Running

So, on my calendar I would just pencil in the distance for each day.

-Monday-

S – 2000yds

B – 40 miles

R – 6 miles

At the end of each 1 week period, I would add up the totals.  This really helps you focus from week to week to see where your weak areas are and how to address them moving forward.  After each race or key date in your training, you’ll want to look back to see what works and what areas need improvement.

There are numerous free online resources for training logs.  Some allow you to plot your workout route if you’re not sure about the distance.

Go here for free Calendars

I currently use Active Trainer‘s Online Training Log(also free).

Check out some of my training routes & Training log “here” if you like.

Also check out other online training logs:
MapMyRun.com
RoadRunnerSports.com

Posted in Running | Tagged , | Leave a comment

How to make your own Ethernet Network cables

Posted on June 29, 2009 by Jeff

If you are planning to build your own CCIE Lab rack, consider making your own network cables.  Making your own network cables is a good way to save money if you need to make a lot of them.  At $7 for a 3ft patch cable, you can spend quite a bit of money if you need, say 30 cables.

For a CCIE Lab rack, you will need the following:

12 x straight-thru cables
18 x crossover cables
—
30 cables x 3 ft = 90 ft of Cat5 cable needed.  You can buy 100ft for $50 or less.

2  x Boxes(of 50) RJ-45 connectors $2-3

Wire cutter/Stripper and Crimping tool $20

See pinouts below:

Demonstration:

Hope this helps!

Posted in Networking, Routing, Switching | Leave a comment

The Bonk

Posted on June 25, 2009 by Jeff

At some point in every runners life you will experience what can only be described as a Bonk!  This is simply “a condition in endurance sports where depletion of glycogen stores in the liver and muscles results in sudden and severe fatigue.”  So this begs the question; “How to avoid bonking” or “What should I eat during a Marathon or long run”.

First, If you’re on the fence and not sure if your stomach will be able to handle nutrition during a Marathon or long run, take this opportunity to try something new and just experiment.

Go to your local GNC, Fleet Feet, or Inside-Out Sports and just buy several different nutritional “Gels”.  They come in all kinds of flavors, so you’ll just have to experiment and see what works best for you on a hot thirsty day.   Examples of ones that I have tried in the past are GU, Power Bar Gel, and Clif Shot.  The ingredients of these are all quite similar in that they will have about 25-30 grams of carbohydrates, with additional Sodium and Potassium.  The carbohydrates help replenish your glycogen stores, and the Sodium and Potassium help battle fatigue and muscle cramping.

How often do I need nutritional gels?

A good rule of thumb is to have 1 Gel packet every 45 – 60 minutes during a Marathon or long run.  I find it easier to remember to take it every 6 miles as the mile marker aid stations approach.  So at mile 6, 12, 18, 24, as I see the aid station approaching, I take 1 GU and then wash it down with water from the aid station. Bring an extra just in case you drop one or need another.

Then keep on running!

Posted in Running | Tagged , , | Leave a comment

Marathon Training Schedule for busy people

Posted on June 23, 2009 by Jeff

If you’re like most people, you find it hard to find time to train during the week, with the demands from work, life, family, etc.  Once you have committed to running a marathon, you need to tell people about it.  This puts you on the hook to make sure you have some accountability and support from your friends.  This will also help others understand if you can’t make it to every “event” that pops up.

The most important part of your training is your long run, so this is where you should concentrate your efforts.  The long run should only be done once per week to avoid over training.  Subsequent runs during the week are to maintain your fitness level, but I consider this more like “filler mileage”, though still very important.

So during the weekdays, all you need to do is run 3 x 4 miles.  This is not set in stone.  There is no need to be rigid about this; just make sure you run 3 out of 5 weekdays.

Ex:  Mon, Wed, and Fri: Run 4 miles

For the long runs, I use the following training schedule:

Long Run Marathon Training Schedule

Week   Long Run

1          6

2          8

3          6

4          10

5          8

6          12

7          8

8          14

9          8

10        16

11        9

12        10

13        18

14        10

15        20

16        11

17        22

18        12

19        12

20        24

21 & 22           taper

As you can see, this is a series of steps, where you gradually build up, and then take break.  Then continue to build up.  I have used this approach several times in the past as it has worked well for me.   I hope this is useful to anyone wanting to complete their first marathon.

Posted in Running | Tagged , , | 1 Comment