NTP and Cisco Time - Part 2 Configuring a Cisco Device as a Time Server

Explaining Clock Stratum

NTP uses a hierarchical structure to refer to different levels of NTP servers or time sources these levels are referred to as a “stratum”.  The stratum number indicates your distance from the time source or reference clock, which is called level 0. 

Logically if you are at stratum level 6 you are referencing a level 5 stratum time source, level 0 to 15 stratum are considered valid, level 16 is considered invalid.

Explaining Leap Seconds:

Even though atomic clocks are very accurate they drift away from solar time.  This means that over a period of time UTC drifts from solar time.  In order to compensate for this a second is added every now and then to compensate.  

“Since the adoption of this system in 1972, firstly due to the initial choice of the value of the second (1/86400 mean solar day of the year 1900) and secondly to the general slowing down of the Earth's rotation, it has been necessary to add 21s to UTC.”  Directly from the GMT website

If you are synchronising to an external stratum 1 clock then this is done for you, however if you are building your own NTP architecture then you can use the following command to add your own leap

Configuring a Cisco Router as an NTP Server

Plan:

• 1Configure master 1 as a server
•  Configure NTP Client 2 as a client
• Configure Master 2 as a secure server
•  Configure Client 2 to authenticate against Master 2 and prefer it.

Requirements:

3 x 3725 routers or equivalents with an IOS that supports the following commands:

I have attached my GNS3 files here:

https://drive.google.com/file/d/0B_pDrKQiRFqacHF6RHF4QnFDem8/edit?usp=sharing

Network Diagram:

Step 1: Basic Configuration

Configure each device according to the following table:

NTP_MASTER_1	FA 0/0	10.10.10.1 /24
NTP_MASTER_2	FA 0/0	10.10.10.2 /24
NTP_CLIENT_1	FA 0/0	10.10.10.3 /24

From NTP_CLIENT_1 ping both NTP servers to confirm connectivity.

STEP 2: Configure NTP_MASTER_1 as the NTP Server

In order for a cisco device to configure itself as an authoritative source for NTP clients you utilise the ntp master command.

ntp master [stratum]

The stratum option refers to the table above and is optional.

Issue this command on NTP_MASTER_1

NTP_MASTER_1(config)#ntp master

NTP_MASTER_1(config)#exit

NTP_MASTER_1#show ntp status

Clock is synchronized, stratum 8, reference is 127.127.7.1

nominal freq is 250.0000 Hz, actual freq is 250.0000 Hz, precision is 2**18

reference time is C0294404.7AE29B59 (00:04:20.480 UTC Fri Mar 1 2002)

clock offset is 0.0000 msec, root delay is 0.00 msec

root dispersion is 1875.02 msec, peer dispersion is 1875.02 msec

NTP_MASTER_1#show ntp associations

      address         ref clock     st  when  poll reach  delay  offset    disp

*~127.127.7.1      127.127.7.1       7     3    64  377     0.0    0.00     0.0

 * master (synced), # master (unsynced), + selected, - candidate, ~ configured

NTP_MASTER_1#

Note: NTP_MASTER_1 is synchronised to itself by using a local loopback address of 127.127.7.1. 

Note: for all of you Microsoft students who have grown up to believe that 127.0.0.1 is the local loopback address the truth is that any address in the 127 range can be used as above.  If you don’t believe me try pinging any address that starts with 127 from the cmd prompt you will get a reply.

Note: the local loopback is designed for hosts not routers, unless you set up a specific loopback address and have a route to that address the router will not respond (just accept it and you will be fine deep deep breaths).

Step 3:  Configure NTP_CLIENT_1 to synchronise time with NTP_MASTER_1

NTP_CLIENT_1(config)#ntp server 10.10.10.1 ?

NTP_CLIENT_1(config)#ntp server 10.10.10.1 source fastEthernet 0/0

NTP_CLIENT_1(config)#exit

NOTE: the source is optional and defines which interface to use, if omitted it will utilise the first interface with a valid address.

NTP_CLIENT_1#show ntp status

Clock is synchronized, stratum 9, reference is 10.10.10.1

nominal freq is 250.0000 Hz, actual freq is 250.0000 Hz, precision is 2**18

reference time is C02947F2.4197CD86 (00:21:06.256 UTC Fri Mar 1 2002)

clock offset is 12.7158 msec, root delay is 28.12 msec

root dispersion is 3892.49 msec, peer dispersion is 3879.75 msec

NTP_CLIENT_1#show ntp associations

      address         ref clock             st     when  poll reach  delay  offset    disp

*~10.10.10.1       127.127.7.1       8     3          64   377    20.2   -2.32     9.8

 * master (synced), # master (unsynced), + selected, - candidate, ~ configured

In the output from the command show ntp associations observe that the ref clock is showing as 127.127.7.1 the loopback address utilised on NTP_MASTER_1.

Another useful troubleshooting command is the debug ntp packet command:

NTP_CLIENT_1#

Mar  1 00:30:28.219: NTP: xmit packet to 10.10.10.1:

Mar  1 00:30:28.219:  leap 0, mode 3, version 3, stratum 9, ppoll 128

Mar  1 00:30:28.223:  rtdel 0422 (16.144), rtdsp 065C (24.841), refid 0A0A0A01 (10.10.10.1)

Mar  1 00:30:28.223:  ref C02949E4.3F4C409C (00:29:24.247 UTC Fri Mar 1 2002)

Mar  1 00:30:28.223:  org C02949E4.428D5426 (00:29:24.259 UTC Fri Mar 1 2002)

Mar  1 00:30:28.223:  rec C02949E4.3F4C409C (00:29:24.247 UTC Fri Mar 1 2002)

Mar  1 00:30:28.227:  xmt C0294A24.382188DB (00:30:28.219 UTC Fri Mar 1 2002)

Mar  1 00:30:28.243: NTP: rcv packet from 10.10.10.1 to 10.10.10.3 on FastEthernet0/0:

Mar  1 00:30:28.247:  leap 0, mode 4, version 3, stratum 8, ppoll 128

Mar  1 00:30:28.247:  rtdel 0000 (0.000), rtdsp 0002 (0.031), refid 7F7F0701 (127.127.7.1)

Mar  1 00:30:28.247:  ref C0294A10.7ADBE1B2 (00:30:08.479 UTC Fri Mar 1 2002)

Mar  1 00:30:28.247:  org C0294A24.382188DB (00:30:28.219 UTC Fri Mar 1 2002)

Mar  1 00:30:28.251:  rec C0294A24.3E7B7E3B (00:30:28.244 UTC Fri Mar 1 2002)

Mar  1 00:30:28.251:  xmt C0294A24.3E7BEAB7 (00:30:28.244 UTC Fri Mar 1 2002)

Mar  1 00:30:28.251:  inp C0294A24.3E5300A2 (00:30:28.243 UTC Fri Mar 1 2002)

You can see from this debug output that an NTP packet has been xmit and rec between the ntp client and master.  Interestingly although the masters ref clock is 127.127.7.1 the 10.10.10.1 on interface fa 0/0 of NTP_MASTER_1 is utilised for the exchange.

Step 4: Configure NTP With MD5 Authentication:

NTP_MASTER_2(config)#ntp authentication-key 1 md5 TIME

NTP_MASTER_2(config)#ntp master 1

Client Configuration:

NTP_CLIENT_1#conf t

NTP_CLIENT_1(config)#ntp authentication-key 1 md5 TIME

NTP_CLIENT_1(config)#ntp trusted-key 1

NTP_CLIENT_1(config)# ntp server 10.10.10.2 prefer source fastEthernet 0/0 key 1

Verifying your configuration:

NTP_CLIENT_1#show ntp associations detail

10.10.10.2 configured, authenticated, our_master, sane, valid, stratum 1

ref ID .LOCL., time C0295478.6C7AE76C (01:14:32.423 UTC Fri Mar 1 2002)

our mode client, peer mode server, our poll intvl 64, peer poll intvl 64

root delay 0.00 msec, root disp 0.03, reach 377, sync dist 22.507

delay 27.86 msec, offset -2610.6546 msec, dispersion 8.54

precision 2**18, version 3

org time C029547E.A0C70860 (01:14:38.628 UTC Fri Mar 1 2002)

rcv time C0295481.40AC248E (01:14:41.252 UTC Fri Mar 1 2002)

xmt time C0295481.397FFA36 (01:14:41.224 UTC Fri Mar 1 2002)

filtdelay =    27.86   35.68   31.89   35.95   32.06   32.12   27.98   32.12

filtoffset = -2610.6 -2610.5 -2620.6 -2618.4 -2624.5 -2624.5 -2618.4 -2624.5

filterror =     0.02    0.99    1.57    2.55    3.52    3.54    3.56    3.57

10.10.10.1 configured, insane, invalid, stratum 8

ref ID 127.127.7.1, time C0295410.7AD064B5 (01:12:48.479 UTC Fri Mar 1 2002)

our mode client, peer mode server, our poll intvl 512, peer poll intvl 256

root delay 0.00 msec, root disp 0.03, reach 377, sync dist 19.913

delay 16.14 msec, offset -8.6274 msec, dispersion 10.77

precision 2**18, version 3

org time C029543D.39517280 (01:13:33.223 UTC Fri Mar 1 2002)

rcv time C029543D.3D981B7C (01:13:33.240 UTC Fri Mar 1 2002)

xmt time C029543D.39752B19 (01:13:33.224 UTC Fri Mar 1 2002)

filtdelay =    16.14   28.06   32.15   35.95   28.14   32.15   20.14   20.14

filtoffset =   -8.63  -10.79   15.95   13.73   -2.72   -4.99   -3.41   13.40

filterror =     0.02    1.08    3.04    4.99    6.94    8.90   10.85   14.76

There are several useful pieces of information from the output of the command: show ntp associations detail

Firstly you can see that the NTP_MASTER_2 is now preferred by client 2 and is authenticated and sane also that it is stratum 1, whereas NTP_MASTER_1 is considered insane, invalid and stratum 8.

SECURITY:

Security Advice:  Unless you have a requirement to synchronise externally I avoid it, especially with recent NTP DDos exploits! Additionally if you are setting up an external NTP server follow the latest patching advice etc..

here is a great article on the subject:

http://blog.cloudflare.com/understanding-and-mitigating-ntp-based-ddos-attacks

NTP and Cisco Time - Part 1

Background:

• The current NTP version is 3 and defined in RFC 1305
• Provides synchronisation services for NTP Servers and Clients, enables services such as SNMP and Syslog to utilise accurate time stamps, this facilitates Troubleshooting, Fault Analysis and Incident tracking (think FCAPS).
• NTP polling can take place from 64 to 1024 seconds depending on network conditions.
• NTP uses two me     thods to ensure that the source is reliable firstly it won’t synchronise with a system that isn’t already synchronised, it will also compare with other systems and won’t synchronise with a system that shows a significant deviation from them. 
• If a Cisco device is acting as a server and configured to synchronise against an internet source then it can be configured to pretend that it has remained synchronised.
•  Cisco Devices maintain two clocks the hardware clock which is an example of an (RTC = Real Time Clock) and a software clock the following section shows how these two interact and can be manipulated.

Introduction:

For me it is: 18:31 on Sunday the 23^rd of March 2014 I know this because the clock on my computer tells me it is, if my clock was wrong it could easily be the 2:00 am on Saturday the 5^th of August 1845 how do I tell?  Well I have to believe my computer, and my computer has to believe someone else.. Get the picture..  It’s pretty much the same on the network.

“The future is something which everyone reaches at the rate of sixty minutes an hour, whatever he does, whoever he is.”

CS Lewis

The rate at which time passes is the same for all of us (I think Einstein would disagree) however if you arrive and everyone disagrees on the arrival time who is right?  If only all clocks where synchronized to the beginning of time; if we knew when time began (now there’s an argument to get the astro physicists going).

I have lost count of the networks I have worked on where there is no time-synchronisation.  This is one of those largely hidden facilities that don’t get noticed until you need it, usually when you need to make sense of a particular log to find out when something occurred.

Time and Management FCAPS:

Keeping accurate time on your Cisco devices is vital for a host of network services.  From an FCAPS perspective accurate time assists with:

Fault Management – knowing the exact time a fault occurs can be vital to the troubleshooting process.

Configuration Management – Knowing the exact time a configuration took place helps with historical troubleshooting.

Accounting Management – Knowing when a particular change was made helps backtrack the who when and why of a change.

Performance Management – Imagine a scenario where you are seeing daily reports of network slowness knowing exactly when those high utilization times occur can significantly reduce your troubleshooting time!

Security Management – Knowing exactly when any of the above issues occurred can be vital to security management, imagine your system has been deliberately sabotaged from within, to find the culprit you are relying on Forensic network analysis, knowing when something occurred can often narrow your suspects down!  

A good example of this is when your ISP contacts your organisation to say that they have reports of one of your IP Addresses being a source of malaware.  When you check your firewall logs to see what internal address was allocated a particular external address you can use the time to narrow down who exactly was logged onto that machine during this time.  This way  you have the machine with the infection, the user and what time it occurred!

 Manually Manipulating the time on a Cisco Device:

 Is your clock authoritative on not?

If you issue a show clock detail you will see an asterisk in front of the time which indicates it’s not authoritative.  If you then set the clock manually and reissue a show clock detail you will get a response that shows the clock is now authoritative and set by the user as per the example below,

Show clock [shows the time and date from the software clock not the hardware clock]

NOTE: if you are using GNS3 instead of live hardware you will see a “no time source” response in the initial output instead of the “Time source is hardware calendar” which is the response from live kit.

Setting the Hardware Clock

Unfortunately the time set by the set clock command is not persistent, when you reboot the clock is resynchronised with the hardware or in the case of GNS3 it is simply reset as there is no RTC (real time clock) in gns3.   

You can however update and check the time on the hardware clock in real devices by issuing the following commands:

Clock update-calendar [synchronises the hardware clock to the software clock]

If you want to check that the current time and date settings for the hardware clock :

show calendar  [shows the time and date for the hardware (rtc) not the software clock]

TIMEZONES:

In order to ensure that your time is set to the correct time-zone and daylight savings, you can utilise the following commands:

Clock timezone [this defaults to UTC] 

Here are some well known keywords for the clock timezone command:

Timezone Keyword	Full Name and offset to UTC
GMT	Grenwich Meantime (same as UTC)
BST	British Summer Time UTC + 1
IST	Irish Summer Time UTC +1

Note: UTC (coordinated universal time) is the standard by which the world sets it’s time.  Historically this was GMT however it has been replaced by UTC.  This time business can be a whole business of it’s own trust me do a search for TIME in Google and you will find a couple of hours have magically passed..

Keeping the time automatically:

Setting the time and date accurately is a step forward however, imagine that multiplied by a hundred or even a thousandfold.  With a large round of applause into the room steps NTP aglow in the adoration of sys admins everywhere.  NTP (Network Time Protocol) will allow you to synchronise the time across your devices automatically and in a resilient manner. 

The rest of this guide will cover configuring your cisco devices as both a client and server, I will be covering the client configuration first as this is probably the most commonly performed task and the least complex, secondly I will cover configuring a device as a time source.  Unless you are carrying out a Greenfield network install or changing  a network design at a high level it is unlikely that you are ever going to have to do this, however having an understanding of how it’s done can greatly aid you in troubleshooting time related issues and just simply figuring out what’s broken.

Configuring your Cisco Device to be an NTP Client:

Specify your NTP Server:

You can specify your NTP Servers in global configuration mode with the following command:

Device(config)#ntp server  192.168.4.5 source vlan 4
device(config)#ntp server 192.168.4.7 source vlan 4

The above commands configure two servers to synchronise against, note the use of the source keyword this optional keyword specifies which interface to use as the source of the NTP communications, in this case the NTP servers are located on VLAN 4 and the vlan 4 interface of this device has been specified.

A note on synchronisation time: According to RFC 1305 the clients clock can only be updated by a certain amount at a time.  To reduce the time to synchronisation a good practice is to first manually set the clock close to the ntp servers time to reduce the gap for synchronisation.

You can check which servers your device has associated to with the following command:

Device#show ntp associations

This will give a similar output to the one below: 

time_test#show ntp associations

      address         ref clock                st  when poll    reach  delay  offset     disp

*~192.162.4.5     .PPS.                     1   211      1024  357     10.3     -0.82      2.3

+~192.162.4.6     193.62.22.82      2   455    1024  375      9.0       -0.74      1.6

+~192.162.4.7     193.62.22.74      2   128    1024  377      5.1        -0.69     4.3

* master (synced), # master (unsynced), + selected, - candidate, ~ configured

 Some useful information can be gained from examining this output:

 A * shows that this device is currently synced against the peer 192.168.4.5
The poll column shows that the device is configured to poll the ntp server every 1024 seconds or roughly every 17 minutes minutes.
The when column shows the time that has elapsed since the last NTP packet was received.

 

time_test#show ntp associations detail

193.62.4.5 configured, our_master, sane, valid, stratum 1

ref ID .PPS., time D6D9BB42.E2B46107 (19:57:54.885 UTC Sun Mar 23 2014)

our mode client, peer mode server, our poll intvl 1024, peer poll intvl 1024

root delay 0.00 msec, root disp 0.24, reach 357, sync dist 19.623

delay 10.30 msec, offset -0.8193 msec, dispersion 2.33

precision 2**22, version 3

org time D6D9BB43.D291C3D6 (19:57:55.822 UTC Sun Mar 23 2014)

rcv time D6D9BB43.D4192800 (19:57:55.828 UTC Sun Mar 23 2014)

xmt time D6D9BB43.D171A00F (19:57:55.818 UTC Sun Mar 23 2014)

filtdelay =    10.30   11.81   24.29   10.41   10.27   10.30   36.59   11.72

filtoffset =   -0.82   -1.10   -7.21    0.95    1.20    1.30  -11.71    1.05

filterror =     0.02   15.64   31.27   46.89   78.14   93.77  109.39  140.64

I have removed the output for servers 193.62.4.6 and 193.62.4.7

A final note on multiple servers:

 It is possible to set a server to be the preferred time source over all of the others configured on your device.  Previous show ntp associations output gave 193.62.4.5 as the NTP master, what if we wanted this client to use 193.62.4.6 as the preferred and master?  This can be accomplished as follows:

Time_test(config)#ntp server 193.62.4.6 prefer source vlan 1

Note if you configure this command it can take a long time to take effect as of the time of writing and my research there is no method to force the device to do an immediate ntp synchronization!!

Final Check:

As a final check you can issue the show ntp status command:

Time-test#show ntp status

Clock is synchronized, stratum 2, reference is 193.62.4.6

nominal freq is 381.4697 Hz, actual freq is 381.4701 Hz, precision is 2**17

reference time is D6D9D645.16CE9FB6 (21:53:09.089 UTC Sun Mar 23 2014)

clock offset is -1.2050 msec, root delay is 10.27 msec

root dispersion is 2.23 msec, peer dispersion is 0.75 msec

OK Folks, that’s it for tonight I will complete the next part at a later date It’s 21:55 according to my computer clock and in this universe that will do for me J