Your company has an IPsec tunnel with another company for achieving network connectivity between servers in 10.10.10.0/24
on your side to 10.20.20.0/24
on theirs.
Lately they complained that their equipment has problems dealing with ESP and requested to migrate this existing IPsec tunnel from Encapsulating Security Payloads (ESP) to Authentication Headers (AH), since encryption/confidentiality was never a requirement for this tunnel. Please note that network connectivity is ok in the initial state, using ESP.
The only change that you did was adding a new transform set T_SET_AH
to use ah-md5-hmac
and updating the crypto map
to use it.
Together with the partner company you establish a maintenance window and perform the migration from ESP to AH. The new tunnel using AH comes up, but unfortunately
You start troubleshooting and conclude the following:
- ISAKMP phase gets established
- IPSEC phase gets established
- the new parameters (AH & MD5) are agreed correctly
- the protected internal subnets
10.10.10.0/24
and10.20.20.0/24
are agreed correctly but there is no network connectivity between them - you see packets being sent but nothing received back
! No connectivity R1#ping 10.20.20.20 source 10.10.10.10 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.20.20.20, timeout is 2 seconds: Packet sent with a source address of 10.10.10.10..... Success rate is 0 percent (0/5) R1#! Phase 1 OK R1#sh crypto isakmp sa IPv4 Crypto ISAKMP SA dst src state conn-id slot status 44.44.44.44 12.12.12.1 QM_IDLE 1001 0 ACTIVE IPv6 Crypto ISAKMP SA! Phase 2 OK R1# sh crypto ipsec sa detail interface: FastEthernet0/0 Crypto map tag: IPSEC_VPN, local addr 12.12.12.1 protected vrf: (none)local ident (addr/mask/prot/port): (10.10.10.0/255.255.255.0/0/0) remote ident (addr/mask/prot/port): (10.20.20.0/255.255.255.0/0/0) current_peer 44.44.44.44 port 4500 PERMIT, flags={origin_is_acl,}#pkts encaps: 5, #pkts encrypt: 5, #pkts digest: 5 #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0 #pkts compressed: 0, #pkts decompressed: 0 #pkts not compressed: 0, #pkts compr. failed: 0 #pkts not decompressed: 0, #pkts decompress failed: 0 #pkts no sa (send) 5, #pkts invalid sa (rcv) 0 #pkts encaps failed (send) 0, #pkts decaps failed (rcv) 0 #pkts invalid prot (recv) 0, #pkts verify failed: 0 #pkts invalid identity (recv) 0, #pkts invalid len (rcv) 0 #pkts replay rollover (send): 0, #pkts replay rollover (rcv) 0 ##pkts replay failed (rcv): 0 #pkts internal err (send): 0, #pkts internal err (recv) 0local crypto endpt.: 12.12.12.1, remote crypto endpt.: 44.44.44.44 path mtu 1500, ip mtu 1500, ip mtu idb FastEthernet0/0 current outbound spi: 0xDEE2A85C(3739396188) inbound esp sas:inbound ah sas: spi: 0x7612ECC1(1980951745) transform: ah-md5-hmac , in use settings ={Tunnel UDP-Encaps, } conn id: 1, flow_id: SW:1, crypto map: IPSEC_VPN sa timing: remaining key lifetime (k/sec): (4530158/3488) replay detection support: Y Status: ACTIVE inbound pcp sas: outbound esp sas:outbound ah sas: spi: 0xDEE2A85C(3739396188) transform: ah-md5-hmac , in use settings ={Tunnel UDP-Encaps, } conn id: 2, flow_id: SW:2, crypto map: IPSEC_VPN
What is wrong ? Why everything works if the IPsec tunnel uses ESP but it is not working with AH?
Even though you requested your peer network engineer working for the Partner Company to give you the config from his router (R3), due to internal policies, he rejected that !... but he confirms your above conclusion on his side: tunnel goes up but no connectivity !
Eventually, you enable crypto debugging for both phases. You also bring in a senior network consultant. He looks at the show crypto
outputs and the debug crypto
below and concludes that
Here is the debug output:
R1#deb crypto isakmp Crypto ISAKMP debugging is on R1# R1#deb crypto ipsec Crypto IPSEC debugging is on R1# R1#sh crypto isakmp sa IPv4 Crypto ISAKMP SA dst src state conn-id slot status IPv6 Crypto ISAKMP SA R1# R1# R1# R1#ping 10.20.20.20 sou R1#ping 10.20.20.20 source 10.10.10.10 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.20.20.20, timeout is 2 seconds: Packet sent with a source address of 10.10.10.10 *Mar 1 00:02:06.175: IPSEC(sa_request): , (key eng. msg.) OUTBOUND local= 12.12.12.1, remote= 44.44.44.44, local_proxy= 10.10.10.0/255.255.255.0/0/0 (type=4), remote_proxy= 10.20.20.0/255.255.255.0/0/0 (type=4), protocol= AH, transform= ah-md5-hmac (Tunnel), lifedur= 3600s and 4608000kb, spi= 0x0(0), conn_id= 0, keysize= 0, flags= 0x0 *Mar 1 00:02:06.179: ISAKMP:(0): SA request profile is (NULL) *Mar 1 00:02:06.183: ISAKMP: Created a peer struct for 44.44.44.44, peer port 500 *Mar 1 00:02:06.183: ISAKMP: New peer created peer = 0x663A6274 peer_handle = 0x80000002 *Mar 1 00:02:06.187: ISAKMP: Locking peer struct 0x663A6274, refcount 1 for isakmp_initiator *Mar 1 00:02:06.191: ISAKMP: local port 500, remote port 500 *Mar 1 00:02:06.191: ISAKMP: set new node 0 to QM_IDLE *Mar 1 00:02:06.191: insert sa successfully sa = 66981718 *Mar 1 00:02:06.191: ISAKMP:(0):Can not start Aggressive mode, trying Main mode. *Mar 1 00:02:06.195: ISAKMP:(0):found peer pre-shared key matching 44.44.44.44 *Mar 1 00:02:06.203: ISAKMP:(0): constructed NAT-T vendor-rfc3947 ID *Mar 1 00:02:06.203: ISAKMP:(0): constructed NAT-T vendor-07 ID *Mar 1 00:02:06.207: ISAKMP:(0): constructed NAT-T vendor-03 ID *Mar 1 00:02:06.207: ISAKMP:(0): constructed NAT-T vendor-02 ID *Mar 1 00:02:06.207: ISAKMP:(0):Input = IKE_MESG_FROM_IPSEC, IKE_SA_REQ_MM *Mar 1 00:02:06.207: ISAKMP:(0):Old State = IKE_READY New State = IKE_I_M.M1 *Mar 1 00:02:06.207: ISAKMP:(0): beginning Main Mode exchange *Mar 1 00:02:06.207: ISAKMP:(0): sending packet to 44.44.44.44 my_port 500 peer_port 500 (I) MM_NO_STATE *Mar 1 00:02:06.207: ISAKMP:(0):Sending an IKE IPv4 Packet..... Success rate is 0 percent (0/5) R1# *Mar 1 00:02:16.207: ISAKMP:(0): retransmitting phase 1 MM_NO_STATE... *Mar 1 00:02:16.207: ISAKMP (0:0): incrementing error counter on sa, attempt 1 of 5: retransmit phase 1 *Mar 1 00:02:16.207: ISAKMP:(0): retransmitting phase 1 MM_NO_STATE *Mar 1 00:02:16.207: ISAKMP:(0): sending packet to 44.44.44.44 my_port 500 peer_port 500 (I) MM_NO_STATE *Mar 1 00:02:16.207: ISAKMP:(0):Sending an IKE IPv4 Packet. R1# *Mar 1 00:02:26.207: ISAKMP:(0): retransmitting phase 1 MM_NO_STATE... *Mar 1 00:02:26.211: ISAKMP (0:0): incrementing error counter on sa, attempt 2 of 5: retransmit phase 1 *Mar 1 00:02:26.215: ISAKMP:(0): retransmitting phase 1 MM_NO_STATE *Mar 1 00:02:26.215: ISAKMP:(0): sending packet to 44.44.44.44 my_port 500 peer_port 500 (I) MM_NO_STATE *Mar 1 00:02:26.219: ISAKMP:(0):Sending an IKE IPv4 Packet. *Mar 1 00:02:26.331: ISAKMP (0:0): received packet from 44.44.44.44 dport 500 sport 500 Global (I) MM_NO_STATE *Mar 1 00:02:26.347: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH *Mar 1 00:02:26.351: ISAKMP:(0):Old State = IKE_I_MM1 New State = IKE_I_MM2 *Mar 1 00:02:26.363: ISAKMP:(0): processing SA payload. message ID = 0 *Mar 1 00:02:26.363: ISAKMP:(0): processing vendor id payload *Mar 1 00:02:26.363: ISAKMP:(0): vendor ID seems Unity/DPD but major 245 mismatch *Mar 1 00:02:26.363: ISAKMP (0:0): vendor ID is NAT-T v7 *Mar 1 00:02:26.363: ISAKMP:(0):found peer pre-shared key matching 44.44.44.44 *Mar 1 00:02:26.363: ISAKMP:(0): local preshared key found *Mar 1 00:02:26.363: ISAKMP : Scanning profiles for xauth ... *Mar 1 00:02:26.363: ISAKMP:(0):Checking ISAKMP transform 1 against priority 10 policy *Mar 1 00:02:26.363: ISAKMP: encryption DES-CBC *Mar 1 00:02:26.363: ISAKMP: hash SHA *Mar 1 00:02:26.363: ISAKMP: default group 1 *Mar 1 00:02:26.363: ISAKMP: auth pre-share *Mar 1 00:02:26.363: ISAKMP: life type in seconds *Mar 1 00:02:26.363: ISAKMP: life duration (VPI) of 0x0 0x1 0x51 0x80 *Mar 1 00:02:26.367: ISAKMP:(0):atts are acceptable. Next payload is 0 *Mar 1 00:02:26.367: ISAKMP:(0):Acceptable atts:actual life: 0 *Mar 1 00:02:26.371: ISAKMP:(0):Acceptable atts:life: 0 *Mar 1 00:02:26.371: ISAKMP:(0):Fill atts in sa vpi_length:4 *Mar 1 00:02:26.375: ISAKMP:(0):Fill atts in sa life_in_seconds:86400 *Mar 1 00:02:26.379: ISAKMP:(0):Returning Actual lifetime: 86400 *Mar 1 00:02:26.379: ISAKMP:(0)::Started lifetime timer: 86400. *Mar 1 00:02:26.379: ISAKMP:(0): processing vendor id payload *Mar 1 00:02:26.379: ISAKMP:(0): vendor ID seems Unity/DPD but major 245 mismatch *Mar 1 00:02:26.379: ISAKMP (0:0): vendor ID is NAT-T v7 *Mar 1 00:02:26.379: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE *Mar 1 00:02:26.379: ISAKMP:(0):Old State = IKE_I_MM2 New State = IKE_I_MM2 *Mar 1 00:02:26.383: ISAKMP:(0): sending packet to 44.44.44.44 my_port 500 peer_port 500 (I) MM_SA_SETUP *Mar 1 00:02:26.387: ISAKMP:(0):Sending an IKE IPv4 Packet. *Mar 1 00:02:26.391: ISAKMP:(0):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE *Mar 1 00:02:26.391: ISAKMP:(0):Old State = IKE_I_MM2 New State = IKE_I_MM3 *Mar 1 00:02:26.491: ISAKMP (0:0): received packet from 44.44.44.44 dport 500 sport 500 Global (I) MM_SA_SETUP *Mar 1 00:02:26.495: ISAKMP:(0):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH *Mar 1 00:02:26.499: ISAKMP:(0):Old State = IKE_I_MM3 New State = IKE_I_MM4 *Mar 1 00:02:26.511: ISAKMP:(0): processing KE payload. message ID = 0 *Mar 1 00:02:26.531: ISAKMP:(0): processing NONCE payload. message ID = 0 *Mar 1 00:02:26.531: ISAKMP:(0):found peer pre-shared key matching 44.44.44.44 *Mar 1 00:02:26.531: ISAKMP:(1001): processing vendor id payload *Mar 1 00:02:26.531: ISAKMP:(1001): vendor ID is Unity *Mar 1 00:02:26.531: ISAKMP:(1001): processing vendor id payload *Mar 1 00:02:26.531: ISAKMP:(1001): vendor ID is DPD *Mar 1 00:02:26.531: ISAKMP:(1001): processing vendor id payload *Mar 1 00:02:26.531: ISAKMP:(1001): speaking to another IOS box! *Mar 1 00:02:26.531: ISAKMP (0:1001): NAT found, the node outside NAT *Mar 1 00:02:26.531: ISAKMP:(1001):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE *Mar 1 00:02:26.531: ISAKMP:(1001):Old State = IKE_I_MM4 New State = IKE_I_MM4 *Mar 1 00:02:26.531: ISAKMP:(1001):Send initial contact *Mar 1 00:02:26.531: ISAKMP:(1001):SA is doing pre-shared key authentication using id type ID_IPV4_ADDR *Mar 1 00:02:26.531: ISAKMP (0:1001): ID payload next-payload : 8 type : 1 address : 12.12.12.1 protocol : 17 port : 0 length : 12 *Mar 1 00:02:26.531: ISAKMP:(1001):Total payload length: 12 *Mar 1 00:02:26.531: ISAKMP:(1001): sending packet to 44.44.44.44 my_port 4500 peer_port 4500 (I) MM_KEY_EXCH *Mar 1 00:02:26.531: ISAKMP:(1001):Sending an IKE IPv4 Packet. *Mar 1 00:02:26.531: ISAKMP:(1001):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE *Mar 1 00:02:26.535: ISAKMP:(1001):Old State = IKE_I_MM4 New State = IKE_I_MM5 *Mar 1 00:02:26.639: ISAKMP (0:1001): received packet from 44.44.44.44 dport 4500 sport 4500 Global (I) MM_KEY_EXCH *Mar 1 00:02:26.647: ISAKMP:(1001): processing ID payload. message ID = 0 *Mar 1 00:02:26.647: ISAKMP (0:1001): ID payload next-payload : 8 type : 1 address : 192.168.1.2 protocol : 17 port : 0 length : 12 *Mar 1 00:02:26.655: ISAKMP:(0):: peer matches *none* of the profiles *Mar 1 00:02:26.655: ISAKMP:(1001): processing HASH payload. message ID = 0 *Mar 1 00:02:26.663: ISAKMP:(1001):SA authentication status: authenticated *Mar 1 00:02:26.663: ISAKMP:(1001):SA has been authenticated with 44.44.44.44 *Mar 1 00:02:26.663: ISAKMP: Trying to insert a peer 12.12.12.1/44.44.44.44/4500/, and inserted successfully 663A6274. *Mar 1 00:02:26.663: ISAKMP:(1001):Input = IKE_MESG_FROM_PEER, IKE_MM_EXCH *Mar 1 00:02:26.663: ISAKMP:(1001):Old State = IKE_I_MM5 New State = IKE_I_MM6 *Mar 1 00:02:26.663: ISAKMP:(1001):Input = IKE_MESG_INTERNAL, IKE_PROCESS_MAIN_MODE *Mar 1 00:02:26.663: ISAKMP:(1001):Old State = IKE_I_MM6 New State = IKE_I_MM6 *Mar 1 00:02:26.663: ISAKMP:(1001):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE *Mar 1 00:02:26.663: ISAKMP:(1001):Old State = IKE_I_MM6 New State = IKE_P1_COMPLETE *Mar 1 00:02:26.663: ISAKMP:(1001):beginning Quick Mode exchange, M-ID of 2112503013 *Mar 1 00:02:26.663: ISAKMP:(1001):QM Initiator gets spi *Mar 1 00:02:26.671: ISAKMP:(1001): sending packet to 44.44.44.44 my_port 4500 peer_port 4500 (I) QM_IDLE *Mar 1 00:02:26.675: ISAKMP:(1001):Sending an IKE IPv4 Packet. *Mar 1 00:02:26.679: ISAKMP:(1001):Node 2112503013, Input = IKE_MESG_INTERNAL, IKE_INIT_QM *Mar 1 00:02:26.679: ISAKMP:(1001):Old State = IKE_QM_READY New State = IKE_QM_I_QM1 *Mar 1 00:02:26.679: ISAKMP:(1001):Input = IKE_MESG_INTERNAL, IKE_PHASE1_COMPLETE *Mar 1 00:02:26.679: ISAKMP:(1001):Old State = IKE_P1_COMPLETE New State = IKE_P1_COMPLETE *Mar 1 00:02:26.775: ISAKMP (0:1001): received packet from 44.44.44.44 dport 4500 sport 4500 Global (I) QM_IDLE *Mar 1 00:02:26.783: ISAKMP:(1001): processing HASH payload. message ID = 2112503013 *Mar 1 00:02:26.787: ISAKMP:(1001): processing SA payload. message ID = 2112503013 *Mar 1 00:02:26.787: ISAKMP:(1001):Checking IPSec proposal 1 *Mar 1 00:02:26.791: ISAKMP: transform 1, AH_MD5 *Mar 1 00:02:26.791: ISAKMP: attributes in transform: *Mar 1 00:02:26.791: ISAKMP: encaps is 3 (Tunnel-UDP) *Mar 1 00:02:26.795: ISAKMP: SA life type in seconds *Mar 1 00:02:26.795: ISAKMP: SA life duration (basic) of 3600 *Mar 1 00:02:26.795: ISAKMP: SA life type in kilobytes *Mar 1 00:02:26.795: ISAKMP: SA life duration (VPI) of 0x0 0x46 0x50 0x0 *Mar 1 00:02:26.795: ISAKMP: authenticator is HMAC-MD5 *Mar 1 00:02:26.795: ISAKMP:(1001):atts are acceptable. *Mar 1 00:02:26.795: IPSEC(validate_proposal_request): proposal part #1 *Mar 1 00:02:26.795: IPSEC(validate_proposal_request): proposal part #1, (key eng. msg.) INBOUND local= 12.12.12.1, remote= 44.44.44.44, local_proxy= 10.10.10.0/255.255.255.0/0/0 (type=4), remote_proxy= 10.20.20.0/255.255.255.0/0/0 (type=4), protocol= AH, transform= NONE (Tunnel-UDP), lifedur= 0s and 0kb, spi= 0x0(0), conn_id= 0, keysize= 0, flags= 0x0 *Mar 1 00:02:26.795: Crypto mapdb : proxy_match src addr : 10.10.10.0 dst addr : 10.20.20.0 protocol : 0 src port : 0 dst port : 0 *Mar 1 00:02:26.795: ISAKMP:(1001): processing NONCE payload. message ID = 2112503013 *Mar 1 00:02:26.799: ISAKMP:(1001): processing ID payload. message ID = 2112503013 *Mar 1 00:02:26.803: ISAKMP:(1001): processing ID payload. message ID = 2112503013 *Mar 1 00:02:26.811: ISAKMP:(1001): Creating IPSec SAs *Mar 1 00:02:26.811: inbound SA from 44.44.44.44 to 12.12.12.1 (f/i) 0/ 0 (proxy 10.20.20.0 to 10.10.10.0) *Mar 1 00:02:26.811: has spi 0x7612ECC1 and conn_id 0 *Mar 1 00:02:26.811: lifetime of 3600 seconds *Mar 1 00:02:26.811: lifetime of 4608000 kilobytes *Mar 1 00:02:26.811: outbound SA from 12.12.12.1 to 44.44.44.44 (f/i) 0/0 (proxy 10.10.10.0 to 10.20.20.0) *Mar 1 00:02:26.811: has spi 0xDEE2A85C and conn_id 0 *Mar 1 00:02:26.811: lifetime of 3600 seconds *Mar 1 00:02:26.811: lifetime of 4608000 kilobytes *Mar 1 00:02:26.811: ISAKMP:(1001): sending packet to 44.44.44.44 my_port 4500 peer_port 4500 (I) QM_IDLE *Mar 1 00:02:26.811: ISAKMP:(1001):Sending an IKE IPv4 Packet. *Mar 1 00:02:26.811: ISAKMP:(1001):deleting node 2112503013 error FALSE reason "No Error" *Mar 1 00:02:26.815: ISAKMP:(1001):Node 2112503013, Input = IKE_MESG_FROM_PEER, IKE_QM_EXCH *Mar 1 00:02:26.819: ISAKMP:(1001):Old State = IKE_QM_I_QM1New State = IKE_QM_PHASE2_COMPLETE *Mar 1 00:02:26.827: IPSEC(key_engine): got a queue event with 1 KMI message(s) *Mar 1 00:02:26.827: Crypto mapdb : proxy_match src addr : 10.10.10.0 dst addr : 10.20.20.0 protocol : 0 src port : 0 dst port : 0 *Mar 1 00:02:26.827: IPSEC(crypto_ipsec_sa_find_ident_head): reconnecting with the same proxies and peer 44.44.44.44 *Mar 1 00:02:26.831: IPSEC(policy_db_add_ident): src 10.10.10.0, dest 10.20.20.0, dest_port 0*Mar 1 00:02:26.831: IPSEC(create_sa): sa created, (sa) sa_dest= 12.12.12.1, sa_proto= 51, sa_spi= 0x7612ECC1(1980951745), sa_trans= ah-md5-hmac , sa_conn_id= 1 *Mar 1 00:02:26.831: IPSEC(create_sa): sa created, (sa) sa_dest= 44.44.44.44, sa_proto= 51, sa_spi= 0xDEE2A85C(3739396188), sa_trans= ah-md5-hmac , sa_conn_id= 2 *Mar 1 00:02:26.831: IPSEC(update_current_outbound_sa): updated peer 44.44.44.44 current outbound sa to SPI DEE2A85C R1# R1#
What did the senior engineer found ? What is the problem?
Post your answer in the 'Comments' section below and subscribe to this blog to get the detailed solution and more interesting quizzes.
Comments
comments powered by Disqus