Skip to main content

Search

Cloudi-Fi Knowledge Base

INCIDENT_ Some of DHCP nodes down_11/05/2023

Alexandre de THOMASSON
  • Updated

Incident Summary

During the incident window, some of our nodes went down (so all VPN tunnels to these nodes went down and so DHCP service was not usable)

 

Status

Resolved

 

Incident time window

Start: May 11th 2023, at 06:07 UTC

  • Problem definition
  • Operation to identify the root cause
  • Workaround and development of the patch

End: May 11th 2023, at 09:03 UTC

 

Root cause

This incident was due to an ID conflict (duplicate) in the routing table used by DHCP service and by our IPSEC service (for information - each packet is tagged with an ID to allow us to segment traffic and route it in the right tunnel). As a result, the IPSec traffic was not routed properly on nodes and all tunnels went down.

 

Impact

All locations using EMEA (node - 87.98.189.202), APAC (51.79.178.35, 139.99.69.206), US (51.81.66.49)

 

Actions plan

Action 1 - System - Done (11/05/2023, 09:00 UTC)

IPSEC - change the ID of the tunnel (that created the conflict)

Action 2 - System - Done (11/05/2023, 09:00 UTC)

IPSEC - remove the ID that created the conflict to prevent from its re-use