Cloudflare DNS Modification Leads to Disruption in Cisco SME Switches

Quick Read

• Cloudflare DNS modification triggered reboot cycles in Cisco SME switches.
• Worldwide repercussions with devices restarting every 10 to 30 minutes.
• Disabling DNS resolution or SNTP synchronization halted the issue.
• Cloudflare rolled back the update to resolve the concern.
• Models impacted: CBS, C1200, and SG series of switches.
• Cisco has not yet provided a firmware update to address the issue.

Cisco Switches Affected by Cloudflare DNS Update

A recent worldwide outage impacted users of Cisco small-to-medium business (SME) switches, forcing devices to reboot every 10 to 30 minutes. The problem was linked to a modification in the response behavior of Cloudflare’s DNS server.

Troubleshooting Steps

Administrators facing the issue found that turning off DNS resolution or SNTP synchronization on the affected switches stopped the rebooting problem. This issue was mainly seen in devices using Cloudflare’s DNS at 1.1.1.1, identified as the source of the problem.

Cloudflare Responds to the Situation

Cloudflare handled the situation by releasing an incident report on January 9, confirming that they rolled back the software update to restore the conventional record ordering. The update had changed the order of CNAME and non-CNAME records, leading to conflicts with specific DNS client implementations.

Models Affected

The issue affected models within the CBS, C1200, and SG series of Cisco switches. While acknowledging the situation, Cisco has yet to release updated firmware for the devices impacted.

Summary

The unforeseen reboot loops in Cisco SME switches were associated with a Cloudflare DNS update. The shift in DNS response resulted in global disturbances, prompting Cloudflare to reverse the update. Administrators successfully mitigated the issue by disabling DNS resolution or SNTP synchronization, awaiting a firmware update from Cisco.

Frequently Asked Questions

Q: What led to the reboots of the Cisco switches?

A: The reboots were triggered by a Cloudflare DNS update that modified the sequencing of DNS records, affecting certain DNS client implementations present in Cisco switches.

Q: What actions can administrators take to stop the reboots?

A: Administrators can prevent the reboots by disabling DNS resolution or SNTP synchronization on the affected devices.

Q: Has the issue been fixed by Cloudflare?

A: Yes, Cloudflare rolled back the DNS update to restore the standard record ordering, resolving the issue.

Q: Are there particular models that are affected by this problem?

A: Yes, the CBS, C1200, and SG series of Cisco switches were impacted.

Q: Has Cisco issued a firmware update?

A: Currently, Cisco has acknowledged the issue but has not issued a firmware update for the affected models.