What’s up?
Cisco released Field Notice 70489 this week making owners of a wide range of Cisco devices of an impending certificate expiration issue that will both cause all generated self-signed certificates to expire on 2020-01-01 00:00:00 UTC and create a situation where new self-signed certificates cannot be created on affected devices after 2020-01-01 00:00:00 UTC.
This issue affects only self-signed certificates that were generated by the Cisco IOS or Cisco IOS XE device and applied to a service on the device. Certificates that were generated by a Certificate Authority (CA), which includes those certificates generated by the Cisco IOS CA feature, are not impacted by this issue.
The following features may be relying on the IOS-generated self-signed certificates, so you will want to check these before the New Year (UTC time).
General features
- HTTP Server over TLS (HTTPS): HTTPS will produce an error in the browser that indicates that the certificate is expired.
- SSH Server: Users who use X.509 certificates to authenticate the SSH session might fail to authenticate. (This use of X.509 certificates is rare. Username/password authentication and public/private key authentication are not affected.)
- RESTCONF: RESTCONF connections might fail.
Collaboration features:
- Session Initiation Protocol (SIP) over TLS
- Cisco Unified Communications Manager Express (CME) with encrypted signaling enabled
- Cisco Unified Survivable Remote Site Telephony (SRST) with encrypted signaling enabled
- Cisco IOS dspfarm resources (Conference, Media Termination Point, or Transcoding) with encrypted signaling enabled
- Skinny Client Control Protocol (SCCP) Telephony Control Application (STCAPP) ports configured with encrypted signaling
- Media Gateway Control Protocol (MGCP) and H.323 call signaling over IP security (IPSec) without a pre-shared key
- Cisco Unified Communications Gateway Services API in Secure Mode (using HTTPS)
Wireless features:
- LWAPP/CAPWAP connections between older Cisco IOS access points (manufactured in 2005 or earlier) and Wireless LAN Controllers; see Cisco Field Notice FN63942 for more details.
If those certificates expire without a new certificate in place, you could end up with situations such as:
- SIP over TLS calls will not complete.
- Devices registered to Cisco Unified CME with encrypted signaling enabled will no longer function.
- Cisco Unified SRST with encrypted signaling enabled will not allow devices to register.
- Cisco IOS dspfarm resources (Conference, Media Termination Point, or Transcoding) with encrypted signaling enabled will no longer register.
- STCAPP ports configured with encrypted signaling will no longer register.
- Calls through a gateway using MGCP or H.323 call signaling over IPSec without a pre-shared key will fail.
- API calls that use the Cisco Unified Communications Gateway Services API in Secure Mode (using HTTPS) will fail.
- RESTCONF might fail.
- HTTPS sessions to manage the device will display a browser warning that indicates the certificate has expired.
- AnyConnect SSL VPN sessions will fail to establish or report an invalid certificate.
- IPSec connections will fail to establish.
The “Lab’s Eye” view
Rapid7 Labs looked at certificates from recent Sonar scan data and found over 80,000 Cisco devices that will likely be impacted by this impending expiration issue.
Top 20 Countries With Likely Impacted Cisco Devices
| Country | n | pct |
|---|---|---|
| United States | 14689 | 21.61% |
| India | 4999 | 7.35% |
| South Korea | 2706 | 3.98% |
| Russia | 2249 | 3.31% |
| Mexico | 2174 | 3.20% |
| Taiwan | 2078 | 3.06% |
| Japan | 2069 | 3.04% |
| United Kingdom | 2024 | 2.98% |
| Brazil | 1948 | 2.87% |
| Canada | 1677 | 2.47% |
| Australia | 1374 | 2.02% |
| Singapore | 1364 | 2.01% |
| China | 1239 | 1.82% |
| Saudi Arabia | 1090 | 1.60% |
| Nigeria | 1020 | 1.50% |
| France | 976 | 1.44% |
| Italy | 849 | 1.25% |
| Germany | 766 | 1.13% |
| Thailand | 751 | 1.10% |
| Zambia | 696 | 1.02% |
Top 20 Networks With Likely Impacted Cisco Devices
| AS Org | n | pct |
|---|---|---|
| Reliance Jio Infocomm Limited | 2664 | 3.92% |
| Comcast Cable Communications, LLC | 1923 | 2.83% |
| AT&T Services, Inc. | 1071 | 1.58% |
| Utah Education Network | 1028 | 1.51% |
| Charter Communications Inc | 1018 | 1.50% |
| Korea Telecom | 890 | 1.31% |
| Alestra, S. de R.L. de C.V. | 812 | 1.19% |
| Liquid Telecommunications Ltd | 793 | 1.17% |
| SingNet | 788 | 1.16% |
| Cox Communications Inc. | 782 | 1.15% |
| National Chiao Tung University | 728 | 1.07% |
| Level 3 Parent, LLC | 712 | 1.05% |
| ALGAR TELECOM S/A | 702 | 1.03% |
| Cable & Wireless Panama | 626 | 0.92% |
| MCI Communications Services, Inc. d/b/a Verizon Business | 585 | 0.86% |
| Taiwan Academic Network (TANet) Information Center | 556 | 0.82% |
| BTC-GATE1 | 544 | 0.80% |
| NTT Communications Corporation | 511 | 0.75% |
| Uninet S.A. de C.V. | 507 | 0.75% |
| TE-AS | 492 | 0.72% |
These external exposure numbers are just the tip of the iceberg, as most organizations wisely do not expose remote interfaces on Cisco routers and switches, but do have to expose Cisco devices that are designed for, say, telephony operations. The internal exposure to this issue will likely be orders of magnitude larger.
What can you do?
Rapid7 customers can use InsightVM to report on expiring certificates in general and filter on Cisco devices in particular to identify impacted systems to facilitate faster remediation. Not a customer? You can get a free 30-day trial of InsightVM today to see if your Cisco devices are impacted.
Anyone can also use the openssl client found bundled with many operating systems and perform the following for each target IP address/hostname:
openssl s_client \
-showcerts \
-connect DEVICE_IP_ADDRESS:SERVICE_PORT 2>/dev/null | \
openssl x509 -noout -dates | \
grep notAfter
If you see the following response, your device is very likely impacted by this issue:
notAfter=Jan 1 00:00:00 2020 GMT
Rapid7 Labs feels compelled to re-emphasize the fact that failure to remediate this issue prior to the expiration date will create a situation where new self-signed certificates cannot be created on affected devices after 2020-01-01 00:00:00 UTC. The last thing most of us want is to ring in the new year with an emergency support call/page, so take some time before the new year to make sure none of your devices are going to be impacted by this issue.