Issue with TCP client quota limits in BIND 9.16.0
The ISC has issued an Operation Notification for BIND 9.16.0 - An error in handling TCP client quota limits can exhaust TCP connections.
In the previous blog article announcing BIND 9.16.0, it was discussed that significant work was done to modernize BIND's networking framework to use libuv, a multi-platform C library that provides async I/O on event loops. Unfortunately, as a result of that work, there was an issue introduced in the code that enforces TCP client quota limits. The issue in the code is that there are situations where the TCP client count is not properly decremented.
Under some circumstances, especially if a server is accepting TCP connections from clients on multiple interfaces, once the TCP client quota is reached, the server may stop accepting new TCP connections. The problem is manifested because the software counter does not accurately reflect the actual number of current TCP client connections. There may be available TCP client connections, but clients are denied since the software counter hasn't been decremented properly to accurately reflect the total number of TCP Client connections that are available and under the quota amount.
Servers which encounter this defect will continue to accept and process UDP queries (which represents the majority of query traffic on most servers) but can lose the ability to accept new TCP connections until the server is restarted.
To avoid this issue, one workaround is to ratchet up the tcp-clients directive high enough so that it is not expected to be encountered in normal operation.
WARNING The tcp-clients directive DOES take up a small incremental amount of memory. Additionally, a nefarious actor could possibly use this configuration setting to trigger a TCP exhaustion attack by purposefully consuming all the available TCP connections.
Since the workaround is not effective against deliberate exploitation, the ISC recommends that operators do one of two things:
- Download a patch diff https://downloads.isc.org/isc/bind9/9.16.0/patches and apply it to the 9.16.0 source code using the patch utility, and recompile to include the behavior fix.
- Revert to a stable production release of BIND from a branch prior to the work done with the libuv framework introduced in 9.15/9.16. A supported stable release would be 9.11 and 9.14 (9.11.16 and 9.14.11 being the most recent)