Docker Hub Rate Limits Effective November 1, 2020

Introduction

Docker has announced rate limits for Docker pulls that go into effect November 1, 2020. We suspect that most Security Onion users will NOT notice this change. Additional details are below. We are monitoring this situation closely and will send out additional information as necessary.

Docker Hub Rate Limits

From https://www.docker.com/blog/docker-hub-image-retention-policy-delayed-and-subscription-updates/:

"Anonymous free users will be limited to 100 pulls per six hours, and authenticated free users will be limited to 200 pulls per six hours. Docker Pro and Team subscribers can pull container images from Docker Hub without restriction as long as the quantities are not excessive or abusive."

For more information, please see:

https://www.docker.com/increase-rate-limits

https://docs.docker.com/docker-hub/download-rate-limit/

https://www.docker.com/pricing/resource-consumption-updates

When does Security Onion do a Docker Pull?

During installation, Security Onion should only do a "docker pull" if performing a network installation (since ISO installations already have the Docker images).

After installation, Security Onion should only do a "docker pull" when you run soup and soup downloads updated Docker images.

The rest of this blog post will focus on soup updating Docker images.

Security Onion 16.04

Security Onion 16.04 installations include a total of 7 Docker images. If you run soup on an older version of Security Onion 16.04 and it pulls updated Docker images, then that would be 7 docker pulls. This wouldn't come anywhere near the rate limit of 100 pulls per six hours. However, if you have a distributed deployment with multiple nodes all behind a single NAT IP address, then it's possible to start approaching that rate limit. If you experience the rate limit, there are a couple of possible solutions. The first option is to authenticate to Docker to increase the rate limit. The second option is to upgrade to Security Onion 2 which should be less likely to hit the rate limit as we'll describe in the next section. Upgrading to Security Onion 2 is a good idea anyway since Security Onion 16.04 reaches End Of Life in April 2021.

Security Onion 2

Security Onion 2 distributes all components via Docker images. Depending on installation type, that could be upwards of 30 Docker images. If you run soup on an older version of Security Onion 2 and it pulls updated Docker images, then that could be up to 30 docker pulls. Even in the case of a distributed deployment with multiple nodes all behind a single NAT IP address, the default configuration is for the manager to update the Docker images for the entire deployment so it should only be 30 docker pulls for the entire deployment. Therefore, Security Onion 2 should be less likely to hit the rate limit than Security Onion 16.04.

Conclusion

We suspect that most Security Onion users will NOT notice this change. We are monitoring this situation closely and will send out additional information as necessary. If you have any questions or concerns, please reach out to the appropriate community support forum as described here:
https://blog.securityonion.net/2020/10/community-support-forum-changes-for.html