It has certainly been an interesting several months for container users. Late last year, we saw the emergence of CVE-2018-1002105, a privilege escalation vulnerability in Kubernetes that allows attackers to subvert the Kubernetes API to gain access to the attached resources.
More recently, researchers published details about CVE-2019-5736 that described an issue in runC — a runtime component used by a variety of container engines. The vulnerability could be exploited by an attacker to modify the runC binary, thereby offering them root privileges on the host running the container engine.
Each of these issues is a potentially big deal in its own right. Either one, for example, could be used by an attacker as a vector for compromise, as a means to move laterally in an environment, or to gain access to applications and data. But coming as they do in rapid-fire fashion, it’s particularly serious and should serve as a lesson about the value of certain architectural principles that can safeguard production use of containers.
These aren’t new principles, but they remain true in containerized environments just as they did in virtualized or physical environments. The difference is that, sometimes, these principles are overlooked in newer technologies like containerization and container orchestration.
With that in mind, let’s look at the specifics of these vulnerabilities to determine what they can teach us about both container security and the orchestration tools that drive them.
Kubernetes API and runC
The first issue we’ll examine is the Kubernetes API issue — CVE-2018-1002105. In short, it allows an authenticated user to make specially formulated requests by abusing HTTP’s 101 Switching Protocols feature to request a persistent socket to subvert how Kubernetes authenticates API requests. An attacker can use this to connect to previously connected back-end services as if they were the API server.
The runC issue — CVE-2019-5736 — operates a little differently, but it is relevant to mention here because it too affects environments that use containers. In this instance, the vulnerability allows a malicious container to overwrite runC to allow arbitrary code to execute as root on the system running the container engine. This allows hackers to access or manipulate other containers on the same host, use the host as a launchpad to attack other devices in the same infrastructure, or gain access to secrets or other valuable information on the host.
Fixing these issues as they stand right now isn’t rocket science. As I’m sure you’ve heard a million times, patching is the best path to remediation. Specifically, upgrading to Kubernetes 1.12.3 — or 1.11.5/1.10.11, if you’re using those branches — can rectify the first issue, while updating to anything after runC 1.0-rc6 will address the second one.
Lessons for the future
One of the most valuable and powerful takeaways from these vulnerabilities is the continued importance of building in strategies to defend against segmentation failures. Anytime you’re using virtualization of any kind — whether it’s containers, OS virtualization, V8 Isolates, or really anything else where processes, memory or secrets must be kept separate — security teams must design ways to minimize the fallout if there is a failure in the segmentation model.
An analogy would be if a swarm of angry bees built a nest inside your garage. You’d obviously want to get that resolved right away. But between the time when you discover them and the time you have someone come in to deal with them, you’d know in the back of your mind that they were there. In fact, you’d probably take a few active measures to keep them out of the rest of the house. You’d be conscientious about keeping the door to the garage closed, you’d probably put something under the door to seal it, and you’d stay alert for bees in the rest of the house. In much the same way, it’s important to maintain an awareness of these segmentation vulnerabilities and plan container security strategies to help mitigate the impact of that boundary breaking down.
In the past, I’ve outlined ways to do this in the abstract. This advice includes ensuring that you’re fully educated about the security features — and limitations — of the platforms you’re employing, using threat modeling to evaluate and address weak points, and being familiar with the security features of the platforms you use. The two issues mentioned above help illustrate the positive influence of these efforts.
For example, in the runC vulnerability, certain Security-Enhanced Linux policies — like those employed by default on Red Hat systems — prevent the issue from occurring. Likewise, because the runC binary needs to be overwritten to exploit the vulnerability, a file integrity monitoring tool can alert an administrator quickly should hackers exploit the issue.
Likewise, the impact of the Kubernetes issue can be somewhat lessened in situations where you have an awareness of exactly which back-end components are present and how you use them. While that knowledge won’t prevent the issue from being exploited, it can help lessen the amount of work involved to ensure that your system wasn’t accessed in a way you don’t condone.
As we’ve seen in the past with vulnerabilities that break the segmentation boundary in an OS virtualization context, building in buffers against segmentation issues and maintaining awareness about these and other container security issues is helpful to ensure organizations stay protected.