It's just a shift to the left, and a shift to the right

Securing containers throughout the entire build-ship-run lifecycle means shifting left and right

Fei Huang
© Shutterstock / alleachday

Security is no joke, especially as more and more companies are moving to cloud-based container setups. The stakes are high, and the price of a security breach can be catastrophic. CEO of NeuVector Fei Huang shares his thoughts about why DevSecOps matters and how to shift left and right to ensure security is considered all through the lifecycle, not just at deployment.


Enterprise container adoption continues to swell, owing to the benefits of how rapidly containers can be deployed within automated CI/CD pipelines. However, the security of these environments is too often treated as an afterthought – bolted on late in the process rather than baked in from the beginning, where security can be most effective. This strategy of “shifting left” works on the basis that security should be incorporated earlier in the application container development and delivery lifecycle, along with scanning for vulnerabilities during the build process and in registries, and establishing protective admission control rules. This idea has a real opportunity behind it, bringing with it a cultural shift toward DevSecOps. With DevSecOps, security teams work alongside DevOps to automate security integration within the CI/CD toolchain that’s now being used as part of container implementation.

Security in a container-driven world

The focus on container security strategy becomes increasingly critical as enterprises advance the technology out of the testing lab and into live production environments. Recent research conducted by Market Cube in June 2019 found that 87% of IT professionals were now running containers, with 90% of those running them in production. These numbers are largely consistent with previous research from ESG analyst Doug Cahill, who reported that – in mid-2018 – 56% of enterprises had containers in production, with another 24% expected to join them over the coming year.

This need to shift left and right – securing containers across the entire build-ship-run lifecycle – is confounded by the fact that traditional server workload security tools have not yet caught up to offer suitable support for containers. That said, transitioning to modern infrastructure is about more than just technology; it also represents a new method of developing and delivering code. This change is front and center in a container context, with respect to the use of Kubernetes to automate and orchestrate the build-ship-run phases, auto-scaling, and run-time of containers within production environments.

What your security strategy needs

Traditional security strategies like next-generation firewalls and endpoint security are blind to container attacks, and it’s not possible to simply configure such solutions to be container-aware. Container environments actually have larger attack surfaces than traditional ones because you have container orchestration tools and systems that can also come under attack. For instance, in the infamous crypto-mining attack on Tesla last year, attackers entered through an open Kubernetes console. Another example is the critical vulnerability exposing the Kubernetes API server, which was the first major weakness in Kubernetes security to be discovered, but certainly won’t be the last. In complex enterprise environments with hundreds of thousands of containers and services spinning up and down dynamically, visibility into what’s happening inside the network – to see connections and exploits happening in real time – is mandatory for security. For the same reasons, automation is also a requisite for effective security that spans the build-ship-run pipeline.

Successful attacks on containerized environments aren’t completed after a single exploit. Rather, they rely on a “kill chain” of events – a series of incidents across multiple vectors. Preventing the range of potential attacks means having the ability to monitor the network, the container, and the host. For example, the Tesla incident involved a host exploit followed by an external connection, which would have been detected through the network. As any kill chain progresses, attacks can be detected from the resulting network, container, or host connections as attackers try to move laterally, steal data, or connect to a command-and-control server. This is why multi-vector visibility and monitoring becomes important: to see these attacks in their earliest stages.

SEE ALSO: Report & interview: Plan your container security as a core component

Traffic monitoring

“North-south” traffic has traditionally been the chief concern in guarding against threats to network connections, with attacks arriving from (and connecting to) external sources. Common practice has been to deploy firewalls or control groups to safeguard that traffic. However, a containerized environment brings with it much more east-west traffic. If attackers are able to get inside, exploit a container, and break out, they can try to expand laterally within the network. Without network inspection for all pod or host-to-host traffic in place to isolate those microservices, the kill chain can quickly expand. Therefore, effective east-west traffic monitoring requires microservice isolation, the detection of custom application protocols to ensure only those are used on the network, and secured connections to non-container workloads. Beyond security, this network visibility can also be useful for debugging microservices’ application connections and for demonstrating regulatory compliance.

Closing thoughts

While Kubernetes and other platforms provide some basic built-in security, security strategy needs to go deeper to enhance these protections. Shifting left and shifting right to secure the full build-ship-run lifecycle means adding Layer 7 deep packet inspection, threat detection, connection visualization and monitoring, as well as the ability to baseline processes running in every container and to detect anomalies. It also means having the ability to monitor all file system activity on a container or within a host. If someone then hacks into a container and tries to update a package or library with a vulnerable version and then exploit it – or if they try to download an executable – those attempts can be detected and thwarted.

With a more complete security strategy baked in from the start, enterprises can push ahead with container implementations with more confidence they won’t make headlines for the wrong reasons.


Fei Huang

Fei Huang is the CEO at NeuVector, a container network security company that protects Kubernetes in production.

Inline Feedbacks
View all comments