Key steps for transforming your DevOps team into a DevSecOps force

Enterprise security has been severely tested in the last year.

All organisations’ attack surfaces have expanded and attackers are challenging security vendors’ and enterprises’ abilities to repel them. Remote working, multiple devices requesting access, and cloud workloads mean a vast space for attackers to probe and find the weak links.

Thus putting more responsibilities on software teams to make the tightest, most secure code – not only to be productive, but also to reduce enterprise risk. 

DevSecOps is the philosophy of developing applications and infrastructure securely from ideation to deployment. It requires consideration of security risks at all stages of the development lifecycle. Traditionally, software delivery teams have been fixed on automating building, testing, and deployment of applications. DevSecOps includes automating security practices to allow teams to increase security without losing velocity as they engineer.

Here’s how to fast-track the necessary changes to use DevSecOps practices on your software delivery team.

Getting started as a DevSecOps force

Continuous integration and continuous deployment (CI/CD) pipelines are core to operation at businesses today. The more resources pipelines have access to (secure secrets, proprietary code, databases, etc.), the more important it is to keep your CI/CD system secure. There are three security practices to address, all being equally important:

  1. Secure pipeline configuration

Use the CI/CD pipeline configuration to lessen the likelihood of security issues happening.

First, safely store secrets that you use in your pipelines for connecting to databases and third-party services. You may have the option to use encrypted-at-rest environment variables, or a ‘contexts’ feature. Contexts are used to provide access to environment variables across projects. Their use can also be restricted to specific security group members.

Then, for sensitive files like code signing keys, add an additional layer of isolation between encrypted files and repositories for added security. Keep them encrypted in your repository and store the decryption key in the environment variables or contexts, or inject them from another source instead of having them inline with the code (in an ideal system), only decrypting them inside your CI/CD jobs when they are needed. This makes it even more difficult to expose or leak these highly sensitive information.

Third, you never want to leave your CI/CD environment running without monitoring it. Make sure that the containers and VMs used for your pipelines are destroyed after jobs that require sensitive information finish running. This can happen automatically if supported by your solution.

Then, pay extra attention to how your CI/CD system handles builds for pull requests that come from forks of your repository. If your build stages require secrets to work, forked pull requests might gain access to them through your pipelines. Set a default behaviour to never pass on the secrets to the forked pull requests to prevent leaking your credentials.

Addressing these four aspects will significantly reduce the risk of security incidents due to the CI/CD pipeline configuration.

2.  Code and Git history analysis

Git features a comprehensive history of changes, allowing you to look through a project history with a simple CLI command. But this means that any sensitive information in the Git history can be accessible to attackers if a repository is exposed – even if the most recent state of the repository doesn’t contain the secrets anymore.

Trufflehog and GitLeaks are two tools that help identify secrets that have been committed to the codebase so that you can deactivate and replace them. These will also scan your Git history for traces of secrets that your team might have added to the repository in the past. Once the Git history is clear of secrets, you can proceed to the next level: making sure that your current revision doesn’t contain any vulnerable dependencies.

Static Application Security Testing (SAST) techniques can look through the application in your commit and analyse its dependencies. If any dependencies contain any issues or known security vulnerabilities, your commit will be marked as insecure and won’t be allowed to proceed to deployment.

Dynamic Application Security Testing (DAST) techniques go one step further and spin up a copy of your production environment inside your CI job in order to scan the resulting containers and executables. The dynamic aspect helps the system catch dependencies that are being loaded at launch time, for example, as those won’t be caught by SAST.

Enforce security policy 

You can’t check all security aspects statically based on known vulnerabilities. Some are specific to your particular company, and need to be codified as policies. These can take the form of automated or manual compliance checks. Some tasks like reviewing the list of all accounts that have access to your repositories, or making sure your onboarding and offboarding processes are in sync, will be manual. Perform these regularly.

Some tasks actually can be easily automated. Third-party services can conveniently codify rules that will match against your CI pipeline, e.g., proving compliance with the regulations that govern your data. If there is a mismatch, the build will fail.

Your team is your most valuable set of security researchers because they know your applications from constantly working on them. They need a clear process to report security issues when found, and dedicated time to fix them. 

Celebrate those who report issues. Make it easy and open – help them report issues quickly. Moreover, don’t blame people who report false positives for wasting the team’s time. Hold blameless post-mortems so that everyone can learn what to do correctly.

All these factors help create the right DevSecOps culture and can make or break a security-minded development culture that strives for high performance.


About the Author

Michael Stahnke is VP Platform at CircleCI running SRE, Security, and tooling. Prior to this, he worked at Puppet running PE, Platform engineering as well as SRE. He is an established author where he has co-authored two State of DevOps Reports and most recently, a Data-Driven approach to continuous integration. He is also a popular speaker and has attended various DevOps Days, CTO Summits, Puppetize conferences and more. He also founded the package repository EPEL and wrote a book on SSH in 2005.

Featured image: ©alexdndz