Data Generated by DevSecOps Practices
DevSecOps replaces practices that in the past have been labor-intensive and error-prone. CI is the automated process by which developers integrate code then build, test, and validate new applications. Its success was not practical until compilers (and the underlying compute hardware) evolved to be able to compile code quickly. Also needed were robust version control, configuration management, and test suites.
CD is the automated process of creating releasable artifacts. Its success depends on the ability of today's tools to automate not only the building of programs, but execution of system tests and delivery of validated code into production. Infrastructure as code—the scripting or virtualization of infrastructure that replicates the operational environment and optimizes computing resources—depends on the availability of encapsulated virtual environments, another recent technological innovation.
The automation that makes these DevSecOps practices possible in turn spawns a large amount of data as a by-product. This data can be made available to enable stakeholders to assess the health of a project including its development performance, operational performance, whether it is sufficiently secure, and how frequently upgrades are being delivered.
What Are Metrics, and Why Do We Need Them?
Software metrics enable stakeholders in the development of software—developers, security personnel, operations personnel, development teams, and executives—to know key things they need to know about software projects, answering such questions as the following:
- Is the service delivering value to the users?
- Is the service operating properly?
- Is the organization achieving its business goals?
- Is the service secure?
- Is the infrastructure able to support throughput, memory constraints, and other requirements?
- Is the service being attacked?
- Can future needs be supported?
- What will be the cost and risk of adding new features?
Data that is generated by the DevSecOps methodology can help provide answers to these and similar questions.
Metrics are measurements of system properties or performance that inform decisions. They can be used to understand what happened or what might happen in the future. They help to determine such things as
- if the process is stable
- if the process is capable
- if goals are being met
- how alternative processes, tools, or products compare
- how to manage change
Limitations of Existing DevSecOps Metrics
Studies have identified four key metrics that support software development and delivery performance. Two relate to tempo and two to stability.
Tempo
- deployment frequency
- lead time from commit to deploy
Stability
- mean time to recover from downtime (mean time to restore [MTTR)]) or mean time between failures (MTBF)
- change failure rate or percentage
