Skip to main content
Exploring what’s next in tech – Insights, information, and ideas for today’s IT and business leaders

How to achieve extreme quality with DevOps

Does testing in the dev pipeline make for higher quality software? The success of Netflix may prove that it does.

Internet sites serving millions of users at any one time have to survive software failures gracefully—in other words, they need to fix things while keeping their service running. Netflix, as an example, is a master of this, but we'll get to that in a moment.

To pull this off, Netflix and others are continually testing software and incorporating feedback from those tests into the development cycle to make everything more reliable. It's all possible by DevOps and related methodologies, critical approaches without which businesses could at times grind to a halt.

DevOps looks a lot like automation. It uses configuration management tools like Chef and Puppet, and ultimately system administrators' shell programming. But it grew out of Agile software development. As Ernest Mueller, an early DevOps leader, puts it, "DevOps can be interpreted as an outgrowth of Agile—Agile software development prescribes close collaboration of customers, product management, developers, and (sometimes) QA to fill in the gaps and rapidly iterate towards a better product. ... From this perspective, DevOps is simply extending Agile principles beyond the boundaries of the code to the entire delivered service."

True, DevOps first transformed system administration, but it soon changed development just as fundamentally. DevOps is also about building faster and better software by bringing—as the name says—development and operations teams together.

What's Agile got to do with it?

To fulfill the promises of Agile, development must take a test-first approach, instead of the old waterfall test-at-the-end development model. In Agile, code is developed and then immediately tested to fix bugs and take small steps forward in functionality. Testing happens all over the development process. What DevOps brings to the table is it extends the Agile team members from developers and testers to include the system operators. This expanded DevOps Agile team can deliver production code several times a day instead of every two to four weeks from a Scrum sprint.

It was from this that continuous integration/continuous delivery (CI/CD) sprang. As Aaron Cois, a software engineer at Secureworks and an adjunct professor at Carnegie Mellon University, puts it, "A cornerstone of DevOps is CI, a technique designed and named by Grady Booch that continually merges source code updates from all developers on a team into a shared mainline. This continual merging prevents a developer's local copy of a software project from drifting too far afield as new code is added by others, avoiding catastrophic merge conflicts."

To avoid those catastrophic merge conflicts, continuous testing must be present to successfully implement a DevOps CI/CD pipeline. Without continuous testing, a.k.a. shift-left testing, CI/CD couldn't exist. Introducing a failure into the process risks exposing the organization to new security, reliability, and compliance risks, or at the very least disrupting the user experience.

Please read: GitOps: The next step in cloud-native development

In this approach, you test your code early and often. The goal is to eliminate all critical bugs well before the code is deployed, even though ideally, you end up deploying code daily.

Quality happens in the CI/CD pipeline

The name of the continuous testing game is to place the testing phase as early in the software development lifecycle (SDLC)—shifting left—as possible. You should avoid old-school testing at the end of the SDLC.

Which brings us back to Netflix. One of the world leaders in CI/CD, Netflix is constantly updating its back-end code even as it delivers programming to tens of millions of viewers every hour.

Netflix's best-known testing program is called Chaos Monkey, which randomly terminates production instances to ensure Netflix can keep delivering video no matter what happens. The company also uses continuous testing techniques long before the code sees the light of day. In Spinnaker, Netflix's open source, multicloud continuous delivery platform, testing is done using third-party testing libraries. These include MinutestMockkSpockStrikt, and Testcontainers.

Of course, Netflix started working on its testing methodology when it moved from renting DVDs to streaming. Your business probably isn't starting from a greenfield. Instead, before you start using continuous testing, you'll need to get a handle on what you're doing now. The best way to do this is to use workflow visualization tools such as flowcharts—yes, in 2021 the flowchart lives—business process maps, or a Kanban board.

You can shift left, too

During this process, you must adjust how you see testing. You'll no longer be following a model where the developers hand off the code to the quality assurance team. Instead, your Agile crew, which will include QA, will need to push testing into the SDLC as early as possible. To do that, QA must do more than help build tests that can be inserted into the CI pipeline. The QA team must help not only automate the test process but redefine the test methodologies as well.

To do this, you'll need to embrace test automation frameworks and tools. These include CucumberStudioParasoft Continuous Quality SuiteSeleniumTravis, and Smartbear Test Complete. Within these frameworks, you can build your own tests.

Please read: Continuous integration and delivery tool basics

These automated tests will include functionality testing, performance testing, API testing, static code analysis, and security testing. Any test that can be automated should be automated and placed as "left" as possible in the CI/CD pipeline. Of course, some testing, such as static code, unit, and functionality tests, can be more easily placed in CI. Other testing, such as performance and security tests, require a virtualized or full production environment and must be done late in the CD stage.

To do this, you should look into adopting the Agile test-driven development approach. In TDD, you start by writing the tests for your code before you write a program. That's not as crazy as it may sound. The idea is, it forces you to describe exactly what your software is meant to do. The process looks like this:

  1. Build unit test for a program's function.
  2. Run the test, which should fail because the program lacks that feature.
  3. Write "just enough" code, the simplest possible, to make the test pass.
  4. Refactor the code until it conforms to the simplicity criteria.

Some types of tests lend themselves more to DevOps automation than others. For example, static analysis tests are designed to analyze source code or compiled versions of code to help find errors. They do this by analyzing the code against a set of rules. Not all errors can be determined at compile time, but many important ones can.

For example, most programmers are familiar with lint, the primal C and C++ static analysis program. It looks for "obvious" problems, such as uninitialized variablesindexing beyond array bounding, and null pointer references. Similar programs, such as ESLint for JavaScript, exist for most languages.

Static testing programs use coding guidelines, such as Perforce's MISRA, or programming standards. The details vary, but the name of the game is always the same: Check for errors against a known standard. Typically, static analysis is used to hunt down defects in source code before a program is run. For example, you'd do it between coding and unit testing or dynamic code testing.

Please read: DevSecOps and containers: How a cloud-native model places new demands on organizational security

Just because some of the tests are old doesn't mean that they're not useful. They can still be very handy. Shuah Khan, a senior Linux kernel maintainer, says, "It is easier to detect and fix problems early during the development process. Static analysis is a necessary component for developing dependable software. I prefer to incorporate static analysis in my development and patch workflow as much as possible."

This all sounds like a lot of work, doesn't it? Well, yes, it is at the start, but it's worth the trouble. In the long run, you'll waste less time waiting around for tests to be completed. In particular, your QA people won't have to wait until developers are implementing their features to contribute to getting ready-to-run production code out the door. After all, testing will be happening before, during, and after development.

In the end, by using DevOps in conjunction with continuous testing to its fullest advantage, you'll improve both the quality of your code and speed up your production. In addition, by incorporating testing into your SDLC, you'll reduce your QA and testing costs.

Lessons for leaders

  • Automation of processing and lengthening of pipelines into ops means that DevOps creates more opportunities for software to be tested.
  • Continuous testing of software in modern DevOps processes has resulted in software of unparalleled resilience.
  • If more and better testing makes for better software, then DevOps and CI/CD are the way to go for quality.

This article/content was written by the individual writer identified and does not necessarily reflect the view of Hewlett Packard Enterprise Company.