Why Cloud Demands DevOps

By Michael Biddick

For some, the words “software development” evoke fear, loathing and flashbacks to failed or costly (or both) projects. But the emergence of cloud has spawned a renaissance in how development happens, and channel providers need to understand how to help customers revamp legacy applications for the modern era.

You’ve probably seen the term “DevOps.” The movement was coined in 2009 as a reaction to decades of failed development projects. DevOps, at its core, is a cultural movement in response to the mistakes commonly made by large agencies developing software that went over budget, continually missed deadlines and then did not meet user requirements. DevOps aims to fix that.

DevOps found initial traction among large public cloud service providers, where much of what used to be considered infrastructure is now part of the code. The lessons learned from Google, Amazon, Twitter and Etsy are now directly applied to new software development projects for the federal agencies that I work with. The concepts are proven and ready to move downstream to small and midsize companies in all sectors and verticals.

The DevOps approach emphasizes communication, collaboration and integration between software developers and IT operations. In the past, these groups typically worked as silos. DevOps acknowledges the interdependence of code and infrastructure. It helps an organization produce software and IT services more quickly, with frequent iterations.

What is tough for most agencies is the cultural change DevOps demands. Shared ownership and collaboration are cornerstones. Functional silos must be broken down so applications can adapt more quickly to user needs. This can only happen with close teamwork.

Agencies also struggle with significant talent gaps. The supply of DevOps engineers comes nowhere near meeting current demand. Progressive federal CIOs have been building teams and augmenting their staff with partners; I expect this to become the norm with all businesses.

Let’s look at how to develop a DevOps practice.

First, you need to lose the idea of coding as a mysterious black box and demand that all software be intuitive for users, reliable, provide easy-to-interpret results and visual cues, and be able to run on modern, virtualized infrastructure.

I’ve found that managing the technical know-how, processes and communications to deliver software that checks these boxes and meets user and business needs is an art, but one worth cultivating. In a business world now driven by software, Accenture says you could speed up customer application development cycles by factor of 30, with 50 percent fewer failures. Return on investment will …

{vpipagebreak}

… improve as more information is shared, not only between developers and operations staff, but also with marketing and business teams. As better software helps customers become more productive and competitive — which, at the end of the day, is the goal of DevOps — they spend more.{ad}

DevOps isn’t just a way of working. There are also tools used in the following stages of the code development and release cycle:

Successful DevOps also means reconciling developer and IT operations goals. The role of developers is to change software. IT operations staff, on the other hand, want stability. By working together and using the tools described above, both goals can be achieved by making sure software is continually produced in a ready-to-deploy state. The cultural change (the close collaboration) and the automated tools are both crucial for improving the speed and quality of software development and deployment. For the enterprise as a whole, DevOps then means higher productivity, faster time to market, and increased competitiveness.

Channel Benefit: Automation

Software ready for the real world still has a virtual last mile to go. After development, testing, continuous integration, and quality assurance, for example, it must be installed on a production server — or perhaps hundreds or even thousands of production servers. When these servers are largely identical, as they are in the cloud, it makes sense to automate deployment and subsequent maintenance. Not only does this save time and effort, it eliminates many of the potential errors inherent in manual deployment.

Deployment management tools offer automation and orchestration capabilities for speedy and reliable deployment over large, decentralized and cloud-based populations of servers. With increasing frequency of incremental software releases, these tools become even more important for IT operations staff to manage their server bases. By using predefined schemas (also called templates, recipes or playbooks), administrators can …

{vpipagebreak}

… “define once and deploy many.” In addition, popular deployment management tools can be linked into a continuous delivery chain. Software releases can then automatically travel from stage to stage for final, automated deployment.{ad}

Four products currently dominate the market for these tools, with a choice of management applications to federate these tools and others for DevOps continuous delivery. Most have partners programs:

Federating applications for these and other deployment management tools include:

Which product should you choose? Larger customers looking for mature, stable products may prefer Chef or Puppet. Those who prefer speed and simplicity may be drawn to Ansible and Salt. As a federating application, Ansible Tower offers ease of integration through its REST API. Foreman on the other hand has the advantage of being open-source and freely available, compared with Ansible Tower’s commercial licensing.

Pick Your Container

In the beginning, applications ran on one operating system on one physical server, period. Then came hypervisors and virtual machines. A VM offers a full operating system and associated resources, and several VMs can run in the same physical machine. However, VMs must still be started or “spun up” in a server. They can consume a relatively large amount of resources and there are often limits on their portability.

Containers offer a new type of virtualization by packaging an application and its dependencies — runtime, system tools, system libraries — together, but while using a host’s operating system. As a result,containers use fewer resources, can be moved from one host to another, and are simpler and faster to deploy. They offer a number of …

{vpipagebreak}

… advantages for DevOps:

Does that mean containers will replace virtual machines? Although their resource savings and speed of deployment make containers attractive, they are less secure than VMs, and they must often all use the same Linux operating system. Coexistence is more likely. In shared environments like public clouds, containers are often implemented within a VM, for this reason.{ad}

Container technology has only recently achieved widespread recognition, starting with the introduction of Docker in 2013. Since then, the number of container technologies has increased:

Which container technology should you use? Developers may find Docker attractive because it has a large ecosystem. They may opt for LXD for its flexibility in mixing and matching different Linux operating environments. Operations staff may prefer for Rocket for reasons of simplicity and security. Microsoft shops may opt for Docker, supported by Microsoft’s Azure, or in the future perhaps — but don’t hold your breath — Microsoft Drawbridge containers.

The closer a team works together on producing and deploying a software application, the better the chances of a timely, high-quality result.

CI: The Road Through Integration Hell

DevOps is the way to achieve this, through tight collaboration between developers and operations staff, and automation of the different release steps. To move ahead in creating code, developers also need to work separately on individual parts of the code.

A risk in working alone, however, is that difficulties will arise when all these individual efforts are integrated. In pre-DevOps days, this integration phase was often…

{vpipagebreak}

…pushed back until after all the development work had been done. This led to “integration hell,” where teams struggled to make different modules work together. Code then had to be rewritten or even scrapped, meaning a waste of time and money.

DevOps reduces this risk radically by using continuous integration (CI). Smaller, but more frequent, code changes are committed to the version control system being used. A CI server then builds the overall application from the latest versions of the modules in the version control system and tests the build. These integration test results shine a spotlight on problems or conflicts, which can then be resolved, giving a solid, tested code base on which the next development can be based. With the higher frequency of integration testing, rework and waste are minimized.{ad}

Continuous integration, which can be performed several times per day, has existed for some time. In DevOps, CI can be extended to a process of continuous delivery and deployment. Code that has passed CI testing is automatically made available, for instance, to a deployment management application for automated installation on a range of production servers. CI products include:

Which one should you use? All are strong choices, but some tools providing continuous integration possibilities also extend to handle much or even all of the overall DevOps workflow. They offer a “pipeline as code” approach, in which a workflow and deliveries from one stage to another can be automated, end to end. If that’s attractive,consider Atlas or Jenkins.

Mind the Versions

In DevOps, the goal is not only to excel in producing applications. It is also to control the IT infrastructure through code, including integration testing, server deployment and configuration, monitoring and reporting. Once the code …

{vpipagebreak}

… is in place for any of these items, it can be automatically triggered and executed. This code-based approach is what makes continuous delivery possible in DevOps: Applications flow from their creation through to deployment in the real world of production.

Code version control is therefore vitally important to successful DevOps. Application code branches must be correctly merged for continuous integration testing. Infrastructure configuration code must be available in the latest version to operations staff.

Version control systems can also hold configurations for performance monitoring and log management systems. In addition, clear records of which code changes were made where, when and why are crucial for speedy problem resolution and auditing.{ad}

Although today’s popular version control systems have similar goals, they have different strengths and weaknesses:

Which one should you use? If your DevOps team is small, SVN may be the better choice. For larger projects or ones in which several developers will be updating the code at different times, Git may be preferable. If developers balk at Git, perhaps for lack of user friendliness, Mercurial may be a suitable compromise. However, the following recent developments associated with Git may have solved major usability issues:

Whichever version control system you use, each code change committed to the system can also automatically trigger continuous integration and/or continuous deployment. Speed and reliability of releases can then go up, while issues and incidents go down.

Round Up the Data

Servers and applications generate a wealth of performance feedback and log data. This data can be used at …

{vpipagebreak}

… various stages of a DevOps release cycle to help pinpoint the origins of issues and indicate opportunities for improvement. With the right tool for collecting data from different locations, collating it and making sense of it, developers and operations staff can:

Rapid analysis of feedback data, at whatever stage, allows developers to produce and test new versions of software in a timely way. These new versions can then follow the same DevOps path to production release as the original version of the application — for instance, via continuous integration and testing, code version control and deployment management.{ad}

Better still, performance analytics can be moved forward (“feedforward”) in the DevOps cycle to evaluate performance of an application even before it is deployed in production. With customers, your team can define key performance indicators to measure performance when an application goes live. 

Well-known application performance and log management tools include:

Which one should you use? That depends on your, or your customer’s, budget and willingness to handle complexity. Out of the four, PaperTrails is the entry-level solution in terms of simplicity, range of functionality and cost of implementation. New Relic offers pay-as-you-go pricing. An implementation of Logstash is also affordable, although the mixed technologies make its implementation more complex. Splunk, on the other hand, can be costly, both in licensing and implementation, positioning it as more of an enterprise-level solution.

As CEO of Fusion PPT, Michael Biddick is responsible for the strategic vision, market strategy, project quality and overall performance of the company. His unique blend of technology experience coupled with business acumen has helped the company achieve triple-digit growth. A published author with more than 60 articles on IT topics, Biddick earned bachelor’s degrees in political science and African-American history from the University of Wisconsin-Madison, and a master’s degree in information systems from John Hopkins University.

LinkedIn: linkedin.com/in/michaelbiddick
Twitter: @michaelbiddick