In DevOps workflows, automation saves time, conserves resources, reduces errors and ensures consistency. Despite this, developers continue to ignore the existence of world-class workflow automation and job scheduling solutions and instead rely on a variety of simple tools to code jobs as they build apps. Since neither these tools nor the developers adhere to the same standards, the applications passed to Operations often fail to meet production standards. And when something breaks, the problem is often hard to find and fix.
Since such large portions of business application processing are performed by job scheduling functions, automating their creation will not only save time during development, testing, and deployment, but also enable developers to focus more time on building the best business logic.
Consider the makeup of an application. Code, such as Java and Python implements the business logic. The application runs on infrastructure, such as a Linux server, web application server, or network. Databases and SQL statements create tables and rebuild indexes. Finally, many dependent jobs and tasks have to execute, often in a particular order. This job scheduling simply determines when jobs run and what to do if a job fails or can’t start when scheduled. When developers spend such a significant amount of their development time defining this essentially administrative functionality, they are, in a very real sense, wasting their time only to create future headaches for Operations that waste even more time.
“Jobs-as-Code” is a way of standardizing and automating job scheduling by embedding code using a simple notation that makes API calls to a scheduling engine, then managing these calls throughout the continuous delivery pipeline, the same way Java or Python code is managed. As such, Jobs-as-Code is fundamentally no different than other code management practices.
Here is how Jobs-as-Code can fit into a streamlined development process:
Don’t think of jobs as business logic
Job scheduling is mere instrumentation. Scripting it requires a significant effort for the build, test and support processes – and errors and inconsistency can carry over into Operations. Instead, a job scheduler that can manage flow relationships, success/failure analysis, output capture, and other functions will ensure consistency across an application and from application to application, saving time and eliminating challenges for both Development and Operations.
Test early and often
Today, the syntax of source code is checked as it’s created. Apply a similar approach to job definitions. In traditional development environments, new, scripted, and therefore inconsistent scheduling may work fine during testing only to fail in production. By automating scheduling, similar notation and interfaces for all jobs can be used at the earliest stages, which in turn allows for early and accurate testing. This is even more effective when “infrastructure as code” is used to provision a test environment that is as close to the production environment as possible, making it possible to anticipate and eliminate resource contention and inconsistencies with other workloads.
Use a source code management system
A source code management (SCM) system (such as GIT, Subversion, CVS, TFS, etc.) creates a central repository for storing application components and managing versions. It enables fallback to previous versions, “diff-ing” versions to identify changes, and managing potential drift in deployed systems. It also provides a quick and reliable method for building or rebuilding the application in new environments.
When any component in the SCM is revised, the build tool (such as Jenkins®) rebuilds and tests the application for errors or functionality regression. Typically, when a test fails, the entire development team must work to fix the problem, causing delays. The use of Jobs-as-Code will naturally reduce the number of errors.
Further, more complex applications require more complex development lifecycle, often involving developing, testing, integrating, staging, pre-production testing, user acceptance testing (UAT), and more. By combining Jobs-as-Code with an SCM and early testing, increased complexity does not have to lead to more downstream challenges. Instead, applications arrive in production as consistent and thoroughly tested as possible.
Consider the value equation
For an application to deliver the greatest return on investment, the development process must be as streamlined and efficient as possible, while the application should run in production with the fewest possible issues. Jobs-as-Code should now be seen as a key element in achieving both these goals.
Finally, keep in mind that even the most rigorous development practices will occasionally fail. Jobs-as-Code adds visibility to applications so that when necessary, the operations and support teams can more quickly identify, analyze, and resolve problems to make the application available again.
