What happened to DSLs
In the early 2010s the concept of DevOps was still finding its feet, and the landscape was full of new and exciting tools (most of them built on top of Ruby). It seemed at the time that every new tool would introduce a Domain Specific Language (DSL) that the tool would use for describing how to use the tool. Some obvious examples of this are Vagrant and Chef. As an example of the type of DSL that was common let us take a quick look at Vagrant.
Vagrant
Vagrant is a tool that helps you define a virtual machine (Often for local development) in the days before containers became the popular choice. Vagrant provided a customised DSL based on Ruby syntax that allows you as the developer a flexible and easy language for defining the virtual machines you wanted to build.
Vagrant.configure("2") do |config|
config.vm.box = "hashicorp/bionic64"
end
The example above gives you the option to build a new virtual machine based on the Ubuntu operating system. If you want to do conditional checks you can just use the default ruby syntax to achieve that as the example below shows.
Vagrant.configure("2") do |config|
if ENV['BOX']
config.vm.box = ENV['BOX']
else
config.vm.box = "hashiCorp/bionic64"
end
Learning how to be good at Vagrant just required you to learn a subset of Ruby, which was designed to be easy to use and learn. If you worked in Ruby already, it was great to be able to define your testing infrastructure with the same syntax.
What about today
Ruby did not stick around in the development community and most DevOps tools today seem to prefer YAML as the definition language. Kubernetes, Ansible and Azure DevOps are common examples of this approach. The problem with YAML is that it is not a language, it is a key / value syntax. So how do complex concepts get represented in YAML, often via syntax sugar such as Jinga Templates for Ansible or Helm for Kubernetes. I find the syntax sugar approach to be a regression from the usage of language DSLs, since they are often inflexible and bespoke solutions. Knowing Jinga template syntax for Ansible does not directly translate to Python skills for example.
I recently started working on migrating Jenkins Groovy pipelines to Azure DevOps and find the YAML syntax that Azure DevOps pipelines use to be somewhat awkward to read and understand compared to plain old Jenkins Groovy scripts. Especially since Azure DevOps introduces strange template concepts to allow for conditions when executing steps in jobs.
Let’s just look at the same example deployment in Groovy compared to Azure DevOps. The pipeline will build a node package, but when building on the main branch will also call publish. First, we shall look at the Jenkins Groovy pipeline (Jenkins often refers to this as a Scripted Pipeline and I realise that Jenkins is also moving away from this approach, but for the sake of an example let’s not get caught up in that discussion :))
node {
stage('Test') {
sh('npm test')
}
stage('Build') {
sh('npm run build')
}
stage('Publish') {
if (env.BRANCH_NAME == 'main') {
sh('npm publish')
}
}
}
The Jenkins Groovy syntax is not too complex to follow, and you can add additional branching or even error handling with try / catch statements. If you want you can also easily write functions or common libraries to be fancy. Generally, I would recommend against making the pipelines too complex, but the option is available.
Let’s compare that same task being defined in Azure DevOps
- task: Npm@1
inputs:
command: 'test'
- task: Npm@1
inputs:
command: 'build'
- task: Npm@1
condition: $
inputs:
command: 'publish'
At first glance, it seems to be simpler, but the amount of syntax sugar knowledge you need to know is substantial. In the Jenkins pipeline, you will have to learn that sh
is a function that executes shell commands. In Azure DevOps, the npm command is wrapped for you with the Npm@1
task. Can you execute plain shell commands in Azure DevOps, sure you can, but if the wrapper is provided then it makes more sense to use that (Since the reader of the configuration knows exactly what you want to use). The condition
key is where the complexity in the pipeline can start to arise since YAML does not have conditional syntax. This means that the functions defined in the Azure DevOps pipeline are completely custom to Azure DevOps pipelines. Unlike the Jenkins Groovy pipeline, which just uses the built-in syntax available to all Groovy scripts, the YAML configuration requires a custom syntax specifically built for the tool.
Building DSLs into YAML is not going anywhere, since so many large tools have already adopted it. It just feels like the YAML DSLs are very limited and when complicated scenarios are represented the syntax can become messy and hard to read. I guess that the language-driven DSLs just felt more comfortable for somebody who has spent a lot of time writing code, while the YAML syntax feels more constrained. It is a pity that so many tools today use YAML as the configuration definitions, instead of a DSL built on top of an existing language.