Baremetal automation
17 October 2024I recently completed a massive project which involved migrating and rebuilding a set of datacenters. The ask was to build a few hundred servers and cluster them together for virtual machine placement. The requirement was to take a clean server, install the required operating system (In this case it was to install them as ESXi hosts) and then configure the clustering solution (Sticking with VMware was VCF).
Thinking about Typescript
14 October 2023Typescript has been popular in the development world for a few years now, introducing a typing system to Javascript development being the primary objective. In this regard, it has been wildly successful and has made (For some developers) a massive improvement over the Javascript development process. Recently, some debate has arisen in the development communities around Typescript vs Javascript and the benefits it brings to the table. While I am far from an expert on Typescript I do work in it daily and have a few points I would like to express around the debate.
Polyglot Development
06 August 2023Today it is pretty common for developers to be involved with end-to-end development, especially when using DevOps principles. With multiple tools and libraries involved in this type of “full” stack development, it is common for developers to use different languages. A developer working on CI / CD will likely have to work with build tools like Make, Gradle or Bazel, and then use CD tools like Helm, Ansible or Terraform. Just by practising DevOps, you will have to be a polyglot developer. A polyglot is defined as somebody proficient in multiple languages.
What happened to DSLs
16 July 2023In the early 2010s the concept of DevOps was still finding its feet, and the landscape was full of new and exciting tools (most of them built on top of Ruby). It seemed at the time that every new tool would introduce a Domain Specific Language (DSL) that the tool would use for describing how to use the tool. Some obvious examples of this are Vagrant and Chef. As an example of the type of DSL that was common let us take a quick look at Vagrant.
Using Buildah to build images and manage containers
28 January 2021With the steady advancement of alternative tooling to the docker ecosystem, I thought it would be a good time to look into the different tooling around building, managing and running images. Buildah is a really interesting tool that is providing a alternative approach to containers and images. The official site provides really good instructions on how to build and run buildah. So the point of the following post is to just write my experience over the last few days of using buildah as an alternative to using docker.
Use PipEnv instead of requirements.txt
15 December 2020Years ago when I first starting looking at Python the default package manager recommended was pip and it was a weak
experience and felt less complete compared to other languages such as ruby’s bundler, C#’s nuget or node’s npm. Compared
to other languages pip felt bare bones and fiddly to configure. Weirdly the method to indicate package dependencies using
pip was to create a requirements.txt, unlike the other languages you had to explicitly state what version of each
package upfront and does not “lock” packages when installed.
Elasticsearch Architecture
02 December 2020Over the last couple years I have built a few clusters and have made some observations around how to design and plan when building a new cluster. Some of the considerations described here would also apply to other systems that have a similar approach to scaling and redundancy. Also the designs discussed in this article should work on any version of elasticsearch and the examples are built using Terraform.
Refactoring Unit Tests using Table Driven Tests
24 November 2020Unit testing is an underused and often forgotten technique for building and debugging software projects. It can be difficult to ensure that tests provide value for the code base. Ideally unit tests should ensure that expectations are validated and help prevent future changes from breaking the existing code base.
Terraform and Docker Swarm
11 March 2019In order to test a local docker swarm cluster you will quickly find a fairly cool tool called Docker Machine, which automates the building of a local cluster. The first time you start playing around with it you will quickly see how easy it can be to create docker machines and manually add them to a docker swarm. The sort of commands would look like the following
Local Docker Swarm
17 September 2018Lately I have been spending a lot of my time dealing with Docker Engines and performing orchestration of containers. Now, two tools exist that help deal with container orchestration. The first tool is Kubernetes, which is amazing, awesome and a lot of work to configure and run. Not really ideal for quick test environments locally. It does work pretty well when using the built in option in the Docker Engine that creates a single node kubernetes cluster. The problem is that you are not really testing a cluster then with one node. The other option is built right into docker, docker swarm, which is what I feel works better in our production environments, since it requires far less configuration and tooling to run.
Elasticsearch and Watches
06 June 2018An important part of the DevOps lifecycle is monitoring and alerting. Often developers can overlook this aspect, since they traditionally did not always support the products they developed. Today, with so many products being deployed to the cloud, it is much more likely that you will be the person responsible for your product. Of course, many tools already exist to help monitor and manage your application, from deeply integrated to something a little less intrusive. Regardless of what applications or software you use, sooner or later you will want your monitoring to alert you to certain conditions. This becomes more apparent when you realise that dashboards do not help much, when you do not have time to always look at them.
Remote Docker API
14 May 2018Managing and enabling the Docker engine can be accomplished in many ways. Initially I treated Docker much like I would treat a local installation. Which means I would use Docker in a pretty inefficient method, and would rely on many other tools to deploy containers to my environment. Then I looked into the Docker remote API and suddenly realized how much effort I wasted…
When Docker is the tool, everything is a container
19 February 2018I recently had an interesting situation occur with my experiences with containers and Docker. What had happened is that we are building a application using the meteor framework, and in order to make the process as containerized as possible, we ended up putting most of the development environment into containers. For those who don’t know, meteor is a Javascript framework that is about a gig in size. And it is not needed for the final production application, instead it is used to build the application.
New Year and Terraform
04 January 2018Happy new year. This is the year that I will finally be able to move on from a vague understanding of DevOps to feeling that I have a fairly good understanding. To go with my new found experience and confidence I have decided to at least once a month put up an article discussing something I have been working on during that month. The idea is to help me remember to share some of my experiences and of course to help other people who might have the same problems that I have experienced.
Multihost Vagrantfile
17 May 2017So, sometimes you just need to try and build a multihost cluster for development or testing reasons. What happens if you want to have multiple multihost environments for testing? How can we achieve this with Vagrant? Vagrant does not provide any specific functionality to define multiple environments in the same Vagrantfile. It does however allow multimachines to be defined in the same Vagrantfile.