devops

The kubebuilder documentation on finalizers has the following example code for removing a finalizer from an object. // remove our finalizer from the list and update it. controllerutil.RemoveFinalizer(cronJob, myFinalizerName) if err := r.Update(ctx, cronJob); err != nil { return ctrl.Result{}, err } I found when doing this, if the operation running above this in the Reconcile() function (typically to remove a remote resource from an API) this r.Update() call would return an error like this:...

If you are working with an application that sends email, you’ve probably come across “DKIM” which is cryptographic signing of emails to prevent impersonation of people within an organisation. Unfortunately AWS SES only supports DKIM signing for domain identities - not single-address identities. This means even if your application only sends email from a single address such as contact@example.com - to set up DKIM you need the entire example.com domain verified....

Running a fleet of Drupal projects presents a lot of challenges and problems to solve. Some things might fly for a handful of projects, but trying to scale that up will send you directly to the seventh circle of Drupal hell. Video

Terraform is an extremely powerful tool, but the out-of-the-box workflow can lead to pitfalls such as forgetting to commit and push changes to manifest and terraform.tfstate files. Use this example CircleCI configuration for a rock-solid pull-request workflow for Terraform projects. The Workflow A project contributor creates a pull request with changes to terraform manifests. CircleCI runs a terraform plan and posts the diff as a comment in the pull request....

Developers often need copies of production data for local development, and standard workflows for getting these dumps have a lot of downsides. Moving SQL dumps around means an increased risk of sensitive data being exposed. Most sanitization methods are a multi-step process. Large databases put strain on slow internet connections. Large databases take way too long to import. The Solution mtk-dump is an extremely powerful replacement for mysqldump. Define your sanitization and minification rules in a simple yaml file, and produce small and safe SQL dumps for development....

Recently I encountered an issue where a CI tool could not fetch certain dependencies as the SSH key used for cloning the main project did not have access to the other repositories. To work around this, I opted to clone the private dependencies via https, passing the credentials in the URI. The magic to force this without changing composer.json: # Set the COMPOSER_USER and COMPOSER_TOKEN environment variables to a user which has access to clone these repos....

I recently worked on a project to push our private docker images to GitHub’s package registry. Our CI only had a GITHUB_TOKEN environment variable set, but docker login requires a username too. Rather than add a new environment variable to the build and an assumption that the username and token had to match, I used this snippet to derive the username from the token and log in to the registry....

Video Slides

I found a neat python tool called s3wipe which brings significant speed improvements when deleting extremely large s3 buckets. It achieves this by using multiple threads and batch deletes. This really helped me out recently when deleting buckets containing several million objects and versions. Example Usage Empty a bucket of all objects, and delete the bucket when done. BUCKET_NAME=project-files-public docker run -it --rm slmingol/s3wipe \ --id ${AWS_ACCESS_KEY_ID} \ --key ${AWS_SECRET_ACCESS_KEY} \ --path "s3://${BUCKET_NAME}" \ --delbucket Remove all objects and versions with a certain prefix, but retain the bucket....

Recently I configured gitlab-runner to operate on an openshift cluster. One quirk of this setup is the containers running as random uids, having the side-effect of the build container being unable to clone the project or fetch dependencies. To overcome this issue, we needed to run the build container with a supplemental group which had write access to the $CI_BUILDS_DIR path. Security Context Constraint The cluster administrators set up a securityContextConstraint which allowed the build service account to use supplemental group 80001....