Restrict FROM address with AWS SES domain identities

If you are working with an application that sends email, you’ve probably come across “DKIM” which is cryptographic signing of emails to prevent impersonation of people within an organisation. Unfortunately AWS SES only supports DKIM signing for domain identities - not single-address identities. This means even if your application only sends email from a single address such as contact@example.com - to set up DKIM you need the entire example.com domain verified....

November 9, 2022

Cleanup Large S3 Buckets

I found a neat python tool called s3wipe which brings significant speed improvements when deleting extremely large s3 buckets. It achieves this by using multiple threads and batch deletes. This really helped me out recently when deleting buckets containing several million objects and versions. Example Usage Empty a bucket of all objects, and delete the bucket when done. BUCKET_NAME=project-files-public docker run -it --rm slmingol/s3wipe \ --id ${AWS_ACCESS_KEY_ID} \ --key ${AWS_SECRET_ACCESS_KEY} \ --path "s3://${BUCKET_NAME}" \ --delbucket Remove all objects and versions with a certain prefix, but retain the bucket....

September 20, 2019

How to generate temporary download links to S3 objects

S3 has a feature which allows you to generate signed URLs which are valid only for a predefined period of time. This makes it much safer to distribute URLs via email/slack etc.. Process Find the object in the S3 console and note the bucket name and object path. Ensure your AWS credentials are loaded into your environment. Use the AWS CLI to create a pre-signed URL: # TTL is the number of seconds until the URL expires....

March 5, 2019

AWS KMS cryptographic operations on the command line

AWS KMS is a managed service for cryptographic functions in AWS. This service allows you to offload the tough job of key lifecycle management to Amazon. These snippets will allow you to perform basic cryptographic functions - encypt, decrypt, and rewrap. It is assumed you already have a KMS key provisioned, and you have a IAM user with permissions to perform the relevant operations. Encrypt the Contents of a File This command encrypts the contents of contents....

February 28, 2019

Encrypted Drupal Database Connections with Amazon RDS

Originally posted 2018-08-08 on the PreviousNext blog. With attackers and data breaches becoming more sophisticated every day, it is imperative that we take as many steps as practical to protect sensitive data in our Drupal apps. PreviousNext use Amazon RDS for our MariaDB and MySQL database instances. RDS supports SSL encryption for data in transit, and it is extremely simple to configure your Drupal app to connect in this manner....

August 8, 2018

How to build self-deploying applications with Terraform and BitBucket Pipelines.

Background A few weeks ago I decided to replace my ageing and bloated Drupal 7 blog. I decided on the following criteria that the solution had to meet: The project git repo must be private. Hosting infrastructure had to be under my control and completely codified. The solution should require very little supporting infrastructure such as databases. Deployment of changes to the site or infrastructure must be automated. These requirements immediately ruled out a few options including GitHub Pages and SaaS blogging platforms like wordpress....

November 17, 2016

How to peer two AWS VPCs using Terraform

During a recent project, I ran into an issue where the new stack (defined using Terraform) needed to connect to a legacy server via private network interfaces. The Terraform stack was encapsulated within its own VPC, and the legacy server was in the default VPC. Both VPCs were in the same region. AWS has a feature called “VPC peering” which establishes a connection between two VPCs - exactly what was required for this use case....

November 15, 2016