In summary
A high level summary of what's to come.
After the planning comes the doing. Every project is different, but my general approach to what I do is detailed here.
It’s worth noting that below is a high-level summary, and while it seems logical that this all happened in the order I have written, it didn’t. Issues came about, or I revisited previous deployments as I learnt more about Terraform and refined my code.
Prerequisites
There was already a number of services set up and in existence including Azure, Azure Devops, MongoDB, Cloudflare and Datadog, the missing piece was Terraform.
Following the companies ethos of cloud first, we made the decision to use the Hashicorp Cloud Platform to host and manage our Terraform infrastructure.
Using Terraform with a central state file managed in a storage account, for example, might be fine for a small company wanting to try Terraform out, but we were underpinning our entire infrastructure on this.
There are challenges when using a storage account and a central state file, including state file locking and corruption, as well as needing careful collaboration amongst a team. There are also issues around security and version control. While I’m sure there are means to mitigate some of these, there is also a perfectly fine solution hosted in the cloud.
We set up our Organisation within the HCP portal and configured a VCS Provider to integrate with a repository in Azure DevOps, which would be used to centrally store configuration files and allow the company to manage and run triggers, collaborate more effectively, and ensure that security was at the forefront of what I was doing.
Building & Deployment
This is where the plans started to come to life. Technology and services were deployed, tested, checked, and refined.
It took me 12 months to get to a point on Greenfield where I had something tangible to show. Being the lead engineer working full-time on the project, I was familiar with a lot of the technology and services, but because I was using Terraform for the first time, the period it took to deploy almost quadrupled.
This is a summary of the stages I went through.
Admin Work
The first work I completed was more administrative, and working out both the Technical and Environment stacks, as well as a naming convention for the stacks and resources.
Terraform
When it came to Terraform, I set up some projects and workspaces that corresponded with my stacks.
I also decided what I would be setting Modules up for (basically reusable code)
Finally, I looked at how to set up a cloud agent because this was a private infrastructure, and I knew that once it was all firewalled off, Terraform needed a way in and out.
Azure
I opted to work from the top down within Azure, and some things made more logical sense to complete first before I moved onto other tasks. But then, as I learnt more about Terraform and coding, I went back and refined what I had done.
Management
I started with access permissions for Azure Management Groups before working out what the rough RBAC & IAM would involve.
As the work went on and resources were set up, I made a note of who might need access and what access they should be given.
Networking
The beauty of starting at the beginning is that the address space is pretty much free. All I needed to do was allocate the address spaces to each virtual network (along with the subnets carved out) and also address spaces for the VWAN and VPN pools.
I also worked out the internal address pools for our AKS clusters.
With the implementation of a firewall, I also set up some IP groups that would whitelist certain ranges internally and externally.
With this admin done, I set up the VWAN, a secure hub (firewalled) and a VPN gateway.
Then I went about setting up an Operations virtual network to start connecting internal resources. The stack virtual networks would be done later with individual environment.
I also reviewed Private DNS Resolver vs a virtual machine with the DNS role installed on to handle internal DNS resolution and determined what Private DNS Zones would need to be set up and linked in.
Operations
As well as networking services, I also deployed some central services such as Key Vaults and Storage accounts that could be used across the company, Update Manager for all virtual machines and a Log Analytics Workspace for auditing.
With this, I started setting up Modules in Terraform for Key Vaults and Storage Accounts.
Our Platform Team had developed a number of tools used internally so I set up the infrastructure so they could migrate them onto it.
Recovery Services
I set up some initial Recovery Vaults for virtual machines and storage accounts in Operations and for any PROD services.
Monitoring and Alerts
Within Azure, I set up email alerts for certain firewall, VWAN, VPN, and IP Group activities. I also set up some audits for storage accounts and network watchers.
Compute
I created a Terraform Module for Virtual Machines and then started building out some virtual machines with it. One being used to host DNS services.
I also created a module for Azure Kubernetes clusters with additional Helm deployments for Ingress and Datadog.
SFTP
I designed the SFTP service and architecture, set up the NonProd and Prod services, and completed successful testing. I also determined who would manage and support SFTP and how best to facilitate that.
App
This involved deploying the relevant services for the Technical stack and then the Environment stack (more details on this here). Five Tech stacks with 18 environments split across them as Environment stacks.
Once the first DEV Tech and DEV1 Environment stacks were deployed, it was then handed over to the Platform team for initial testing of the functionality and ability to deploy our applications. Then, we allocated access to the relevant internal teams for further testing of DEV and some feedback before starting the migrations.
Agents
We use Virtual Machine Scale Sets as part of our pipeline deployment, so I had to set up an infrastructure to host that and push out the VMSS the Platform team needed.
As part of that, I set up Azure Devops Service Connections into Azure to allow the pipelines access to various KV secrets, etc
DNS
Several external DNS records needed to be reviewed as they determined what needed to be changed come migration.
MongoDB
As part of the Environment stack, I needed to deploy a Project & Cluster as well as set up private link access into our private infrastructure. I also needed to ensure a backup was completed of the existing Cluster so it could be migrated onto the new Cluster.
Datadog
A lot of this was already in place, I just needed to ensure it all worked once the agents were deployed.
After 12 months of changing and refining the above, I completed a few demos with the team. I then refined it more until I felt it was ready to show off to the rest of the company.
Eventually, I deployed the DEV technical stack and then the DEV1 environment stack, and the platform team deployed the application with success. It was handed over to the internal teams to play with and provide some user feedback.
As of writing, the migration process is still ongoing.
I hope you enjoyed the first parts of my Greenfield Project Diary.
There will be more to come, where I will detail the technical aspects of the work of the above, so stay tuned.