Implementing Secure, Streamlined Kubernetes Infrastructure for NYK

The Solution: Infrastructure as Code and Controlled CI/CD

The solution centred on leveraging Microsoft Azure services, codified infrastructure management via Terraform, and enforced deployment controls through Azure DevOps Pipelines.

1. Foundational Infrastructure (Terraform)

Given NYK’s existing usage of Azure and the team’s Kubernetes expertise, Azure Kubernetes Service (AKS) was selected as the natural fit for hosting the containerised application.

Terraform was used extensively to provision a private AKS cluster and all supporting elements, strictly adhering to the principles of least privilege. The comprehensive set of resources provisioned using Terraform included:

• Networking components: Subnet, Network Security Groups (NSG) and NSG assignment, Private DNS Zone, VNET Link, AKS network assignment, AKS DNS Zone, AKS VNET link, and Bastion Configuration.
• Core AKS resources: AKS cluster and AKS node pool.
• Identity and Registry: User assigned identity, DNS role assignment, Container Registry (ACR), and AKS ACR role assignment.
• Other services: Azure Cache for Redis, Keycloak disk, DNS A Records (including those required for the database).

Architectural Refinement: Initial plans included using an App Service and a Container App. However, the team made the strategic decision to consolidate workloads by hosting the NextJS front-end application within the AKS cluster instead of the App Service. This decision was justified because:

• Monitoring: The least privilege restrictions made App Service deployment difficult to monitor, and there was no real visibility of the necessary logs. Moving the workload to AKS provided more control and effective log monitoring.
• Deployment Simplicity: The deployment of the App Service using Terraform was perceived as overcomplex, making the full migration to AKS a simpler process.
• The planned long-running asynchronous task was successfully trialled running within the AKS cluster, negating the need for the Container App.

2. Governance and Deployment (Azure DevOps Pipelines)

To manage all changes securely, Azure DevOps pipelines were utilised. While the business had experience with Jenkins as a platform-agnostic DevOps tool, Azure DevOps pipelines were chosen for this project due to their speed, simplicity, and inherent compatibility within an Azure environment.

Pipelines were essential for maintaining the principles of least privilege:

• Access Control: Individual users lacked the authority to run commands directly in the environment; only the DevOps service connection held the required authority.
• Auditability: Using the Azure DevOps Git repository ensured that only authorised users could modify the deployment scripts and run the pipelines. Furthermore, all changes were tracked and required explicit approval.

The pipelines executed several critical functions:

• Infrastructure Provisioning: Provisioning both production and non-production environments using Terraform (with the state securely stored in Azure Storage Accounts).
• Workload Deployment: Deploying and managing workloads using Helm.
• Image Management: Moving Docker images from the central repository to the relevant Azure Container Registry (ACR).
• Environment Management: Running necessary commands to help manage the environment.
• Secrets Management: Azure Key Vault was used for secrets management. Theses were then transformed into Kubernetes secrets using the Azure Key Vault provider for Secrets Store CSI Driver.

The Outcome: Enhanced Security, Control, and Observability

The implementation of Terraform and Azure DevOps pipelines resulted in a secure, auditable, and highly operational environment for NYK.

1. Enhanced Security and Compliance

By enforcing all changes through pipelines and leveraging the DevOps service connection, the project successfully upheld the principles of least privilege. This ensured that human error or unauthorised direct access could not compromise the private AKS environment, as all modifications were tracked and approved via the Git repository.

2. Improved Operational Monitoring

To successfully track the performance of Kubernetes workloads, Azure Managed Grafana was enabled.

This tool collects metrics and allows for the creation of dashboards that clearly display relevant workload performance statistics, providing immediate insight into environment health (as shown in the provided example dashboard).

Crucially, alerts were configured based on these collected metrics, ensuring that support teams are informed immediately if any performance metric shows signs of degradation.

3. Simplified Architecture

The decision to remove the App Service and consolidate the front-end application within the AKS cluster not only simplified the deployment process (making the Terraform configuration less complex) but also provided the necessary control to monitor logs and track potential issues more effectively within a single platform.