Terraform on-premise with Spinifex: a complete guide

Most Terraform configs that target AWS are more portable than their authors expect. The resource definitions, modules, variable files, and state are all completely decoupled from which endpoint the provider talks to. When you point the AWS provider at Spinifex, a fully AWS-compatible cloud platform running on your own hardware, the resources you've already written apply without modification. What changes is a single block in your provider configuration.

This post covers the Terraform-specific details: the two-endpoint architecture, the provider flags that assume real AWS, state management, and the workbooks in the Spinifex repository that cover common infrastructure patterns.

The two-endpoint architecture

Spinifex exposes two endpoints. The main AWS gateway runs on port 9999 and handles EC2, IAM, STS, EKS, ECR, ECS, VPC, ELB, and other compute and identity services. S3-compatible object storage is served by Predastore on port 8443. The AWS provider routes different services to different endpoints, and S3 requires an additional configuration flag.

A provider block targeting compute services only looks like this:

variable "spinifex_endpoint" {
  type    = string
  default = "https://127.0.0.1:9999"
}

provider "aws" {
  region = "ap-southeast-2"

  endpoints {
    ec2 = var.spinifex_endpoint
    iam = var.spinifex_endpoint
    sts = var.spinifex_endpoint
  }

  skip_metadata_api_check = true
  skip_region_validation  = true
}

When your config also provisions S3 resources, add the Predastore endpoint and enable path-style addressing:

variable "predastore_endpoint" {
  type    = string
  default = "https://127.0.0.1:8443"
}

provider "aws" {
  region = "ap-southeast-2"

  endpoints {
    ec2 = var.spinifex_endpoint
    s3  = var.predastore_endpoint
    iam = var.spinifex_endpoint
    sts = var.spinifex_endpoint
  }

  s3_use_path_style           = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true
  skip_region_validation      = true
}

The s3_use_path_style flag is required because Predastore uses path-style bucket addressing (https://host:8443/bucket-name) rather than virtual-hosted-style (https://bucket-name.host). Without it, the provider will generate request URLs that Predastore does not recognise.

The skip_* flags

The AWS provider runs a few checks before it sends a resource request. Two of them assume real AWS, and every Spinifex workbook turns them off:

skip_metadata_api_check: skips looking for credentials in the EC2 instance metadata service, which only exists when Terraform runs inside a cloud instance
skip_region_validation: skips the check that the configured region is one AWS publishes

The provider still validates your credentials. Spinifex implements STS, so the sts:GetCallerIdentity call the provider makes at start-up succeeds against the gateway the same way it does against AWS.

Configurations that manage S3 need two more flags, skip_credentials_validation and skip_requesting_account_id. Without them, AWS provider 6.x reads bucket tags through S3 Control, whose endpoint puts your account ID in the hostname and cannot resolve against an IP address. Set both together: skip_requesting_account_id on its own has no effect, because the credentials check supplies the account ID first.

Passing endpoint addresses

The defaults in the variable declarations (https://127.0.0.1:9999 and https://127.0.0.1:8443) are useful for running Terraform locally on the same machine as Spinifex but won't work when Terraform runs on a separate host. Override them with a terraform.tfvars file:

spinifex_endpoint   = "https://192.168.1.10:9999"
predastore_endpoint = "https://192.168.1.10:8443"

Or via environment variables, which avoids committing host addresses to source control:

export TF_VAR_spinifex_endpoint="https://192.168.1.10:9999"
export TF_VAR_predastore_endpoint="https://192.168.1.10:8443"

The environment variable form works well in CI pipelines where the Spinifex host address is an infrastructure secret that shouldn't appear in repository files.

AMI data sources

Terraform configs that look up AMIs with data "aws_ami" need one adjustment. AWS AMIs are owned by specific AWS account IDs, but Spinifex uses 000000000000 as the placeholder account for imported images. The filter block looks like this:

data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["000000000000"]

  filter {
    name   = "name"
    values = ["*ubuntu-24.04*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

Configs that filter by a real AWS account ID (such as 099720109477 for Canonical's official Ubuntu AMIs) won't match any Spinifex images. Changing owners to ["000000000000"] is the only modification required.

State

Local state works with Spinifex exactly as it does with AWS. The documented workbooks all use a local state backend:

terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.0"
    }
  }
}

Remote state using Predastore as an S3 backend is not covered in the documented workbooks. If you need shared remote state for a team workflow, use a local backend with a shared filesystem or an external state store such as Terraform Cloud. Plan for this before you're managing non-trivial infrastructure with multiple contributors.

Credentials

Terraform picks up credentials from the same places as the AWS CLI: environment variables, the ~/.aws/credentials file, or an instance profile if Terraform is running on a Spinifex-launched instance. After spx admin init, the profile named spinifex-<nodename> has the access key and secret for the node. Set AWS_PROFILE=spinifex-<nodename> in your environment and the provider will use it without any credential configuration in the HCL.

The workbooks

The Spinifex repository includes a set of Terraform workbooks that provide complete, working configurations for common infrastructure patterns. They're organised under docs/terraform-workbooks and cover:

bastion-private-subnet: a VPC with public and private subnets, a bastion host, and a private instance behind it
s3-webapp: a Flask application on EC2 that reads from Predastore using IMDS credentials via an instance profile, with no long-lived S3 keys baked into the instance
nginx-webserver: a simple EC2-hosted web server behind an Application Load Balancer
nginx-alb: a minimal ALB configuration for routing traffic to EC2 instances
rds-quickstart: a VPC, subnet group, and RDS instance connected to an EC2 app server
ecs-quickstart: a full ECS stack including VPC, IAM, cluster, task definition, container instances, and an ALB-fronted service
eks-quickstart: an EKS cluster with a node group, suitable as a starting point for Kubernetes workloads
eks-https-ingress: extends the EKS quickstart with private subnets, a NAT gateway, and HTTPS via the AWS Load Balancer Controller
eks-gitops-argocd: adds Argo CD and EBS-CSI persistent storage to the EKS cluster

Clone just the workbooks directory with a sparse checkout to keep the download small:

git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform-workbooks

The pattern for running any of them is the same:

export AWS_PROFILE=spinifex-<nodename>
cd docs/terraform-workbooks/<workbook-name>
tofu init
tofu apply

The workbooks use OpenTofu (tofu) as the CLI, which is a drop-in replacement for terraform and is compatible with all Terraform configuration syntax and provider versions. If you prefer terraform, substitute it directly.

Any Terraform config you run against AWS today can be pointed at bare metal with one provider change.

Get started

Full platform documentation, including the Terraform workbook references, is at docs.mulgadc.com. The source is on GitHub. Or sign up for the free sandbox to run Terraform plans against a live Spinifex endpoint before deploying on your own hardware.

Using Terraform to Manage On-Premise Infrastructure