Thoughts of a software developer

22.09.2019 12:13 | Modified 28.03. 14:50
AWS CDK setup for autoscaling with ec2 spot instances

AWS Cloud Development Kit is an excellent new way to describe and code your infrastructure to cloud. On thing I hadn’t tried yet, was ec2 spot instances, and I thought I’d give it a go with cdk.

So, first I made a vpc with public- and private subnets. Public subnet would only have an application load balancer (ALB) which only listens for port 80 (for now, getting a certificate for more serious work later).

Autoscaling group in private subnet would then allow traffic from application load balancer and guide it through to ec2 spot instances. I’m describing the process below, the code is not totally here, just the important bits. You can see the whole project at Github.

The start of the cdk stack was this (rds part is not yet done):

First the VPC

const vpc = new ec2.Vpc(this, 'TheVPC', {
    cidr: "10.229.0.0/24",
    subnetConfiguration: [
        {
          cidrMask: 28,
          name: 'public subnet',
          subnetType: ec2.SubnetType.PUBLIC,
        },
        {
          cidrMask: 28,
          name: 'private subnet',
          subnetType: ec2.SubnetType.PRIVATE,
        }
    ],
    gatewayEndpoints: {
        S3: {
          service: ec2.GatewayVpcEndpointAwsService.S3
        }
    }
});

Pretty simple, public- and private subnets and a S3-gateway so that you can use S3 without going through public internet.

Building the app

We have a small go web server which just prints hello world.

func main() {
    http.HandleFunc("/", hello)
    http.ListenAndServe(":8080", nil)
}
func hello(w http.ResponseWriter, r *http.Request) {
    w.Header().Add("Content-Type", "text/html")
    fmt.Fprintf(w, message)
}
const message = `
    <html>
        </head></head>
        <body>Hello world!</body>
    </html>
`

We build and zip it with a separate bash script

#!/bin/bash
set -e
cd app
GOOS=linux GOARCH=amd64 go build -o app.linux
zip app app.linux

After that, we upload the zip to S3 with:

const asset = new Asset(this, 'appAssetToS3', {
    path: 'app/app.zip'
});

Public application load balancer

We make a ALB into public subnet and listen to port 80.

const lb = new elbv2.ApplicationLoadBalancer(this, 'appLoadBalancer', {
      vpc,
      internetFacing: true,
});
const httpListener = lb.addListener('httpListener', {
      port: 80,
      open: true,
});

Later on, we’ll bind the target port to 8080 and to autoscaling group where our ec2 instances will be at:

httpListener.addTargets('ApplicationSpotFleet', {
    port: 8080,
    targets: [appAutoscalingGroup],
});

Autoscaling

Now, this is the part where we create the autoscaling, ec2 instances and the decision to use spot instances. We are using t3.micro instances with newer Amazon Linux images. We are allowing all outbound traffic and the desired capacity of instances is one, while the maximum is 3. Maximum spot price is set to 0.009, if the sport price goes up, for example to 0.01, then we have no instances at all. There are ways to address this, but I’m leaving it at this point out.

var appAutoscalingGroup = new autoscaling.AutoScalingGroup(this, 'ASG', {
      vpc,
      instanceType: ec2.InstanceType.of(ec2.InstanceClass.BURSTABLE3, ec2.InstanceSize.MICRO),
      machineImage: new ec2.AmazonLinuxImage({
        generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
      }),
      allowAllOutbound: true,
      maxCapacity: 3,
      minCapacity: 1,
      desiredCapacity: 1,
      spotPrice: "0.009", // $0.0032 per Hour when writing, $0.0108 per Hour on-demand
      updateType: autoscaling.UpdateType.REPLACING_UPDATE,
      healthCheck: autoscaling.HealthCheck.ec2(),
});

We are scaling with cpu usage.

appAutoscalingGroup.scaleOnCpuUtilization('cpuScale', {
      targetUtilizationPercent: 10,
      cooldown: cdk.Duration.minutes(5),
      estimatedInstanceWarmup: cdk.Duration.minutes(1),
});

Getting the app to the ec2 instances

You can set user data for the autoscale object and get the app from S3 to the starting instance and execute it.

appAutoscalingGroup.addUserData(`
    #!/bin/bash
    yum update -y
    yum -y install unzip
    mkdir /app
    cd /app
    chown -R ec2-user:ec2-user .
    aws s3 cp s3://` + asset.bucket.bucketName+ "/" + asset.s3ObjectKey + ` /app/app.zip
    unzip app.zip
    ./app.linux
`);

You also have to give rights to be able to get the zip from S3

appAutoscalingGroup.addToRolePolicy(new iam.PolicyStatement({
    effect: iam.Effect.ALLOW,
    resources: [asset.bucket.bucketArn + '/*'],
    actions: [
        's3:ListBucket', 
        's3:GetObject', 
        's3:HeadObject', 
        's3:ListObjectsV2',
    ],
}));

Now that’s enough to get the application running, you can use the public cname which the application load balancer has, I put it as cname for cdk-spot-asg.jelinden.fi, so you can view the result at http://cdk-spot-asg.jelinden.fi/.

Github project aws-cdk-training

Working application: http://cdk-spot-asg.jelinden.fi/

(If and when the application is not working anymore, it just said Hello world!)

Since 2017