VPC Endpoint for S3

In the Architecting on AWS course we discuss a common VPC design, and that is my starting point for testing the VPC Endpoint feature. For this test we can use a single AZ, and we don’t need to use a NAT gateway. What we need is:

  • A public subnet
  • A private subnet
  • An IGW
  • A public route table with a default route targetting the IGW, associated with the public subnet.
  • A private route table with only the default entry in it, associated with the private subnet.
  • A bastion host in the public subnet
  • A private instance in the private subnet with a role allowing access to S3

For the instances I used the AMZ Linux AMI, and gave them a role to allow access to S3.

That is my starting point for testing the VPC Endpoint for S3 feature.

SSH into the bastion and then into the private instance. One way to do this is to use putty, agent forwarding enabled, with your keys loaded into pagent.

On the private instance, try the command to list all your S3 buckets.

aws s3 ls

It should not work. S3 has public endpoints, and to reach them, the private instance needs internet access. To have internet access would need a NAT GW in the public subnet, and a route to the NAT gateway in the private route table.

The VPC Endpoint feature came about because customers wanted a way to access S3 using private IP addresses, without needing to use a NAT gateway and without sending traffic over the public internet.

To create a VPC endpoint for S3, navigate to:

VPC>Endpoints>Create Endpoint>Select the S3 Endpoint>Select the VPC>Select the private route table>Create Endpoint.

Inspect the private route table. it will now have an entry to similar to this

Once again try the command.

aws s3 ls

It should now work. Traffic is using the AMZ private network.

To see what IP addresses correspond to the prefix-list (running the command from my Bastion Host)

It is now possible to use the same idea to reach other services, such as the ec2 service in the command above), and services provided by a third party, using something called interface endpoints. Thats another story!

IPv6 in your VPC

Nothing to do with the course or the exam, but just for fun I configured IPv6 in my VPC.

As a starting point I used a completed lab of Architecting on AWS. The final lab is as follows:

So its the classic 2 public, 2 private subnets across 2 AZs, with an ALB handling incoming traffic to the Web App. To test the incoming traffic, we browse to the DNS name of the ALB. To test the Nat Gateways, the Web App is querying an internet site freegeoip for its coordinates. The public IP we see is the EIP of one of the Nat Gateways.

Copying the coordinates (removing the “/”) into Google Maps shows that the IP is located in the middle of a canal in Dublin.

Which just happens to be about 200m from Amazon Ireland.

Lets change to using ipv6 for this outgoing traffic.

VPC>Actions>Edit CIDRs>Add ipv6 CIDR

We get a CIDR range for the VPC.

2a05:d018:1d6:1e00::/56

Its always a /56, which means we have 8 bits to assign a unique prefix to each subnet, which means in theory we can have 256 subnets, at least according to the IPv6 rules.

The addresses are from a public range, as is usual with IPv6.

For each Subnet:

Actions>Edit IPv6 CIDRs>Add IPv6 CIDR. You will be presented with something like this:

2a05:d018:1d6:1e00::/64

and you are prompted to enter the last 2 digits of the prefix, that is the bit before the double colon. This uniquely identifies each subnet, and you could use the range 00-03 for the 4 subnets.

For each EC2 instance:

Actions>Networking>Manage IP Addresses>Assign IPv6 Address

You get something like this:

2a05:d018:1d6:1e03:10b4:f418:a79:76ec

The last 64 bits is the auto-generated interface ID.

According to the rules of IPv6 we could have 2 to the 64 addresses in the subnet.

For the Load Balancer:

Actions>Edit IP address type>dualstack

Now the LB has an IPv4 and an IPv6 stack.

For the route tables associated with the private subnets, add a route ::/0 and choose the internet gateway as the target.

The security groups need to allow outbound traffic. In this case mine was already wide open, it had a rule ALL Traffic ::/0.

Refresh the web page. it worked!

The IPv6 address we see is now the IPv6 address of the EC2 instance.

We no longer need the Nat gateways, so we can save about 8 pence an hour!

Now the private subnets are no longer private as far as IPv6 is concerned. To increase security we could use an “Egress Only Internet Gateway” or tighten up the security groups.