VPC Endpoint for S3

In the Architecting on AWS course we discuss a common VPC design, and that is my starting point for testing the VPC Endpoint feature. For this test we can use a single AZ, and we don’t need to use a NAT gateway. What we need is:

  • A public subnet
  • A private subnet
  • An IGW
  • A public route table with a default route targetting the IGW, associated with the public subnet.
  • A private route table with only the default entry in it, associated with the private subnet.
  • A bastion host in the public subnet
  • A private instance in the private subnet with a role allowing access to S3

For the instances I used the AMZ Linux AMI, and gave them a role to allow access to S3.

That is my starting point for testing the VPC Endpoint for S3 feature.

SSH into the bastion and then into the private instance. One way to do this is to use putty, agent forwarding enabled, with your keys loaded into pagent.

On the private instance, try the command to list all your S3 buckets.

aws s3 ls

It should not work. S3 has public endpoints, and to reach them, the private instance needs internet access. To have internet access would need a NAT GW in the public subnet, and a route to the NAT gateway in the private route table.

The VPC Endpoint feature came about because customers wanted a way to access S3 using private IP addresses, without needing to use a NAT gateway and without sending traffic over the public internet.

To create a VPC endpoint for S3, navigate to:

VPC>Endpoints>Create Endpoint>Select the S3 Endpoint>Select the VPC>Select the private route table>Create Endpoint.

Inspect the private route table. it will now have an entry to similar to this

Once again try the command.

aws s3 ls

It should now work. Traffic is using the AMZ private network.

To see what IP addresses correspond to the prefix-list (running the command from my Bastion Host)

It is now possible to use the same idea to reach other services, such as the ec2 service in the command above), and services provided by a third party, using something called interface endpoints. Thats another story!

IAM Policy Simulator

Lets say a user sees an Access Denied message when trying to access a resource, for example S3.

This is likely the result of a policy applied to the user, or the users group, or it could be a bucket policy. Or it could be that the user or group has no explicit permissions at all.

One way to troubleshoot the issue is to use the IAM Policy Simulator.

I am already logged in as an admin, and it displays a list of users in my account.

Select the user, and simulate an action, here I select S3 as the service.

For the actions, I select “All Actions” (you can also choose a specific action on a specific bucket) and click Run Simulation.

It will list which actions are allowed or denied. You can drill down on any denied action and it will display the policy and even the section of the policy which is denying access.

In this case the user was in a group with the “AdministratorAccess” policy, but the above policy applied to the user was overriding the Allow.