Skip to content

Who's using my certificates?

A few weeks ago, aws informed me that it was unable to renew some of my ACM, naturaly because I moved it all over to Cloudflare a few months ago. Things got interesting when I tried to clean it up..

The backstory

A few years ago when I started my journey in cloud computing, I decided to move this website to aws. I experimented with a few things, running Wordpress on EC2 at one point, and then ending up using AWS S3 with a Cloudfront distribution. This pattern worked quite well for a number of years.

I used services like Digital Ocean before, and one thing that stood out from them, is I never had to pay for hosting my DNS zones. This site's DNS fees came in around $2 per month, $1 for hosting the zone, and the rest for serving the DNS entries. That's not a lot, and I can afford it, but it stuck with me -- why should I be paying AWS for hosting, when other companies offer it as part of their service?

Moving on from DNS, I am hosting a statically generated site with MKDocs. AWS S3 worked great for it. Again, like anything in AWS, nothing is ever free. The volume of data hosted in the S3 bucket is in the MB (not even GB), yet AWS would chage me (in the cents) for hosting the site, and API calls to the S3 bucket.

I tried using Github Pages, and I found it a bit convoluted. Cloudflare has a similar service, also called Pages. I did a bit of testing, and found the Free site on Cloudflare offers me unlimitd bandwidth for my site (did I mention it was free?).

I moved the DNS, the web hosting, and the web redirection all to Cloudflare. As it stands right now, the hosting of this website is completely free. The only fee I pay every year is the domain registration,which still resides with AWS.

So what's with the AWS certificates?

When the domains moves to Cloudflare, so did the SSL/TLS certificates. They are managed and generated by Cloudflare, so no effort or input needed on my part. Through the migration process, I neglected to remove the old certificates - they remained in AWS. That in itself is not a problem, until a few weeks ago when AWS dropped me their automated email to inform me that they were unable to renew the certificate. This is because they are trying to validate the certificate against the DNS domain, and the required entries they are looking for to validate the certificate does not exist.

I logged onto my AWS console, headed over to AWS in us-east-1 (where the global certificates are stored), and there they are.... It should be simple to delete the certificate... Just click, and "Delete", but no...

Certificate in use

1 of the 1 selected certificates to delete is in use (associated with other AWS resources) and cannot be deleted. Disassociate the certificate below from each of its associated resources and try again.
* massyn.net (8c4f2137-af3e-45a2-9a76-36aafff7e7a5)

What's using it?

In the certificate manager, it shows what the associated resources are. This is where it got interesting. It showed this.

arn:aws:cloudfront::856456143310:distribution/E323BF8AQVJCNN

Now I know what you're thinking.... Phil, you muppet! You forgot to remove the Cloudfront distribution! If only... What the arn shows you, is that the distribution is owned by account 856456143310 - this is not my AWS account. What is odd, is both of the domains in ACM are used by the same AWS account.

How did it get there?

I opened a case with AWS Support to investigate this (though I am not expecting any feedback anytime soon since my $5p/m account does not really qualify as an enterprise level supported account)

Here's what I do know...

  • ACM does not support resource level permissions like S3, SQS or Lambda, so it's not that.
  • No resources are shared via the Resource Access Manager
  • A Cloudfront distribution from outside my control has somehow managed to latched itself onto a certificate in my AWS account.

Where to from here?

~~I cannot delete the certificates because they are "in use", and I can't detach them because I don't have access to the cloudfront distribution that is associated to the certificate. Let's see what happens with the AWS Support Case.~~

AWS Support got back to me! The "rogue" resources were in fact AWS Cognito. At some point around 3 years ago, I was testing Cognito, and configured a custom domain to the user pool. What was odd, was that ACM never indicated the resource in use was Cognito maskerading as Cloudfront. At least the mystery is solved.

Why does it work like this?

If you've worked with AWS for a while, you'll realise that AWS creates core services, things like S3, Lambda and EC2 that does one thing, and does it well. They then create these (shall we call them "secondary" services?) like Elastic Load Balancers (ELBs) that effectively are EC2 instances, wrapped around another service. Quite a few services in AWS work like this. Cognito, for example, is effectively a wrapper for CloudFront, DynamoDB, S3 and Lambda, wrapped into a new service.

I'm glad AWS support was able to help me track this down. It would be great if they could include this into the ACM console, to clearly show that Cognito is the service using the certificate.