Notice AWS pricing

For most AWS certified solution architects, I think it’s easy to think of transitioning rarely accessed S3 objects to cheaper storage classes, such as Standard-IA (Infrequent Access) or even Glacier. I recently did same thing when I redesigned my blog access log structure, but after I added those beautiful lifecycle rules, I did another price calculation and deleted the transitioning rule.I thank myself for doing this extra calculation.

Here is one of beautiful transitioning rules:

resource "aws_s3_bucket_lifecycle_configuration" "cloudfront_log" {
  bucket = aws_s3_bucket.access_logs_bucket.id

  ......

  rule {
    id     = "blog-transition-partitioned-logs-to-glacier-after-30-days"
    status = "Enabled"

    filter {
      prefix = "AWSLogs/${data.aws_caller_identity.current.account_id}/CloudFront/Blog/Partitioned/"
    }

    transition {
      days          = 30
      storage_class = "GLACIER"
    }
  }
}

This rule move the access logs from S3 Standard storage class to Glacier Flexible Retrieval class after 30 days. When I did this, I thought this will help me save money in the long run because I rarely need to revisit historical logs. Maybe only when I do yearly review, I will need to check them again, but that kind of situation could only happen a few times throughout many years. This kind of access pattern seems to match the use case of transitioning to cheaper storage class, so I started picking what storage class suits my need.

I consulted my best pal ChatGPT, and it gives me a summary of each storage class like this:

So my thinking point was that manual restore is acceptable to me since I won’t use Athena. If I do any data analysis, I would most likely just to download a copy (actually now I am giving it a second thought, and this appears to be a bad idea). I am not that patient to wait for hours, so Glacier Flexible Retrieval sounds fine to me.

Then I implemented the transitioning rule as above, and I noticed a catch for minimal storage duration , which I think is fine. In the linked AWS documentation, we can see that most storage classes come with a min storage duration. In my case, I am transitioning from S3 Standard to Glacier Flexible Retrieval on the 30th day of the creation, while the minimal storage duration for Glacier Flexible Retrieval is 90 days. This means, after the transitioning, if the object is deleted prior to the 90-day minimal duration, I am still billed for 30 days of Standard storage + 60 days of Glacier Flexible Retrieval storage. This came to my awareness when I made this rule, but I don’t think it affects me because I don’t expect to delete logs prior to the minimal storage duration anyway.

What I wasn’t aware are these fees: one-time lifecycle transition fee and data retrieval fees for Glacier class.

According to AWS S3 Pricing page, in the “Requests & data retrievals” section, we can see those fees. Here is a summary for Singapore region:

So for my case, I did a quick math:

During 2023/11/27 - 2026/03/08, 93925 objects (90603371 Bytes or ~90MB) have accumulated in my bucket. If I transition them all into Glacier Flexible Retrieval, the transitioning fee itself will be 93925/1000*0.036=3.3813. However, the storage price between S3 Standard and Glacier Flexible Retrieval isn’t significant at all at my scale. S3 Standard is at $0.025 per GB for the first 50 TB / Month, which will definitely be my pricing tier for the whole lifetime. S3 Glacier Flexible Retrieval is $0.0045 per GB. Therefore, if I transition those log files to Glacier, the storage cost will be reduced by 0.025-0.0045=0.0205. But this is exactly where the catch is. For the transitioning action itself, I need to pay ~$3. Let alone there is potential data retrieval fee too. So in my context, transitioning data storage class isn’t a worthy investment at all.

This made me learn that common AWS practice doesn’t always apply to all situations. Perhaps this is particularly true for hobbyist projects where the traffic is extremely low. I am not sure if the math would be different for CloudFront distribution access logs if the traffic is very high. For my blog, I am definitely way lower than that tipping point. So, cloud is easy to use, and it’s tempting to make things fancy, but do that math wisely. You might not always need it.