Serverless best practices
Finally I got some time to watch AWS re:Invent 2022 this week. This blog is what I learned so far from the talk Best practices for advanced serverless developers given by Julian Wood, the senior developer advocate, Serverless in AWS.
To begin, I’d like to disclaim that serverless is the domain I am especially not familiar at. Notwithstanding the high quality of the talk, I could only rephrase the bit that I understand, which will definitely not fully summarise the talk in whole. I would suggest you watch the video by yourself if you are really interested.
Ok, without further ado, here’s what I learned:
1. Consider serverless as a mindset, not a technology
When I think of serverless, I always relate it with architectural decision, API gateway, event driven design etc. This is not wrong, but Julian points out this mindset is not why serverless is invented. Many people including me would instantly think Lambda when it comes to serverless on AWS, but interestingly, AWS didn’t mention serverless when it released Lambda in 2016, and AWS has been fond of the idea of serverless way earlier back in 2006 when it launched S3. One AWS fun fact I found from the video is that AWS S3 and SQS were actually released earlier than the prevailingly popular AWS product EC2. Julian used S3 as an example to illustrate the value of serverless. It is just a method to allow customers using the cloud resources more easily. It’s like Wi-Fi or wireless. People don’t feel the wire, and they just use the network, and that is the vision of serverless.
So, first of first, the first best practice of using serverless is to switch the mindset. When it comes to the why of choosing serverless, it should not be a decision because of technology. The choice of serverless (at least prospectively) allows the development team focusing more on the business and increase their agility for the changes of the market.
In another perspective, serverless can also be seen as an operational model. It does not mean there would be no ops, but the ops would be different.
Choosing serverless allows us building software IN the cloud, from which we can take advantages of the security, agility and scale, whereas software would be built ON TOP OF the cloud otherwise.
2. Lambda ≠ Serverless on AWS
As mentioned earlier, AWS Lambda and serverless on AWS are often equalized. People may argue managed services like S3 and SQS are still somewhat different from the software code that we write by ourselves. That’s totally true, but even when we talk about the actual software code, the statement that Lambda does not equal to serverless on AWS is still true.
Julian used the example of moving a software on-premise to cloud. Typically the software consists of authentication, caching, throttling, error handling, retry logic, the business logic and more. As I thought moving serverless would be all in Lambda, these components would be all in Lambda too. This is another bad practice Julian pointed out, as the complexity is not reduced, but just moved. The better way to migrate these components is to choose the right service for them (e.g. API Gateway for authentication and throttling, S3 for frontend…), and only keep the necessary logic in Lambda, and sometimes you don’t really need to use Lambda.
This rings a bell on EC2 as well. Similarly, many people also think using AWS means moving the server from data centre to AWS EC2 and possibly leveraging S3 in some cases. I have seen a few cases where EC2 instances were provisioned for everything even for database. To be fair, it is not worse than the situation in data centre, and the overhead of DC management is avoided. Moreover, the elasticity of cloud is also a benefit. Nonetheless, I believe people with some cloud experiences will find this ridiculous.
3. Use a framework
I am not sure how long before this will become the go-to solution in the serverless field as other software fields like frontend or backend, but in the year of 2022, using a framework is indeed suggested by AWS for serverless.
AWS has introduced an open source serverless framework named SAM (Serverless Application Model) , which enables template reusability, local testing and fast deployment at ease.
SAM consists of quite a few different sub-services, which seems to worth a deeper dive very much.
Other takeaways
Apart from the best practices, I also learned a couple of other new things about serverless on AWS:
- EDA (Event Driven Architecture) is not always good. It has more moving parts which make dependencies identification and more things harder. It is different in terms of testing and observability as well. It also brings about eventual consistency.
- To make perfectly decoupled architecture, the event consumer should ideally not depend on the event provider when it consumes the event. For example, if the event passed by the event broker contains some transaction ID instead of the transaction detail, the consumer would still need to make a GET request to the upstream service to get the information corresponding to that ID to consume the event. This is still coupled architecture. To decouple them, the event-carried state transfer pattern can be used. This pattern enriches the event with the necessary information the downstream consumers will use, so no dependencies are created when consuming the event. This will involve some discretion on what data the consumers will need and cautions not to inadvertently expose confidential information (e.g. PII)
- There are two ways to organise the serverless software: orchestration and choreography. The examples for them are respectively AWS Step Function and Amazon EventBridge
Links grabbed to be visited later
Slide decks and resources of the talk:
Other resources: