Sharing a distributed computing system design from a real software problem

TL;DR

Re-engineering PDF generation on AWS Lambda improves efficiency and performance significantly.

Transcript

all right i wanted to share a little real life work example that i've been kind of tackling with um a group of other developers and i think this example is a good overview of like more complex problems that a beginner might not really understand and i wanted to kind of walk you through like what we're trying to do on our my project and how we're tr... Read More

Key Insights

• ⌛ AWS Lambda is limited by execution time, making it crucial to design solutions that distribute tasks efficiently.
• ✋ Asynchronous programming enhances performance, especially in high-demand scenarios, by breaking down single tasks into parallel processes.
• 😶‍🌫️ Using queuing systems like SQS facilitates better resource management in cloud environments by handling task overloads gracefully.
• 😶‍🌫️ Monitoring and managing system limits is essential in cloud architectures to avoid service throttling and ensure operational integrity.
• 🥺 Combining various AWS services can lead to a sophisticated architecture that supports complex workflows and improves overall efficiency.
• 👤 It is beneficial to implement limiters in systems to handle sudden spikes in user activity without overwhelming resources.
• 🦮 Understanding the nuances of each AWS service and their limits can guide effective system designs for scalable applications.

Questions & Answers

Q: What problem was faced with the original PDF generation setup in AWS Lambda?

The initial setup attempted to generate all requested PDFs in a single Lambda function, which exceeded the maximum execution time of 15 minutes due to the extensive calculations and file generation needed for 200 to 300 records. This approach was inefficient and impractical for handling larger data sets.

Q: How did the team decide to solve the timeout issue with AWS Lambda?

The team opted for an asynchronous processing model, utilizing a message queue (SQS) to distribute the workload across multiple workers. This allowed for parallel processing of the tasks, preventing any single Lambda from running longer than the timeout limit and thereby drastically improving performance.

Q: What role did S3 and DynamoDB play in the new architecture?

S3 was used for storing the generated PDF files, while DynamoDB tracked the progress of each worker. Once a worker completed its task, it updated the database to indicate that processing was complete, allowing the main API to monitor overall progress and handle final output once all tasks were done.

Q: Why was throttling implemented in the email sending process?

Throttling was necessary because the system initially faced issues due to an overload of emails sent simultaneously, leading to throttling exceptions from AWS Simple Email Service (SES). Implementing a limiter ensured that bursts of email requests did not exceed SES capacity, maintaining the system's stability under pressure.

Summary & Key Takeaways

• The project involves generating and combining multiple PDF files (potentially thousands of pages) triggered by user interaction through an API using AWS Lambda, which has a strict 15-minute execution limit.

• Initial attempts to run the PDF generation in a single Lambda function failed due to timeout issues; the solution involved using asynchronous processing and a queue to distribute tasks among multiple workers.

• By leveraging AWS services such as SQS for queuing, DynamoDB for tracking progress, and S3 for storage, the overall processing time was reduced from 15 minutes to about one minute.