Ioan-Alexandru Gheorghita

In this case study, we replaced AWS Kinesis Firehose with a purpose-built serverless consumer for a client streaming millions of IoT records per minute into their datalake — taking on full ownership of batching, buffering, file layout, and durability in exchange for a fraction of the cost. 

The catch? We built it on AWS Lambda, which is stateless by design. We'll walk through how we made it stateful using Kinesis tumbling windows, how an S3 multipart-upload state machine is carried across invocations, the retry semantics that guarantee zero data loss, and the trade-offs we deliberately made — including the ones where doing the "right" thing would have quietly made things worse. 

 If you've ever wondered what's actually happening inside a managed streaming service, or what it takes to build one yourself, this one's for you.