Hands-on Configuration for Amazon Kinesis Streams and AWS Lambda

Introduction

In my previous articles, we talk a lot about serverless and why we use it. In this article, we will try to use Amazon Kinesis to analyze the real-time streaming data.

Solution Diagram

We will learn event-driven programming with Kinesis and Lambda. By using Amazon Kinesis,, we can easily send data to Amazon Lambda to trigger the function. Finally, we will see the logging in CloudWatch.

Create an Amazon Kinesis Stream

In Amazon Kinesis page, we select Kinesis Data Streams and click “Create data stream”.

It is up to you to set the name, you could just follow my options if you have no preference. Click Create data stream.

Wait for a few seconds, the data stream will be created quickly.

Create a Lambda Function

Go to AWS Lambda page, click “Create Function”.

Select blueprints and search for “kinesis-process-record-python”. Blue print is just a set of pre-defined function of AWS Lambda, you could definitely write your own function.

Click Configure.

In configuration page, we give the function name first, then create a new rule to let lambda has enough right. Also, Enable to trigger Kinesis stream. For the lambda function code, it loops through each of the records received, decode the data and print the data to the debug log.

Then click Create function.

Testing

We click test on the right top corner.

We just have to create a new test event called stream, then click create.

Choose the stream event and click Test now.

We are able to see the successful message.

In CloudWatch, we can click log group and select the kinesis group and log stream.

We are able to see the logging here.

Cleanup

Delete function in AWS Lambda.

Select stream-demo and delete this data stream.

Leave a Reply