We will use Serverless Framework to create the application.
Copy
Ask AI
kafka-examples git:(master) serverless What do you want to make? AWS - Node.js - HTTP API What do you want to call this project? produce-in-lambdaDownloading "aws-node-http-api" template...Project successfully created in produce-in-lambda folder
Then we will initialize a node project and install axios dependency.
You need to replace the endpoint, username and password above with the values
that you copy from the Upstash Console.The above code simply creates a producer and sends the message to Kafka.
Now let’s validate that the messages are pushed to Kafka. We can consume the
Kafka topic using the REST API. You can copy the curl code to consume from the
Upstash Console.
We can also use a native Kafka client (e.g. KafkaJS) to access our Kafka
cluster. See
the repo
for both examples. But there is a latency overhead if connecting (and
disconnecting) to the Kafka with each function invocation. In our tests, the
latency of the function with REST is about 10ms where it is about 50ms when
KafkaJS is used. Kafka client's performance could be improved by caching the
client outside the function but it can cause other problems as explained
here.Troubleshooting: If Lambda function outputs internal error, check the
cloudwatch log (Lambda > Monitor > View logs in CloudWatch).