Almost free server monitoring with AWS Lambda

A 4 minutes story written on Feb 2018 by Adrian B.G.

image

The problem

I created some websites as pet projects on which I don’t have a full stack monitoring system (like new relic), but I want to be alerted when they go offline (for any reason), with minimal (or no) costs.

The solution

To solve this issue I can:

But … I only need to check every X minutes if the websites are online, so if I were to use a machine or container that runs all the time it would be a waste of resources because I would pay for the sleep(x minutes) too. As a result, I decided to go with the 4th solution and use a serverless function that triggers every x minutes, so this way I will only pay the resources I need.

The result will be “almost free”, as in you will pay only a few cents per month, depending on how many lambda functions you create and their running time (based on the response latency of your URL endpoints and the schedule time of your functions).

The stack

This solution can be implemented with any of the 3 big cloud providers using AWS Lambda, Google Functions or Azure Functions. I choose AWS for this, for my own reasons.

There are multiple open source lambda functions that solve this problem, I selected the simplest one that allows checking multiple URLs using only 1 function instance: jethrocarr/lambda-ping

Install and deploy

You will need an AWS account and access to the following services:

  • AWS Lambda (for the code)
  • AWS CloudWatch (for the trigger and alarms)
  • AWS Simple Notification Service (for the notifications)

To ease the setup I used the serverless framework which allowed me to deploy the function using only 1 CLI command:

  1. installand configure the AWS CLI
  2. install the serverlessframework (optional step)
  3. deploy the lambda function (follow these steps)

The code is now “in the cloud”, ready to go at your command, the next step is to put it to use. Its logic is very simple, it takes a list of HTTP endpoints, make a request to them and log the results in AWS CloudWatch metrics (errors, latency and http status code).

Run it

You can invoke a lambda function in multiple ways, but using CloudWatch Schedule functionality is pretty easy, and you can do it from the web interface (AWS Console). You need to create a Rule that triggers the lambda every x minutes wite your HTTP endpoints as a parameter.

image
Create the CloudWatch rule
image

You can have multiple schedule with different URLs

Now wait and let the function run at least once. Check the logs and if everything is ok you can proceed to the next step.

image

Alert Notification

Using the AWS SNS you can create a topic (channel) and at least 1 subscription (receivers), one for email and one for SMS. If you want SMS you will have to choose one of the following AWS regions:

image

Warning! Not all regions and countries can use the SMS feature

Connect CloudWatch to SNS

After the function ran successfully at least once, you should see a new endpoint in the CloudWatch Metrics section, more exactly 2 metrics for each URL you added to the json input:

image

Using these metrics you can setup alarms, one for each URL, and connect them to the notification topics:

image
Alarms menu

image
Alarms page

Each status code will be a data point in a metric, as a number

As you can see I added an alert for the case when the HTTP response code is bigger or equal to 300, you can add one for latency checks too.

The Alert Period must be at least the period of the lambda trigger schedule, or you can disable the “Thread missing data as bad”.

The alert name will be the title of the notification (email subject or SMS text).

Test

In order to test it I chosen to use the httpbin API as an input, you can use custom status codes, redirects and delayed responses.

httpbin(1): HTTP Client Testing Service

Costs

I said “almost free” but is mostly free for a regular user, AWS Lambda has the following costs:

  • per request (trigger), the first million of them are free. If you have 1 lambda that runs every 15 minutes you will “spend” around 4% of them.
  • per memory used, the lowest option (128Mb) is enough for any HTTP checker, and the first 3200000 seconds are free. For a 5s avg run duration for one lambda that runs each 15 minutes you will “spend” around 7% of the free tier.
image

Because you pay for each request, is more efficient to group multiple URLs in the same function.

There are other (more complex) lambda functions out there, but I don’t have any experience with them and my needs were pretty simple. Anyway I think it would be nice to mention them, so here it is:

Thanks! 🤝

Please share the article, subscribe or send me your feedback so I can improve the following posts!

I curate a list of articles, talks and papers for one/two times per month. They are mostly related to computer science, distributed systems, databases, Go, containers and Cloud solutions.

comments powered by Disqus