Almost free server monitoring with AWS Lambda
A 5 minutes story written on Feb 2018 by Adrian B.G.
The problem
I created some websites as pet projects on which I don’t have a full stack monitoring system (like new relic), but I want to be alerted when they go offline (for any reason), with minimal (or no) costs.
The solution
To solve this issue I can:
- use a managed service, the free tier usually is limited to 1 URL
- install a dedicated solution like PHP Server Monitor
- write a custom solution (I did while tinkering with Go)
But … I only need to check every X minutes if the websites are online, so if I were to use a machine or container that runs all the time it would be a waste of resources because I would pay for the sleep(x minutes) too. As a result, I decided to go with the 4th solution and use a serverless function that triggers every x minutes, so this way I will only pay the resources I need.
The result will be “almost free”, as in you will pay only a few cents per month, depending on how many lambda functions you create and their running time (based on the response latency of your URL endpoints and the schedule time of your functions).
The stack
This solution can be implemented with any of the 3 big cloud providers using AWS Lambda, Google Functions or Azure Functions. I choose AWS for this, for my own reasons.
There are multiple open source lambda functions that solve this problem, I selected the simplest one that allows checking multiple URLs using only 1 function instance: jethrocarr/lambda-ping
Install and deploy
You will need an AWS account and access to the following services:
- AWS Lambda (for the code)
- AWS CloudWatch (for the trigger and alarms)
- AWS Simple Notification Service (for the notifications)
To ease the setup I used the serverless framework which allowed me to deploy the function using only 1 CLI command:
- [install] (https://docs.aws.amazon.com/cli/latest/userguide/installing.html)and configure the AWS CLI
- install the [serverless] (https://serverless.com/framework/docs/providers/aws/guide/installation/)framework (optional step)
- deploy the lambda function (follow these steps)
The code is now “in the cloud”, ready to go at your command, the next step is to put it to use. Its logic is very simple, it takes a list of HTTP endpoints, make a request to them and log the results in AWS CloudWatch metrics (errors, latency and http status code).
Run it
You can invoke a lambda function in multiple ways, but using CloudWatch Schedule functionality is pretty easy, and you can do it from the web interface (AWS Console). You need to create a Rule that triggers the lambda every x minutes wite your HTTP endpoints as a parameter.
You can have multiple schedule with different URLs
Now wait and let the function run at least once. Check the logs and if everything is ok you can proceed to the next step.
Alert Notification
Using the AWS SNS you can create a topic (channel) and at least 1 subscription (receivers), one for email and one for SMS. If you want SMS you will have to choose one of the following AWS regions:
Warning! Not all regions and countries can use the SMS feature
Connect CloudWatch to SNS
After the function ran successfully at least once, you should see a new endpoint in the CloudWatch Metrics section, more exactly 2 metrics for each URL you added to the json input:
Using these metrics you can setup alarms, one for each URL, and connect them to the notification topics:
Each status code will be a data point in a metric, as a number
As you can see I added an alert for the case when the HTTP response code is bigger or equal to 300, you can add one for latency checks too.
The Alert Period must be at least the period of the lambda trigger schedule, or you can disable the “Thread missing data as bad”.
The alert name will be the title of the notification (email subject or SMS text).
Test
In order to test it I chosen to use the httpbin API as an input, you can use custom status codes, redirects and delayed responses.
httpbin(1): HTTP Client Testing Service
Costs
I said “almost free” but is mostly free for a regular user, AWS Lambda has the following costs:
- per request (trigger), the first million of them are free. If you have 1 lambda that runs every 15 minutes you will “spend” around 4% of them.
- per memory used, the lowest option (128Mb) is enough for any HTTP checker, and the first 3200000 seconds are free. For a 5s avg run duration for one lambda that runs each 15 minutes you will “spend” around 7% of the free tier.
Because you pay for each request, is more efficient to group multiple URLs in the same function.
There are other (more complex) lambda functions out there, but I don’t have any experience with them and my needs were pretty simple. Anyway I think it would be nice to mention them, so here it is:
