Less than two weeks ago Apple released the Swift AWS Lambda Runtime. The runtime allows us to develop Swift based serverless functions to run on the Amazon Web Services (AWS) Lambda platform. Given my time spent on AWS SDK Swift this has piqued my curiosity. So I spent the last weekend trying to implement my first Swift Lambdas. What would I develop though? I decided I would try to replace a couple of the Node.js Lambdas that I use in managing my AWS setup. For this article I will discuss my Simple Notification Service Slack publisher. I'll leave the other Lambda for a later article.
SNS to Slack
On your slack channel, add the 'email integration' app. Once done, Slack will provide you an email address with slack.com domain. Emails sent to this address will be imported into your slack channel. Then, on SNS create an email subscription and provide the slack email above.
- AWS Lambda function SNS - Slack Channel. GitHub Gist: instantly share code, notes, and snippets.
- From Slack you can now create your own private channel: Invite the bot to the channel: Then send it a message. Note that it may take several seconds for the bot to respond due to Slack push notification delays, SNS Topic delays, and Slack incoming message delays.
For those who don't know the Simple Notification Service (SNS) is an AWS web service that manages the sending of messages to subscribing clients. In SNS you create a topic. Services can publish messages to the topic and then clients who are subscribed to the topic will have these messages passed onto them. Clients can include email, web servers, Amazon SQS queues or AWS Lambda functions. For example messages can be sent when someone puts an object in an S3 bucket, when a server shuts down or when an email is received. Pretty much every AWS service can send notifications of events to SNS topics.
I wrote a small Node.js Lambda which took these messages and formatted them a bit and sent them to a Slack webhook, which would then post the message in a Slack channel. The Lambda involved about 50 lines of js code and was fairly simple. This should be a good start in my Swift AWS Lambda runtime investigations.
During this article I'm not going to give you much detail about setting up the Slack side as this isn't what it is about. There is more details in the README.md from the Git Hub repository containing the sns-to-slack code.
Kicking off
First you need to setup a project. Fabian Fett details this in his great blob post Getting started with Swift on AWS Lambda. I'm going to assume you have read that already and can already create a basic Swift Lambda project. The only exception to Fabian's setup is we need to add an additional dependency. We are going to be posting to a webhook so we need an http client. For this example I'm using the swift-server AsyncHTTPClient. Add the package in the dependencies
section and the add the product in the dependencies
section of the target.
You will also notice I have included the product AWSLambdaEvents
in the list of dependencies. This library contains a list of input structures for when a Lambda has been triggered by another AWS system. We are triggering our Lambda from SNS so will need the event SNS.Event
.
Attempt #1
So far the only code examples I had seen were structured as follows. They used a Lambda.run
function which you passed a closure which accepted a context object, your input object and a callback to call when your Lambda was done. Given that my input is the SNS.Event
object and I didn't need to return anything I started like this.
Format the event message from the SNS.Event
, send the message to the Slack Hook URL and when that completes call the callback. For brevity I haven't included all the code. You can find all the code to my SNS to Slack Lambda here.
Anyway I thought job done, that was easy. I ran the code using the local test server and it immediately complained about me not calling syncShutdown
on the the http client. My code was creating a HTTPClient
, executing a request and then immediately deleting the HTTPClient
because it went out of scope. No worries I can fix this. I add a defer block to call httpClient.syncShutdown
when we exit the function and I add a wait
on the futureResult
to ensure the http request has finished before leaving the function.
While this code will work something about it doesn't feel right though. I shouldn't be required to add a wait()
into my code. It made me think back to one of the Swift Server working group (SSWG) requirements for their packages 'Packages should be non-blocking (w/ async API) unless not possible (blocking C libs, etc)'. Could a swift-server group package be forcing me into writing blocking code?
Local test server
As an aside before I continue, the local test server is a godsend. Uploading the Lambda to AWS every time I needed to test it would have slowed down development ten fold. In Xcode add an environment variable LOCAL_LAMBDA_SERVER_ENABLED
set to true
and you can run the Lambda locally and communicate with it via curl
. I setup a file sample-event.json
which contained json for an SNS.Event
and called the following to test the Lambda. You should really add in a 'Content-Type' header but it never complained if I didn't include it.
Attempt #2
Is there a way I can write asynchronous code using Swift AWS Lambda runtime? Of coures there is. Swift AWS Lambda runtime provides a EventLoopLambdaHandler
protocol which can be used to manage your Lambda code. It provides opportunity for more complex code and object lifecycle management. You create a handler class that conforms to EventLoopLambdaHandler
. Conforming requires you include a handle
function which is called when the Lambda is invoked. This handle
function is called with the Lambda context and the input object and is expected to return an EventLoopFuture
which will be fulfilled with the output object. The input and output objects are defined by the In
and Out
typealiases.
Now I have a struct, I can manage the life cycle of my HTTPClient
outside of the handle
function so I don't need the call to wait
. EventLoopLambdaHandler
conforms to the protocol ByteBufferLambdaHandler
which has a shutdown
method. I can place my HTTPClient
shutdown code in there.
In theory I should not be able to call httpClient.syncShutdown
in this function because I could be shutting down an EventLoopGroup
while running on an EventLoop
which causes issues. The non-sync version of the shutdown is not available at the moment (we have to wait for the next release of AsyncHTTPClient). In this situation though because I am using the EventLoop
from the Lambda there is no EventLoopGroup
shutdown and everything is ok.
To invoke this new style of Lambda change your Lambda.run
call to the following, where your closure returns an instance of the SNSToSlackHandler
class.
Finally fill out your handle
function to actually do the work. The handle should return an EventLoopFuture
that'll be fulfilled once the HTTP post has completed.
Run your Lambda, run the curl
command from above and you should see a message posted to your Slack channel.
An added advantage of using EventLoopLambdaHandler
is you can move all your object creation into the init
function of the handler class. The handler is created on a 'cold start' of the Lambda, but if you are invoking a Lambda that is already running, a 'warm start', it will reuse the already existing handler. So I only create the HTTPClient
once instead of everytime I invoke the Lambda.
Deploying your Lambda
Before you continue you need to be sure you have Docker Desktop and the aws command line interface installed. You can install Docker Desktop from here and you can install the aws cli via Homebrew with the following.
Building and deploying your Lambda involves a number of steps that include various calls to docker and the aws cli. Very quickly I became bored of looking up the correct syntax for all the commands to build and deploy, so looked to write a number of scripts to simplify this process. I imagine this is going to be one of the bigger obstacles for encouraging people to develop Swift Lambdas. You can break down the process into four stages.
- Build Docker image for building Lambda with amazonlinux2 Docker image
- Compile Lambda code
- Package Lambda code along with the libraries it needs to run
- Deploy Lambda to AWS. I use the aws cli to deploy Lambdas, but you can deploy them through the AWS web console or using the AWS Serverless Application Model (SAM) cli.
I wrote/stole (mainly from the swift-aws-lambda-runtime repository) scripts to cover each of these stages. I then wrote one script install.sh
to run all the stages in one go. I now run the install.sh
script and have a working deployed Lambda.
Aws Sns Webhook
Once the Lambda is deployed you can go to the AWS dashboard, select SNS, create an SNS Topic and press the 'Create subscription' button. Select the protocol 'AWS Lambda' and then select your Lambda from the 'Endpoint' dropdown. You can test it works by pressing the 'Publish message' button from the topic page. Now any service that sends messages to this topic will have these output in your Slack channel.
Final Code
You can find the final version of the SNS to Slack Lambda here. On top of what I detailed above I have added some error handling, the Slack URL is acquired via an environment variable and I support outputting multiple messages from one SNS.Event
.
Finally Tom Doron, Fabian Fett and everyone who worked on Swift AWS Lambda Runtime have done a great job. It has certainly made the development of Swift based Lambdas a more positive experience. Prior to this it was quite a soul destroying process. There are still obstacles but I'm sure these will be improved upon as more people take time to develop their own Lambdas. I hope you found this article of interest. If so, please provide feedback on Twitter where you can find me @o_aberration, or on the Vapor Discord where I am @adamfowlerphoto.
Amazon Chime Webhook
#!/usr/bin/env python |
# -*- coding: utf-8 -*- |
'' |
Follow these steps to configure the webhook in Slack: |
1. Navigate to https://<your-team-domain>.slack.com/services/new |
2. Search for and select 'Incoming WebHooks'. |
3. Choose the default channel where messages will be sent and click 'Add Incoming WebHooks Integration'. |
4. Copy the webhook URL from the setup instructions and use it in the next section. |
Follow these steps to encrypt your Slack hook URL for use in this function: |
1. Create a KMS key - http://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html. |
@hayd note: It seems to be important that the key, role and lambda are all in the same region... |
2. Encrypt the event collector token using the AWS CLI. |
$ aws kms encrypt --key-id alias/<KMS key name> --plaintext '<SLACK_HOOK_URL>' |
Note: You must exclude the protocol from the URL (e.g. 'hooks.slack.com/services/abc123'). |
3. Copy the base-64 encoded, encrypted key (CiphertextBlob) to the ENCRYPTED_HOOK_URL variable. |
@hayd note: This is the output of the above `aws kms encrypt` command verbatim. |
4. Give your function's role permission for the kms:Decrypt action. |
Example: |
{ |
'Version': '2012-10-17', |
'Statement': [ |
{ |
'Effect': 'Allow', |
'Action': [ |
'kms:Decrypt' |
], |
'Resource': [ |
'<your KMS key ARN>' |
] |
} |
] |
} |
'' |
from __future__ importprint_function |
importboto3 |
importjson |
importlogging |
importre |
frombase64importb64decode |
fromurllib2importRequest, urlopen, URLError, HTTPError |
# value of the CiphertextBlob key in output of $ aws kms encrypt --key-id alias/<KMS key name> --plaintext '<SLACK_HOOK_URL>' |
ENCRYPTED_HOOK_URL='CiC9...' |
HOOK_URL='https://'+boto3.client('kms').decrypt(CiphertextBlob=b64decode(ENCRYPTED_HOOK_URL))['Plaintext'] |
logger=logging.getLogger() |
logger.setLevel(logging.INFO) |
deflambda_handler(event, context): |
logger.info('Event: '+str(event)) |
state_dict= {'Ok': ':thumbsup:', 'Info': ':information_source:', 'Severe': ':exclamation:'} |
d=dict(line.split(': ') forlineinevent['Records'][0]['Sns']['Message'].splitlines() if': 'inline) |
transition=re.match('Environment health has transitioned from (.*) to (.*?).', d['Message']) |
iftransition: |
original, became=map(lambdax: state_dict.get(x, x), transition.groups()) |
d['Message'] ='*Health*: '+original+u' ⟶ '+became+'n_'+d['Message'].split('. ', 1)[1] +'_' |
slack_message= { |
'channel': 'build'if'New application version was deployed'ind['Message'] else'beanstalk', |
'text': d['Message'], |
'username': d['Environment'], |
'icon_emoji': ':tophat:' |
} |
req=Request(HOOK_URL, json.dumps(slack_message)) |
try: |
response=urlopen(req) |
response.read() |
logger.info('Message posted to %s', slack_message['channel']) |
exceptHTTPErrorase: |
logger.error('Request failed: %d %s', e.code, e.reason) |
exceptURLErrorase: |
logger.error('Server connection failed: %s', e.reason) |