Puppet webhook proxy

What is this

This is an AWS Chalice app that will let github send webhook requests to an Enterprise Puppetmaster running Code-Manager (such as an AWS OpsWorks puppetmaster) and hosted in a private subnet

Files

• app.py Chalice app fil
• install.sh shell script to copy files about FYI
• policy-dev.json AWS policies to set permissions. Set your account #
• requirements.txt required external python modules

How it works

The chalice app is exposed via AWS API Gateway to the Internet. A github repository is set up to send a webhook to the address of the API Gateway. API Gateway passes the request to a Lambda function written in python

The Lambda function app checks that the HMAC SHA1 signatures from github are correct and then forwards the request to the Puppetmaster. This triggers a git pull on the Puppetmaster. The git pull fetches the latest files from the github repository that originated the request

The Puppetmaster is configured to use a private subnet but the external requests reach it via the API Gateway/ Lambda

If you don't want this extra complication then run the Puppetmaster in a public subnet. But your Puppetmaster will be externally exposed and the HMAC checking will not be in place

Prerequisites

There is some set up of the Puppetmaster. It must be configured to know where to pull its configuration from on github, and it must have a ssh key that the github repository knows. With AWS Opsworks this is part of the setup

After this setup, login to the new Puppetmaster and use "puppet-access login" to make a token as detailed here https://puppet.com/docs/pe/2017.3/code_management/code_mgr_config.html#configuring-code-manager

Next, setup the Chalice part with the install.sh. This should work on a vanilla Centos 7 server. It will give a URL

Then setup Github. Use the settings / webhook page from the project that you will be using to host the repository for the Puppetmaster.

The payload URL is the chalice URL with the extra path "/hooky" on the end

Set the secrets and url

Go into the AWS SSM Parameter store (it's part of the EC2 stuff on the AWS web Console)

You should now have

• the puppet-access "token"
• the shared secret from github setting page
• the url of the puppetmaster

Set up the secret as /nonprod/puppet/sharedkey, the token generated by "puppet-access login" as /nonprod/puppet/token and the url of the puppetmaster as /nonprod/puppet/url

Then try a commit and push. The changes should pop up on the Puppetmaster

notes on the code

The most difficult part was getting the HMAC signature correct

All the code examples I could find for this were in Ruby. But the tool I was using for the API Gateway/Lambda integration used Python. The key insights were that

1. the header is called X-Hub-Signature

2. the raw body is in the correct format for the Python 3.6 HMAC library. The shared key has to be converted to bytes

3. the code for calculating the HMAC signature is ps = hmac.new(sharedkey, app.current_request.raw_body, 'sha1').hexdigest()

4. github webhooks have a 10 second time out. With our server config this isn't enough (usually it was 13 seconds). By setting the "wait" part of the request from the Lambda to the Puppetmaster to be false the request returns immediately, I assume it is asynchronous

There is debug logging in many places but by default the lines to set this up are commented out

If you wish to alter the ssm parameter store paths then modify the policy as well as the get_parameter calls in the code