Amazon AWS and your project

[Pschittt]

Hello,

I'm very interested by your project. I have the same issue as you, I don't trust my own server availability for this critical resource.

I only known AWS from name, your project made me to check a little more on AWS.

Just to fully understand, you're planning to use :

• AWS API gateway -> to manage the http request
• AWS Lambda -> to retrieve the data from the DB
• AWS dynamodb -> as the DB

Is that correct ?

Your code is for the 3 above or just AWS Lambda ? Why using Node.js and not python for this "Engine" ?

Last question, if you plan to add the web vault, in which AWS solution it will be ?

Sorry for all this question. I'd like to fully understand before participating.
I added the web vault compatibility to JCS/bitwarden-ruby. So maybe I could help here.

Thanks.

[vvondra]

Yes, the components are as you say, API Gateway proxies the HTTP request to AWS Lambda which is executable code which AWS automatically scales and containerizes for you. Dynamo is the DB (since its the only way in AWS DB not to pay for hourly costs, only consumed capacity).

That's the idea of the project, in API gateway you pay per request, in DynamoDB for provisioned capacity, for Lambda for time ran*memory consumed. All of them should well fit in the Free Tier that AWS gives you for testing.

Node.js vs Python? Arbitrary, I'm more familiar with JS, but it's coded in ES7, so all the modern stuff is in.

For Web vault, as far as I saw in the extension/app, it sits on the base URL of the API, so we could add https://github.com/bitwarden/web (with the gulp compilation steps) to the serverless framework deployment and there are two options:
a) host it on S3 (next to zero costs as well) and configure API gateway to serve / and not to proxy to lambda, but to plain file hosting on S3
b) make a Lambda which returns the index.html file and necessary JS files (a bit ugly, but works)

[Pschittt]

For the web vault, maybe we can use the official docker ?
That’s what I use, and there is only 2 settings file to modify.

How do you manage SSL with AWS ? Via the API Gateway ?

Last question, how do you test your development ? You need to upload it to AWS for each test ?

[vvondra]

If you use the official docker, you will need an EC2 instance to run for which you pay hourly costs (and it's no longer serverless :)). In general no need for docker/webserver for plain static files (the web vault is a Angular JS SPA)

SSL: Yes, API Gateway is HTTPS only. They generate an .aws.amazon.com endpoint for you with a certificate, but you can also set up your own domain and generate a free SSL certificate for it, instructions are in the README

Development: yes, I do sls deploy --stage test and test against that. You can very quickly deploy an update to a single function with sls deploy --stage test -f function_name. Serverless also has some mock capabilities on your local machine, but I haven't used those yet, the dependency missing there would be DynamoDB (but it's possible to run a local DynamoDB on your machine as well), Ill be happy to merge that change, but Im ok with the current set up

For continous integration, Travis sets up a fresh stack every build https://github.com/vvondra/bitwarden-serverless/blob/master/test.sh, you can check the output https://travis-ci.org/vvondra/bitwarden-serverless

[Pschittt]

Thanks.

So I tried your serverless bitwarden version, and it works on AWS. I only tested the Chrome plugin.

I've also checked the dynamoDB structure, and from what I saw, it seems that some columns are missing (which will be needed for the web vault) :
User table :

• updated_at
• email_verified
• name
• password_hint
• private_key
• public_key
• totp_secret

Cipher table :

• created_at
• updated_at
• folder_uuid
• organization_uuid
• type (int)
• data
• favorite
• attachments

Devices table :

• push_token
• access_token
• token_expires_at

Folders table :

• created_at
• updated_at
• name

By the way, why do you store the JWT token on the user table ? AWS doesn't automatically handle this kind of thing ?

[vvondra]

DynamoDB is a document database and one of its behaviours is it doesn't store columns which are null (it doesn't have a strict schema).

In the API response I think I return most of the fields you mention with sane defaults even if they are not set (see https://github.com/vvondra/bitwarden-serverless/blob/master/src/lib/mappers.js)

I actually do the token differently than jcs/bitwarden-ruby (they store the token, i only store the signing secret). There they store the access token and its expiration date. I do not load the user by access token (https://github.com/jcs/bitwarden-ruby/blob/master/lib/api.rb#L24) but I parse the token and load the user by it's UUID. I then check the expiration from the token data itself and verify the token is correctly signed. The access token itself is not stored on my side.

The second different thing is the signing secret. The ruby version generates automatically a RSA keypair which it uses to sign the tokens (https://github.com/jcs/bitwarden-ruby/blob/master/lib/bitwarden.rb#L174). It uses the RS256 algo to sign the JWTs and all the signatures are signed with the same key.
I generate a random secret separate for each user (and store it) and use the HS256 algo to sign the JWTs.

Both approaches are valid, the important part with JWTs is how to invalidate tokens for a single user. In my case I will just change the JWT secret field on the user. In the ruby version, you would delete the access token on the user table or put expiration to a past date.

[Pschittt]

Indeed, I just saw in models.js (I imagine that the function which declare the DB model) that the column missing were declared.

Ok, I understand the jwtKey now.

Last question, now that I have deployed the serverless to AWS (which is tagged as prod), to update it, I need to :

• remove it (but how ?!), and deploy again
• or just deploy it again with serverless deploy --region us-east-1

And to help you in your project, correct me if i'm wrong, but I need to :

• declare the new/missing http request in : serverless.yml
• and create/update an handler in the folder src/ without touching to the src/lib/ folder
  Is that correct ?

I'll try to add the dev that I've done for the web vault in jcs/bitwarden-ruby to your project.

[vvondra]

• declare the new/missing http request in : serverless.yml
  Yes, exactly.
• and create/update an handler in the folder src/ without touching to the src/lib/ folder
  Is that correct ?
  You can update any of the code if you find it necessary.

To update in AWS, simply run sls deploy a second time exactly like when you deployed it the first time, it will update changed functions.
If you want only to update the source code of one function, you can also use sls deploy -f function_name

[Pschittt]

Great !

I'm trying to set my devlopment environment with the web vault.
My web vault is configured to request the AWS Gateway API.

I'm running into errors (nothing wrong with that), but I'd like to debug and see the server side logs.

I've run "serverless logs -f login" (I assume it's the login function because i'm running the error at login).
I get this message in return "The specified log group does not exist."

That's the difficult part for me, I like to see the server side logs (in live if possible to match the client side request), and I don't really understand how is it possible with AWS.

I'm sorry for this questions.

[vvondra]

I'm happy to help, keep asking.

The command you wrote should work, it really means the login endpoint was never triggered.

Makr

[Pschittt]

Ok, i'll keep asking.

So I've started, and saw some missing header to make your api works with the web vault.

First :

• For ALL response, the web vault is checking for the header : Access-Control-Allow-Origin :'*' (and not just for the OPTIONS request (CORS check) )
• At login page, the web vault is not sending 'devicename' and 'devicetype', so you need to remove it from the param check in login.js

I've made those change locally, but got some errors (which i don't fetch with sls logs -f login).

By the way, when you check if User is correct and password too, you should not tell "user not found" or "password not match", it gives too much information for a malicious person. You should set a generic response (the same for both) like "Invalid username or password".

[vvondra]

Sounds like you're on the right path! Feel free to make and submit those changes, I added the little stuff I saw here #1

Maybe you get some CORS errors in the browser console and the request never goes to the server. I think I saw something similar when i tested in two minutes a few days ago

[Pschittt]

I've made these change in my local source file.
I've also created a keys.js to handle the account/keys request just after the connect/token

I'm stuck here, because the web app doesn't send the Bearer token when doing the account/keys request (the Bearer token has been received after connect/token).

The CORS has been handled by adding the correct headers.
I even set my own domain for the api.

Still looking why the web app is not storing and sending the bearer token.

[vvondra]

There's a few gotchas like that it sends the bearer in the Content-Language header bitwarden/web@595cf6c#diff-bbaf1860cc4806373b585f73c46fc33eR16
maybe this?

[Pschittt]

You're right, for some request the bearer is in the Content-Langage header.

But for the account/keys, there is no bearer in any header or body at all in my test environment.
(Which is being sent if I use the web vault environment with the ruby api).

I'll keep looking if there is a link with AWS API.

[Pschittt]

If you're interested in testing I've uploaded my files to a "Webvault" branch, here : https://github.com/Pschittt/bitwarden-serverless/tree/webvault

Like I said above, still stuck at the account/keys request. The browser doesn't send any token, so the api cannot identity the user. Don't know why the token is not stored (I've checked the browser storage, and no trace of the token received from connect/token).

[vvondra]

The identity/connect/token response doesn't return the token after login?

[Pschittt]

The identity/connect/token response is OK. The browser receive the token, and the token seems to be well formed.

Just after the identity/connect/token request, the web vault do a api/account/keys request, to update the privatekey and publickey of the account. But when it's doing this request, the web vault doesn't send the Bearer Token received from identity/connect/token.

When I check the browser storage, I don't see any Bearer Token for the web vault, like if it wasn't store, so it can't be added on the header for the others requests.

I hope it's more clear.

[Pschittt]

Is it possible that the web vault doesn't know how to handle a JWT with HS256 algorithm ? And that's why the JWT is not stored when received, and then reused ?

[Pschittt]

I've checked with a RS256 JWT, still same problem. So I don't think it's coming from the JWT.

[vvondra]

Let's continue in #7