Skip to content

Commit

Permalink
Merge pull request #1 from vt-digital-libraries-platform/sam
Browse files Browse the repository at this point in the history
update VTDLP Resolution Service to using SAM and one-click button
  • Loading branch information
soumikgh authored Jul 24, 2020
2 parents 2d0b93a + e7ec489 commit 811057a
Show file tree
Hide file tree
Showing 9 changed files with 455 additions and 59 deletions.
247 changes: 247 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@

# Created by https://www.gitignore.io/api/osx,linux,python,windows,pycharm,visualstudiocode

### Linux ###
*~

# temporary files which can be created if a process still has a handle open of a deleted file
.fuse_hidden*

# KDE directory preferences
.directory

# Linux trash folder which might appear on any partition or disk
.Trash-*

# .nfs files are created when an open file is removed but is still being accessed
.nfs*

### OSX ###
*.DS_Store
.AppleDouble
.LSOverride

# Icon must end with two \r
Icon

# Thumbnails
._*

# Files that might appear in the root of a volume
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent

# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk

### PyCharm ###
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839

# User-specific stuff:
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/dictionaries

# Sensitive or high-churn files:
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.xml
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml

# Gradle:
.idea/**/gradle.xml
.idea/**/libraries

# CMake
cmake-build-debug/

# Mongo Explorer plugin:
.idea/**/mongoSettings.xml

## File-based project format:
*.iws

## Plugin-specific files:

# IntelliJ
/out/

# mpeltonen/sbt-idea plugin
.idea_modules/

# JIRA plugin
atlassian-ide-plugin.xml

# Cursive Clojure plugin
.idea/replstate.xml

# Ruby plugin and RubyMine
/.rakeTasks

# Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml
crashlytics.properties
crashlytics-build.properties
fabric.properties

### PyCharm Patch ###
# Comment Reason: https://github.com/joeblau/gitignore.io/issues/186#issuecomment-215987721

# *.iml
# modules.xml
# .idea/misc.xml
# *.ipr

# Sonarlint plugin
.idea/sonarlint

### Python ###
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
.pytest_cache/
nosetests.xml
coverage.xml
*.cover
.hypothesis/

# Translations
*.mo
*.pot

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints

# pyenv
.python-version

# celery beat schedule file
celerybeat-schedule.*

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/

### VisualStudioCode ###
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
.history

### Windows ###
# Windows thumbnail cache files
Thumbs.db
ehthumbs.db
ehthumbs_vista.db

# Folder config file
Desktop.ini

# Recycle Bin used on file shares
$RECYCLE.BIN/

# Windows Installer files
*.cab
*.msi
*.msm
*.msp

# Windows shortcuts
*.lnk

# Build folder

*/build/*

packaged.yaml
out

# End of https://www.gitignore.io/api/osx,linux,python,windows,pycharm,visualstudiocode
92 changes: 76 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,88 @@
# VTDLP Resolution Service
This service resolves VTDLP's permanent link. For example, it redirects ```http://idn.lib.vt.edu/ark:/53696/r335gm22``` to ```https://iawa.lib.vt.edu/archive/r335gm22```

## Lambda Function
* [lambda_function.py](lambda_function.py): Handle URL redirection
* Env variables
* Region: ```us-east-1```
* TargetTable: ```DDBtablename```
* Image404: ```404.jpg```
This project contains source code and supporting files for a serverless application - VTDLP resolution service. This service resolves VTDLP's permanent link. For example, it redirects ```http://idn.lib.vt.edu/ark:/53696/kr10gt01``` to ```https://iawa.lib.vt.edu/archive/kr10gt01```.

## Parameter Override
```
{"TargetTableName":"resolutiontable","Region":"us-east-1","Image404":"https://images/404.jpg"}
```
The application uses several AWS resources, including Lambda functions and an API Gateway API. These resources are defined in the `template.yaml` file in this project.

## Lambda Function
* [app.py](apps/app.py): Handle URL redirection

## API Gateway
* /{arklabel}/53696/{shortid}/GET
* ```GET``` https://xxxx.execute-api.us-east-1.amazonaws.com/Prod/{arklabel}/53696/{shortid}
* arklabel: ```ark:```
* shortid: ```Noid```

## URL redirection
## DynamoDB Table
* [Table Schema](example/table_schema.json)
* [Sample record](example/record.json)


### Deploy VTDLP Resolution Service application using CloudFormation stack
#### Step 1: Launch CloudFormation stack
[![Launch Stack](https://cdn.rawgit.com/buildkite/cloudformation-launch-stack-button-svg/master/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?&templateURL=https://vtdlp-dev-cf.s3.amazonaws.com/7db2bd3b29f387d3ea3639726f7e535c.template)

Click *Next* to continue

#### Step 2: Specify stack details

| Name | Description |
|:--- |:------------|
| Stack name | any valid name |
| TargetTable | a DynamoDB table |
| Image404 | 404 Image URL |
| REGION | a valid AWS region. e.g. us-east-1 |

#### Step 3: Configure stack options
Leave it as is and click **Next**

#### Step 4: Review
Make sure all checkboxes under Capabilities section are **CHECKED**

Click *Create stack*

### Deploy VTDLP Resolution Service application using SAM CLI

To use the SAM CLI, you need the following tools.

* SAM CLI - [Install the SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html)
* [Python 3 installed](https://www.python.org/downloads/)
* Docker - [Install Docker community edition](https://hub.docker.com/search/?type=edition&offering=community)

To build and deploy your application for the first time, run the following in your shell:

```bash
sam build --use-container
```

Above command will build the source of the application. The SAM CLI installs dependencies defined in `requirements.txt`, creates a deployment package, and saves it in the `.aws-sam/build` folder.

To package the application, run the following in your shell:
```bash
sam package --output-template-file packaged.yaml --s3-bucket BUCKETNAME
```
curl https://xxxxx.execute-api.us-east-1.amazonaws.com/Prod/ark:/53696/8c10n70v
Above command will package the application and upload it to the S3 bucket you specified.

Run the following in your shell to deploy the application to AWS:
```bash
sam deploy --template-file packaged.yaml --stack-name STACKNAME --s3-bucket BUCKETNAME --parameter-overrides 'TargetTableName=resolutiontable Region=us-east-1 Image404=https://images/404.jpg' --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM --region us-east-1
```
Output

## Postdeployment
* URL redirection
```
{"statusCode": 301, "location": "https://lib.vt.edu/}
curl https://xxxxx.execute-api.us-east-1.amazonaws.com/Prod/ark:/53696/kr10gt01
```
* Output
```
{"statusCode": 301, "location": "https://iawa.lib.vt.edu/archive/kr10gt01"}
```

ps. Setting up custom domain names (e.g. http://idn.lib.vt.edu ) for REST APIs. [Guide](https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html)

## Cleanup

To delete the sample application that you created, use the AWS CLI. Assuming you used your project name for the stack name, you can run the following:

```bash
aws cloudformation delete-stack --stack-name resolution-service
```
Empty file added apps/__init__.py
Empty file.
35 changes: 35 additions & 0 deletions apps/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import boto3
import json
import os


region_name = os.getenv('Region')
table_name = os.getenv('TargetTable')
image_404 = os.getenv('Image404')
ddb = boto3.resource('dynamodb', region_name = region_name).Table(table_name)

def lambda_handler(event, context):

short_id = event.get('short_id')

try:
item = ddb.get_item(Key={'short_id': short_id})
long_url = item.get('Item').get('long_url')

# increase the hit number on the db entry of the url
ddb.update_item(
Key={'short_id': short_id},
UpdateExpression='set hits = hits + :val',
ExpressionAttributeValues={':val': 1}
)

except:
return {
'statusCode': 301,
'location': image_404
}

return {
"statusCode": 301,
"location": long_url
}
1 change: 1 addition & 0 deletions apps/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
requests
Loading

0 comments on commit 811057a

Please sign in to comment.