Skip to content
This repository has been archived by the owner on Jan 7, 2025. It is now read-only.

idealo/terraform-emr-pyspark

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Terraform + EMR Bootstrap PySpark with Anaconda

This code should help to jump start PySpark with Anaconda on AWS using Terraform.

This project is no longer actively maintained and has been archived on Jan 7, 2025.

Getting Started

  1. Install Terraform on Mac: brew install terraform
  2. Adjust the scripts (bootstrap_actions.sh and pyspark_quick_setup.sh) in scripts if necessary
  3. Set parameters in terraform.tfvars
  4. Start cluster:
terraform init
terraform apply
  1. Destroy cluster:
terraform destroy

Notes

  • Configure AWS on your local machine: aws configure
  • AWS instance cost for eu-central-1

Maintainers

Copyright

See LICENSE for details.