Skip to content

Latest commit

 

History

History
155 lines (115 loc) · 5.01 KB

s3buckets.md

File metadata and controls

155 lines (115 loc) · 5.01 KB
Raw

Using S3(Simple storage service) to save media files and other files:

If your users are going to be uploading images such as profiles etc, you can also serve static files from S3 but in this, we will focus on how to serve media files. You should definitely use S3 to store them and serve them directly from S3 instead of Django serving those files.

To set up S3 follow the below instructions:

  1. Install the following packages and put them in requirements.txt file:
pip install boto3
pip install django-storages

requirements.txt

boto3==1.20.26
django-storages==1.12.3

add storages in installed apps:

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',

    'storages', # <----- here   
    'my-app'
]

django-storages is a library to manage storage backends like Amazon S3, OneDrive, etc.

boto3 is a public API client to access the AWS resources such as AWS S3.

  1. Go to S3 by searching for it on AWS console:

  2. Click on create bucket

Create-bucket

  1. Now give the bucket a name of your choice and uncheck block all public access. (make sure the region you are in is the same as in the EC2 instance)

  2. Now scroll down and click on create bucket.

Creating IAM Role:

You ideally don't want your django-app to be able to access all the services by AWS for security reasons. So you create an account that has access only to the S3 buckets from the IAM role.

Now search for IAM in the search bar:

  1. Click on Users from the left navbar and click on "Add users".

IAM

  1. Type in a user name and click on Access-key: programmatic access: programmatic-access

  2. Click on next and under "Add user to group" click on create group and search for S3 full access, check the check-box. Create-group

  3. Click on review and if everything seems correct then you can click on create user.

  4. Take note of all the information including the User, Access Key and the Secret Access Key. We will be using this to give Django permission to access the S3 buckets.

Now in your settings.py add the below(you can add this anywhere in your settings I'll be adding it at the bottom):

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
AWS_STORAGE_BUCKET_NAME = os.environ.get('AWS_STORAGE_BUCKET_NAME')

AWS_S3_REGION_NAME = os.environ.get('AWS_S3_Region')
AWS_QUERYSTRING_AUTH = False
AWS_S3_CUSTOM_DOMAIN = f'{AWS_STORAGE_BUCKET_NAME}.s3.amazonaws.com'  
AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}
AWS_DEFAULT_ACL = 'public-read'

You should add the access key id etc in .env file and in production you should add them as environment variables. (read the Hiding sensitive information topic).

Now in the same directory as settings.py create a file called storages.py and add the following.

from storages.backends.s3boto3 import S3Boto3Storage

class MediaStore(S3Boto3Storage):
    location = 'media'
    file_overwrite = False

now back in settings.py add

DEFAULT_FILE_STORAGE = '<project_name>.storage.MediaStore'

replace <project_name> with your project name. The above will work if the storages.py is in the same directory as settings.py If you have it somewhere else provide the path to that eg: app.storages.MediaStore.

Creating bucket policy

Once you are done with the setup, go back to s3 in aws console. Click on your bucket -> permissions tab. Now set "block all public access" to off. Now edit the bucket policy and set the below code to allow only read access

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::media-name",
                "arn:aws:s3:::media-name/*"
            ]
        }
    ]
}

replace arn:aws:s3:::media-name with the arn you get when you click on edit bucket policy.

Cross-origin resource sharing (CORS)

Sometimes the browser might report cors error so scoll down in the permission tab to the CORS policy and add the following:

[
    {
        "AllowedHeaders": [
            "Authorization"
        ],
        "AllowedMethods": [
            "GET",
            "HEAD"
        ],
        "AllowedOrigins": [
            "http://mydomain.com",
            "https://mydomain.com"
        ],
        "ExposeHeaders": [
            "Access-Control-Allow-Origin"
        ]
    }
]

That's it, it should now be up and running.