Creating an authenticated api from scratch, with django, django rest framework, and djoser.
- pipenv and installation of python 3.6+
- a restful testing program, this is going to act as our client (more on that soon):
- I'm going to do this all in Ubuntu, so instructions will be a bit different but similar for mac and windows.
- On windows you'll need to use powershell, WSL (windows subsystem for linux)
- On Mac you'll need to use terminal or bash.
- If you're using VSCode then you should have a terminal built in.
- git (if you want to clone the repo later.)
- Some basic knowledge of django is great but it's alright if you don't. What do you need to know?
- It's a web framework that you can serve small and large websites (how big? Instagram big).
-
talk about restful api
- Why it's important?
- What is it?
- Who uses it?
- Some places where you can discover some more RESTful APIs for yourself.
-
the full shabang of building our RESTful api.
- create our environment and install the packages.
- create a django project in our environtment.
- add the djangorestframework and djoser packages
- hook up the authentication api and configure it to our web project.
- create an app with models (and add them to the admin), serializers, views, urls (more on all of these later)
- create a RESTful api for the app above and hook it up to our web project.
-
If we have time we can discuss.
- Add some javascript that can "consume" our api.
- Deploy to heroku.
- Add a lot of data from an opendata platform using management commands.
A RESTful api is basically a way to transfer information from a server (where you store information and do tasks) to a client. A client can be a mobile app, a website (loaded on your computer), games that store high scores or anything that will "consume" information.
-
Used by mobile apps, all website you go to, iot devices and much more!
-
Used by data scientists (and machine learning) professionals to get their data.
-
Here's some RESTful APIs that you folks can use in your own projects:
- Awesome Public APIs: A list of public free APIs for your projects.
- Google API Library: Want to do something fancy with computer vision or anything on the google cloud platform? You can use those APIs in that link.
Note: We're essentially going to create one of these.
- we're going to be creating the api using the following packages:
- Django
- this is a web framework and an orm (Object Relationship Mapper) which is basically just an sql interface (so creating tables, and read/write/update/delete from those tables as well).
- Django Rest Framework
- this is going to allow us to really simplify the api creation process.
- We're also going to use this for login with tokens as well.
- Djoser
- this is going to handle our registration (through the api).
- Django
The following is going to be in your terminal/powershell/wsl.
-
Create a folder for your project.
- you could use
mkdir my-super-cool-project-lol
- you could use
-
Create a virtual environment
-
if you're using python virtualenvs (most of you)
python3 -m venv ./venv
- This is going to create a virtual environment so that you can install django and other packages isolated from other environments.
source venv/bin/activate
- This is going to start your environment.
python3 -m pip install <package name>
- you're going to use this to install the packages needed.
- if you're using pipenv (many of you won't so you can ignore the next two points)
- initialize your virtualenv with the following command
pipenv shell --python=3.7
- install the packages
pipenv install <package name>
(where ever you seepython3 -m pip install
you can replace it withpipenv install
)
Note: If you're totally stuck during the tutorial at any time you can do the following.
git clone https://github.com/dgmouris/zero_to_api.git
do a git clone of the repository- Create your environment, and init initial (look above)
- Install all the packages required.
python3 -m pip install -r requirements.txt
- You'll need to do this in a completely separate folder/environment, if this is your first time through just ignore this note and come back later if you're stuck.
- Django installation:
python3 -m pip install django
- this is going to install the latest version of django.
- Django rest framework installation
python3 -m pip install djangorestframework
- latest version.
- Django Djoser
python3 -m pip install djoser
- Django Cors Headers (so any client can access it.)
python3 -m pip install django-cors-headers
- Django has a set of command line programs to help you set up your project.
- if you type in django-admin in your shell then it should pop up with a bunch of options.
- We're going to use the
django-admin.py startproject
to begin the project.- I'm calling mine zero_to_api. (eg.
django-admin.py startproject zero_to_api
if you want to do the same.)
- I'm calling mine zero_to_api. (eg.
- Now you should see the barebones of your project (go into the folder
cd zero_to_api
your project name)- in the folder of your project you should see the following files:
manage.py
a file that helps gives us handy things that will help us with our project- migrations for our database (this is the orm part)
- run a local server so that we can test stuff on our computer
- this is a bit like
django-admin.py
, but more specific to your website. - a lot of other handy things that we might get to.
- a folder which has the same name as your project (
zero_to_api
) which contains the following filesurls.py
- this is going help us define our endpoints (ie. http://localhost:8000/v1/auth/token/ login, but note we haven't created this yet.)
settings.py
- all of the static settings variables that are essential to our app.
- as well it defines the apps (parts of our website) that are going to be used.
wsgi.py
- we're not going to worry about this today.
- For curious cats, this is how Django interfaces with webservers (More info here)
db.sqlite3
- this is going to be the database we're going to use for today.
- Note: This is only going to show up once you "migrate", we're looking at this next so it's going to pop up shortly.
- in the folder of your project you should see the following files:
-
go into your project folder. Let's see if we can run django! in your terminal run
python manage.py runserver
-
now if you look at your terminal you will see something like the following:
-
what this means is that django wants to make some stuff in our database and we also want it to do that!
-
running migrations (a simplified overview for our talk.)
python manage.py makemigrations
- this checks to see if anything changed in your project that will need to modify your database. Note: All of the models (pythonic way of interacting with the database) are from external packages, so this is only going to have changes if we create our own models.
- this is mostly going to apply when you're writing your own models (like tables in sql except for python)
python manage.py migrate
- this applies the migrations to your database.
- Having problems here can become a tough thing to cover as it can be a bit complicated, we won't be doing that today.
-
Let's run them now! in your terminal (that's inside your project) run the following command
python manage.py migrate
-
now if you run your local server (
python manage.py runserver
) you won't get any warnings.
Note: You should be able to see db.sqlite3
in your folder now too!
-
Let's add the authentication which will add the login endpoints to our website (you'll see)
-
first let's go to our settings.py
- go to your
INSTALLED_APPS
, and let's add a couple.- add the following lines within the existing list.
- go to your
INSTALLED_APPS = [
# my installed apps
'django.contrib.auth', # should already be there.
#...other packages ...
'rest_framework',
'rest_framework.authtoken',
'djoser',
'corsheaders'
]
- To the middleware you'll have to use CorsMiddleware right before the CommonMiddleware this is really really important if you want to use multiple clients. You'll also be specifying that any client can access our api with
CORS_ALLOW_ALL_ORIGINS = True
MIDDLEWARE = [
# other middlewares
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
# more middlewares
]
# Cors (you can add this to the bottom)
CORS_ALLOW_ALL_ORIGINS = True
Note: This makes it so that you can control who can access your website, you can make it so one or multiple domains (eg. www.github.com), one or multiple ips (eg. 127.0.0.1) can access your site.
-
now if you go back to your terminal and run your local server (
python manage.py runserver
) -
let's run our migrations again so that we can apply the changes to the database!
python manage.py makemigrations
- this will tell us no changes detected because we haven't changed/created models of our own, we just need to add new packages.
python manage.py migrate
- this is going to add some tables from the django-restauth package
- urls.py package add the following line in the
urlpatterns list
from django.contrib import admin
from rest_framework.authtoken import views
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('v1/auth/', include('djoser.urls')),
path('v1/auth/token/', views.obtain_auth_token),
]
That's what you urls.py
should look like!
- To be able to use this from your rest client you'll need specify the authentication classes that your project uses. We're using token authentication to create our api. In your
settings.py
add the following object.
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
'rest_framework.authentication.SessionAuthentication', # we're going to remove this
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.BasicAuthentication',
]
}
- Let's run our server now (
python manage.py runserver
) - if you go to http://localhost:8000 you'll get a 404
- this is not a bad thing. this is a good thing, let's go explore them.
- http://localhost:8000/admin/
- but I can't login! what's going on! oh my god dan has such cute cats and I can't concentrate!
- we'll deal with this very soon.
- http://localhost:8000/v1/auth/users/
- http://localhost:8000/admin/
- let's add our first user (yourself)
- go back to the terminal and write the following command
python manage.py createsuperuser
- let's go test it.
- run your server
- go to http://localhost:8000/admin/
- if you can login you've created your user!
- Now you can go to http://localhost:8000/v1/auth/users/ and you'll see yourself there.
- go back to the terminal and write the following command
- let's open your rest client! I'm using the "Firefox Rest Client" add on. Note if you don't have one of these please see the links in the "Prereqs section".
- Comment out the line in your
settings.py
in yourREST_FRAMEWORK
object
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
# 'rest_framework.authentication.SessionAuthentication', # we're going to remove this
# ...other auth classes...
]
}
- to walk you through this do the following steps.
- open your rest client
- use the "POST" method
- enter in the following url
http://localhost:8000/v1/auth/token/
- You may need to add the header:
Content-Type: application/json
- You may need to add the header:
- in the body enter the following parameters.
- username: the_user_name_you_used_earlier
- password: the_password_you_entered_earlier the body should look like this.
{
"username": "daniel",
"password": "temptemp"
}
-
!IMPORTANT! What this key (or token) is used for is to login your user later on, keep it on hand!
-
You'll need to add the
'rest_framework.authentication.SessionAuthentication'
line whenever you access the admin (more on that later).
Let's create an app (part of your website.) that requires authentication and we're going to make it accessible.
- how do we do this?
- First we're going to create the app. I'm calling mine cats.
- to do this you have to run the following line in your terminal (and your project root folder.)
python manage.py startapp cats
- you should now see a "cats" folder with a few things, that I'm going to cover as we need them.
- to do this you have to run the following line in your terminal (and your project root folder.)
- let's add cats to our installed apps so that our website knows to use that part of the site:)
- go to the zero_to_api folder in your project root.
- in the settings.py
INSTALLED_APPS
list add the following lines.
- in the settings.py
- go to the zero_to_api folder in your project root.
INSTALLED_APPS = [
# all of other apps.
'cats',
]
- What are apps?
- Apps are essentially a piece of functionality for your site, django tries to keep different pieces of functionality separate from other pieces. Account management for example, is different than any cats functionality we have in our app.
- Now let's first add your models in the
models.py
in your cats folder (or what ever your created.)- models are like tables in your database that you can manipulate with python.
- We're going to create two models and add them.
- Cat Breed
- Cat
- Here's what the models are going to look like.
from django.db import models
class CatBreed(models.Model):
name = models.CharField(max_length=255)
description = models.CharField(max_length=255)
def __str__(self):
return F"{self.name}"
class Cat(models.Model):
name = models.CharField(max_length=255)
cat_breed = models.ForeignKey(CatBreed,
on_delete=models.SET_NULL,
null=True)
def __str__(self):
return F"{self.name}, {self.cat_breed}"
- What are models you ask? Models do a few things in Django: - First generally maps to a single database table, the fields provide the "schema" (essentially structure) of that table. - It also gives you a pythonic interface to add remove update and delete data in your database.
Note: if you want to learn more about django models you can take a look here Model docs
-
Let's do the "makemigrations" because we've made changes to our own app that we've created with the following command
-
If that was successful then we can apply our migrations with the following command
-
What are Migrations you ask?
- Any changes in your models need to be reflected in your database tables (see Models as well)
- So "makemigrations" creates a special Django python file, that will create pending changes to our database (we can edit this too).
- Once we've done the above we execute "migrate" it's going to execute this file which will change our database (
db.sqlite3
for us if you're wondering)
-
Now that we have tables in our database, let's add them to the admin project so that we can add some data
- go to the admin.py in your cats folder and modify it the following contents.
from django.contrib import admin
from .models import Cat, CatBreed
admin.site.register(Cat)
admin.site.register(CatBreed)
This makes it so that you can add rows (data) to your models (in this context read: database tables) via the admin.
-
now go back and run your server and login to the admin page (http://localhost:8000/admin/)
- you should see
Cat
andCatBreed
models included.
- you should see
-
Click on each of them and add some data. I already have some:)
-
That's Great but....
- You guys want this to be accessible via an api! That's the goal of today!
- So let's create one.
Note: If you want to learn more about the admin you can go look at the admin docs
- let's make it accessible, so that we can add cats from our client!
- In your
cats
Folder we're going to create aserializers.py
file.- this is going to serialize our data, I'm not going to go into this in depth, but you can think of them as something that allows you have the functionality to modify your models, as well as get data from them via an api.
- for more information go to http://www.django-rest-framework.org/api-guide/serializers/
- our serializers are going to look like the following and are going to use the ModelSerializer, they look like this.
- this is going to serialize our data, I'm not going to go into this in depth, but you can think of them as something that allows you have the functionality to modify your models, as well as get data from them via an api.
from .models import Cat, CatBreed
from rest_framework import serializers
class CatBreedSerializer(serializers.ModelSerializer):
class Meta:
model = CatBreed
fields = '__all__'
class CatSerializer(serializers.ModelSerializer):
class Meta:
model = Cat
fields = '__all__'
- now that we have to create viewsets, in the django rest framework documentation they define it as follows.
"ViewSet classes are almost the same thing as View classes, except that they provide operations such as read, or update, and not method handlers such as get or put."
http://www.django-rest-framework.org/api-guide/viewsets/
- so you can think them as django views (which I won't get into today but you can ask me about it later.)
- in your views.py file your viewsets are pretty easy they use the following:
- a queryset which you can think of the data that your viewset is going to use.
- your serializer.
- your serializers will end up looking like the following if you look at the documentation:
from django.shortcuts import render
from rest_framework import viewsets
from .models import Cat, CatBreed
from .serializers import CatBreedSerializer, CatSerializer
class CatBreedViewSet(viewsets.ModelViewSet):
queryset = CatBreed.objects.all()
serializer_class = CatBreedSerializer
class CatViewSet(viewsets.ModelViewSet):
queryset = Cat.objects.all()
serializer_class = CatSerializer
- Let's hook it up to our urls so that we can use it!
- create a
urls.py
file in your cats folder. - now we're going to include a router which is a bit like urls in django, this adds our viewsets to a url path.
- it should look like this:
- create a
from rest_framework import routers
from .views import CatBreedViewSet, CatViewSet
router = routers.DefaultRouter()
router.register(r'cat-breeds', CatBreedViewSet)
router.register(r'cats', CatViewSet)
urlpatterns = router.urls
- This is going to add a bunch of paths to each url, a get, update, create, list and delete view of the items of your data in your remove
- Now that we have our urls defined in our app we have to add it to the overall web project.
- it's one line of code in the
zero_to_api/urls.py
file. the line is:
- it's one line of code in the
path('v1/catapp/', include('cats.urls')),
- This will connect all of the routes from our app to the entire project.
- run your localserver and go to the following destinations in your browser you should see something crazy cool!
- Congratulations you just created your first api!
- Can we access this from our REST Clients? Yes!
- Let's enable this for the post method so that we can add something.
- we need to enable token authorization so that we can post! or else we're going to get some csrf failure.
- remove the
rest_framework.authentication.SessionAuthentication
in youzero_to_api/settings.py
file. You might have already done this but this is super important!
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [
# 'rest_framework.authentication.SessionAuthentication', # Remove this now!!!
'rest_framework.authentication.TokenAuthentication',
'rest_framework.authentication.BasicAuthentication',
]
}
- this will also allow us post as well as give us functionality private.
- now if you look at the following picture you should be able to add a new cat! via an api (using arc) Note: On post requests you need to add a traling slash to all requests (unless you specify otherwise in the settings!)
Note: You can go here to learn more about django rest framework settings
- we don't really care if people can see our cat types, but we don't want people to know our cat names!
- let's make the cat types public, but let's make the cats private, where you need to login.
- in your viewsets.py file you need to "include permissions" add the following line if you don't want to make it public
# other imports ....
from rest_framework import viewsets, permissions # we added the permissions!
# ... other code
class CatViewSet(viewsets.ModelViewSet):
permission_classes = [permissions.IsAuthenticated] # we added this line!
queryset = Cat.objects.all()
serializer_class = CatSerializer
- the entire file should look like this:
from django.shortcuts import render
from rest_framework import viewsets, permissions
from .models import Cat, CatBreed
from .serializers import CatBreedSerializer, CatSerializer
class CatBreedViewSet(viewsets.ModelViewSet):
queryset = CatBreed.objects.all()
serializer_class = CatBreedSerializer
class CatViewSet(viewsets.ModelViewSet):
permission_classes = [permissions.IsAuthenticated]
queryset = Cat.objects.all()
serializer_class = CatSerializer
- now if you go to http://localhost:8000/v1/cats/ it'll show you that you don't have any credentials, and won't permit you yay!
- How can I test this? Rest Client to the rescue.
Note: If you want to learn more about permissions, you can go to the fantastic documentation here permissions docs If you want to learn more about viewsets you can look here (viewset docs)[https://www.django-rest-framework.org/api-guide/viewsets/]
- go to your advanced rest client, use the "POST" method and put in the login url (http://localhost:8000/v1/auth/token)
- put in your credentials in the body (like we did the first time) and copy the key somewhere handy so you can copy it. if you need a refresher see:
- now we're going to access our stuff via token authentication!
- Use the "GET" method and enter the url "http://localhost:8000/v1/catapp/cats/"
- in your advanced rest client select the "Headers" section
- in the header name enter "Authorization"
- in the header value enter "Token "
- it should look like the following:
- Djoser will allow you to sign up users and deal with all of the complications of doing that.
- You'll need to add a dummy backend for email, you'll need to configure one at some point if you want to deploy. The following is going to go into your
settings.py
.
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'
This is essentially going to show you emails in your console, which is what we need for now.
- If you go to
http://localhost:8000/v1/auth/users/
you should be able to register a new user. - You can also go to the rest client and use the "POST" method to
http://localhost:8000/v1/auth/users/
with the following body:
{
"email": "rick@rick.com",
"username": "rick",
"password": "temptemp"
}
- then you can login at
http://localhost:8000/v1/auth
with the body
{
"username":"rick",
"password": "temptemp"
}
- and then access our private cats api here
http://localhost:8000/v1/catapp/cats/
with the HeaderAuthorization: Token <token from above here>
to see the information.
- What have we done here?
- created and used a virtual environemnt.
- created a django project
- added the django, djangorestframework, djoser, django-cors-headers packages
- hooked up the authentication api and configured it to our web project.
- created an app with models (and added them to the admin), serializers, views, urls,
- created an api for the app above and hooked it up to our web project.
- This code doesn't use best practices necessarily so look them up if you are worried.
- Feel free to use this, I hope it was helpful.
- If you liked this talk/tutorial and you want to help me, star the repo, add me on linkedin if you'd like as normally I share when edmontonpy is.
- you've reated your first RESTful so now go conquer the world! Apps, Websites, IOT, and what ever else you want to do!
- Hopefully this was useful and you're as excited as Steve Ballmer.
- Add some javascript that can "consume" our api.
- Deploy to heroku.
- Add a lot of data from an opendata platform using management commands.
You're thinking to yourself, I can do this and I want to learn more. Here's some more resources that you can go check out.