Training Guide

User Manual:

Open the PDF directly: View PDF .
Page Count: 44

Download
Open PDF In Browser	View PDF

Ashland Daily Photo
http://traveljapanblog.com/ashland/2012/08/portland-oregon-from-the-air/

Django Rest Framework, API
Development, and Tooling

High-level takeaways
●

Development pattern/what are we building?

●

Workﬂow overview

●

Project creation with cookiecutter

●

API development with Django Rest Framework

●

GIS Support (GeoDjango, django-rest-framework-gis, PostGIS)

●

Tools/Deployment

What are we doing?
●

●

Hack Oregon doesn’t just build apps, we build systems.
○

APIs provide a public facing gateway to our data

○

Our APIs will power our front end applications as well as future data science

We work with Django as our web framework
○

●

“Batteries included model”

3rd party Django Rest Framework applications
○
○

Third Party Packages allow developers to share code that extends the functionality of Django
REST framework, in order to support additional use-cases
Installable through a package repository

A look at our existing deployment chain ...

API Deployment Quickview

Cookiecutter
●

Cookiecutter is a templating engine

●

Simpliﬁes our process of building APIs and distributable packages
○

Provides boilerplate and development tooling w/minimal conﬁguration

○

Help developers get up and coding faster

Create a Project

●
●

Install cookiecutter

pip install cookiecutter

●

cd 
git init

●

Use cookiecutter to create repo from
github template

cookiecutter
gh:hackoregon/2019-backend-cookiecutter-django

●

Follow prompts to ﬁll in templates

Create git repo

●
git
git
git
git

Create a remote repo in
Hack Oregon
organization (matching
the “github_repo”
template value
Make initial commit
remote add origin 
add .
commit -m "initial commit"
push origin master

●

Make new branch

git checkout -b
cool-new-feature

●

Create .env and set

cp env.sample .env

●

Build container

./bin/build.sh -d

●

Start container

./bin/start.sh -d

●

Open url in browser

api_1 | [2019-05-18
16:59:44 +0000] [26] [INFO]
Listening at:
http://0.0.0.0:8000 (26)

●

Start coding

Project Naming Conventions
hack_oregon_team - what project/theme team you are working on
Ex: Transportation Systems
python_package_namespace - this should generally be: hackoregon_< your team>
Ex: hackoregon_transportation_systems
python_subpackage - name of the more specific application/module you are developing. Should generally be able to stand alone as
individual/reusable package of code, though can pull in other packages as dependencies
Ex: passenger_census, toad, biketown
*you may have more than one subpackage, but this cookiecutter will spin up first one automagically. A pattern for adding additional
subpackages will be provided

Project Naming Conventions
version - you will generally start with 0.1.0 as a first version, with version 1.X being the production/demo day release (we will provide
more info on versioning)
gis - whether this project should include django-rest-framework-gis package as a dependency (can be added later manually by
updating setup.py)
Ex: True or False
"data_science_repo_url" - URL of repo where team data science work is contained "data_science_repo_name" - Name of repo where
data science work is contained

Create a Project (Demo)

●
●

Install cookiecutter

pip install cookiecutter

●

cd 
git init

●

Use cookiecutter to create repo from
github template

cookiecutter
gh:hackoregon/2019-backend-cookiecutter-django

●

Follow prompts to ﬁll in templates

Create git repo

●
git
git
git
git

Create a remote repo in
Hack Oregon
organization (matching
the “github_repo”
template value
Make initial commit
remote add origin 
add .
commit -m "initial commit"
push origin master

●

Make new branch

git checkout -b
cool-new-feature

●

Create .env and set

cp env.sample .env

●

Build container

./bin/build.sh -d

●

Start container

./bin/start.sh -d

●

Open url in browser

api_1 | [2019-05-18
16:59:44 +0000] [26] [INFO]
Listening at:
http://0.0.0.0:8000 (26)

●

Start coding

Join a Project

●

Clone the repo

git clone 

●

Make new branch

git checkout -b cool-new-feature

●

Create .env and set

cp env.sample .env

●

Build container

./bin/build.sh -d

●

Start container

./bin/start.sh -d

●

Open url in browser

api_1 | [2019-05-18
16:59:44 +0000] [26] [INFO]
Listening at:
http://0.0.0.0:8000 (26)

●

Start coding

2019-backend-cookiecutter-django
●

Let’s look at the template

●

Base README, LICENSE, contribution, ignore ﬁles…

●

Dockerﬁle and docker-compose

●

.travis.yml

●

Pytest conﬁguration

●

Pypi deployment

●

Local_settings and api conﬁguration

Building an API
●

Django Rest Framework APIs
○
○
○
○
○
○

●

Conﬁguration
Testing
Models
Serializers
Views/Viewsets
Routes

Other Concepts
○
○
○

Filtering
Pagination
Status codes/exceptions

whitenoise is
used in place of
Web Server in
this context

DRF uses
serializers to
serve json data
Does not use
templates
directly outside
of swagger
frontend

Building an API - Conﬁguration
●

Django Conﬁguration
○

●

Docker project provides some basic conﬁguration
○

●

Conﬁgured in the settings.py ﬁle
https://github.com/hackoregon/2019-backend-docker/blob/staging/backend/hacko_settings.py

You can override in local project
○

https://github.com/hackoregon/2019-backend-cookiecutter-django/blob/fixed/%7B%7Bcookiecutter.github_repo%7D%7
D/local_settings/settings.py

●

Objects need to be “set” in total, cannot append
○

●
●

Important to include all “installed apps” in you local settings

We are conﬁgured to use whitenoise for static hosting/Gunicorn for web
server
“Debug” variable

Building an API - Testing
●
●
●

Pytest-django
Coverage
Live Tests (against production database)
○
○
○

●
●

https://www.django-rest-framework.org/api-guide/testing/#live-tests

Test your code, not Django ORM
APIClient
○
○
○

●

Conﬁguration to prevent tear down/change of production data
Works locally and in travis

Test api request/response
Status codes
Logic

More resiliant code against updates/changes

Building an API - Models
●

A model is the single, deﬁnitive source of information about your data. It contains the essential ﬁelds and behaviors of the data you’re
storing. Generally, each model maps to a single database table.

●

The above Person model would create a database table like this:

Building an API - Models
●

Auto-generation of model from an existing database (run in docker interactive shell)

docker exec -it  bash
python manage.py inspectdb > //models.py

●

Gets you part-way there
○
○
○

Validate data types
Check relationships
If using multi-database router:
■ import django.db.models.options

as options

options.DEFAULT_NAMES = options.DEFAULT_NAMES + ('in_db',)

○

Add to meta ﬁelds:
■ in_db = 'passenger_census'

Building an API - Models (Demo)
class PassengerCensus(models.Model):
summary_begin_date = models.DateField(blank=True, null=True)
route_number = models.IntegerField(blank=True, null=True)
direction = models.IntegerField(blank=True, null=True)
service_key = models.TextField(blank=True, null=True)
stop_seq = models.IntegerField(blank=True, null=True)
location_id = models.IntegerField(blank=True, null=True)
public_location_description = models.TextField(blank=True, null=True)
ons = models.IntegerField(blank=True, null=True)
offs = models.IntegerField(blank=True, null=True)
x_coord = models.FloatField(blank=True, null=True)
y_coord = models.FloatField(blank=True, null=True)
geom_4326 = models.GeometryField(blank=True, null=True)
census_block = models.CharField(max_length=255, blank=True, null=True)
class Meta:
managed = False
db_table = 'passenger_census'
in_db = 'passenger_census'

Building an API - Serializers
●
●

●
●
●
●

Serializer fields handle converting between primitive values and internal datatypes. They also deal with validating input values, as well
as retrieving and setting the values from their parent objects.
Can be very simple:
○
class AnnualRouteRidershipSerializer(serializers.ModelSerializer):
class Meta:
model = AnnualRouteRidership
fields = '__all__'
Use the “ModelSerializer” when working with serializers which meet up with models (or “GeoFeatureModelSerializer”, more on this
later)
Can specify only specific fields (via “fields”), but can also customize/add fields using the serializer fields (important when creating
more custom views)
If serializer is returning more then one object, then set many=True
Relationships generally look at the primary key, but you can provide more control/also set depth

Building an API - Serializers (Demo)
class PassengerCensusSerializer(serializers.GeoFeatureModelSerializer):
class Meta:
model = PassengerCensus
geo_field = 'geom_4326'
id = 'id'
fields = '__all__'

class PassengerCensusInfoSerializer(serializers.ModelSerializer):
class Meta:
model = PassengerCensus
fields = ['summary_begin_date', 'service_key']
total_routes = IntegerField()

Building an API - Views and Viewsets
REST framework provides an APIView class, which subclasses Django's View class.
APIView classes are different from regular View classes in the following ways:

●

Requests passed to the handler methods will be REST framework's Request instances, not Django's HttpRequest instances.

●

Handler methods may return REST framework's Response, instead of Django's HttpResponse. The view will manage content
negotiation and setting the correct renderer on the response.

●

Any APIException exceptions will be caught and mediated into appropriate responses.

●

Incoming requests will be authenticated and appropriate permission and/or throttle checks will be run before dispatching the
request to the handler method.

https://www.django-rest-framework.org/api-guide/views/

Building an API - Views and Viewsets
●
●
●

●

●

Views in Django can be function based or class based
Most likely you will stick with class based views (https://wsvincent.com/class-function-based-views/)
Building off class-based views, you can use Generic Class based views
○
These already include some of the basic functionality needed in a view
○
http://www.cdrf.co/
Finally, you can combine similar logic into a “viewset”
○
Django REST framework allows you to combine the logic for a set of related views in a single class, called a ViewSet. In other
frameworks you may also find conceptually similar implementations named something like 'Resources' or 'Controllers'.
If you only need list/retrieve access to a model, let the ReadOnlyModelViewset handle the work
○
https://www.django-rest-framework.org/api-guide/viewsets/#readonlymodelviewset

Building an API - Views and Viewsets (Demo)
class PassengerCensusViewSet(viewsets.ViewSetMixin, generics.ListAPIView):
"""
This viewset will provide a list of individual Passenger Census counts by TRIMET.
"""
queryset = PassengerCensus.objects.all()
filter_backends = (PassengerCensusListFilter,)
serializer_class = PassengerCensusSerializer

Building an API - Views and Viewsets (Demo)
class PassengerCensusInfoViewSet(viewsets.ViewSetMixin, generics.ListAPIView):
"""
This viewset will provide a info about the Passenger Census by TRIMET.
Returns:
Distinct Census Dates
The total number of routes for the census
The total number of stops for the census
"""
serializer_class = PassengerCensusInfoSerializer
def list(self, request, *args, **kwargs):
census = PassengerCensus.objects.all()
census = getCensusTotals(census)
return Response(census)

Building an API - Views and Viewsets (Demo)
class PassengerCensusInfoViewSet(viewsets.ViewSetMixin, generics.ListAPIView):
"""
This viewset will provide a info about the Passenger Census by TRIMET.
Returns:
Distinct Census Dates
The total number of routes for the census
The total number of stops for the census
"""
serializer_class = PassengerCensusInfoSerializer
def list(self, request, *args, **kwargs):
census = PassengerCensus.objects.all()
census = getCensusTotals(census)
return Response(census)

Building an API - Routing
●
●
●
●

Routing will connect url paths with the specific view which should be returned
Happens in the urls.py file
○
https://www.django-rest-framework.org/api-guide/routers/
REST framework adds support for automatic URL routing to Django, and provides you with a simple, quick and consistent way of
wiring your view logic to a set of URLs.
https://www.django-rest-framework.org/api-guide/routers/#defaultrouter

Building an API - Routing (Demo)

Building an API - Other Concepts
●
●
●

Filtering
Pagination
Status Codes

Building an API - Filtering
●
●
●

●
●

The default behavior of REST framework's generic list views is to return the entire queryset for a model manager. Often you will want
your API to restrict the items that are returned by the queryset.
We will want to filter based on url query parameters
Django filter backend is included in our template
○
https://www.django-rest-framework.org/api-guide/filtering/#djangofilterbackend
○
Provides basic extensions including the “filter_fields” metafield on views
Can create custom filters
○
https://www.django-rest-framework.org/api-guide/filtering/#custom-generic-filtering
Other controls can be exposed to the schema through custom filters
○
https://www.django-rest-framework.org/api-guide/filtering/#pagination-schemas

Building an API - Pagination
REST framework includes support for customizable pagination styles. This allows you to modify how large result sets are split into individual
pages of data.
The pagination API can support either:
●

Pagination links that are provided as part of the content of the response.

●

Pagination links that are included in response headers, such as Content-Range or Link.

The built-in styles currently all use links included as part of the content of the response. This style is more accessible when using the
browsable API.
Pagination is only performed automatically if you're using the generic views or viewsets. If you're using a regular APIView, you'll need to call
into the pagination API yourself to ensure you return a paginated response. See the source code for the mixins.ListModelMixin and
generics.GenericAPIView classes for an example.

https://www.django-rest-framework.org/api-guide/pagination/

Building an API - Status Codes and Exceptions
Using bare status codes in your responses isn't recommended. REST framework includes a set of named constants that you can use to make
your code more obvious and readable.
●
●
●
●

DRF provides for full gamut of status codes
○
https://www.django-rest-framework.org/api-guide/status-codes/
Use appropriately
https://www.django-rest-framework.org/api-guide/exceptions/
If you build custom views will need to handle expections

Building an API - A second on performance
Django Rest Framework performance relies on similar concepts to Django.
As said in our Postgres Training, “First Get it Right”.
Read this:
●
●

https://docs.djangoproject.com/en/2.2/topics/performance/#
https://docs.djangoproject.com/en/2.2/topics/db/optimization/#retrieve-everything-at-once-if-you-know-you-will-need-it

Test with jMeter:
●

https://jmeter.apache.org/

GIS, GeoDjango, and django-rest-framework-gis
●
●

●
●
●

GeoDjango intends to be a world-class geographic Web framework. Its goal is to make it as easy as possible to build GIS Web
applications and harness the power of spatially enabled data.
Requires some lower level c-libraries
○
GEOS - Geometry Engine Open Source
○
GDAL - Geospatial Data Abstraction Library
○
PROJ.4 - Cartographic Projections library
PostGIS enabled databases support our backend, provides native support for geo datatypes
Geodjango can be used independently of database/speciﬁc django settings if you need
GIS support is brought to DRF via the “django-rest-framework-gis” package
○
https://github.com/djangonauts/django-rest-framework-gis
○
GeoFeatureModelSerializer
○
GeoFilters
○
“Bounding box”
○
https://github.com/djangonauts/django-rest-framework-gis#geofeaturemodelserializer

Geospatial DEMO

Versioning
●

Github
○
○

●

Tagged and released versions
https://help.github.com/en/articles/creating-releases

PyPi (Python Package Index)
○

PyPI helps you find and install software developed and shared by the Python community. Learn about
installing packages.

○
○
○

https://pypi.org/

Releases are versioned
Think of a requirements file

Python Package Index and Deployment
●
●

PyPi (Python Package Index) - what is it?
Setup.py
○

●

Developing locally you will do a local import
○

●
●

Provides commands to create and publish the individual python package
-e ./{{cookiecutter.python_package_namespace}}

Travis will handle deployment to PyPi
We are using an implicit namespace to identify our org and team
○
○

https://www.python.org/dev/peps/pep-0420/

Python 3.x convention

Documentation
●
●
●
●

Readme
Swagger/OpenAPI
Docstrings
Documentation generation

Workﬂow tools
●

Travis and deployment
●

More Information
●
●
●
●
●
●

https://www.django-rest-framework.org/
http://www.cdrf.co/
https://www.docker.com/
https://github.com/djangonauts/django-rest-framework-gis
https://www.django-rest-framework.org/api-guide/schemas/
https://swagger.io/docs/specification/about/

Thank you!
Brian Grant

Source Exif Data:

File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.4
Linearized                      : No
Warning                         : Root object (4 0 obj) not found at 16

EXIF Metadata provided by EXIF.tools

Training Guide

Navigation menu

Versions of this User Manual:

Views

Navigation