Various programming stuff

A comprehensive Django CBV guide

Class Based Views (CBV) is one of my favourite things about Django. During my first Django projects (using Django 1.4 around 6 years ago) I was mainly using functional views — that’s what the tutorial recommended then anyway. However, slowly in my next projects I started reducing the amount of functional views and embracing CBVs, slowly understanding their usage and usefulness. Right now, I more or less only use CBVs for my views; even if sometimes it seems more work to use a CBV instead of a functional one I know that sometime in the future I’d be glad that I did it since I’ll want to re-use some view functionality and CBVs are more or less the only way to have DRY views in Django.

I’ve heard various rants about them, mainly that they are too complex and difficult to understand and use, however I believe that they are not really difficult when you start from the basics. Even if it is a little work to become comfortable with the CBV logic, when they are used properly they will greatly improve your Django experience so it is definitely worth it.

Notice that to properly understand CBVs you must have a good understanding of how Python’s (multiple) inheritance and MRO work. Yes, this is a rather complex and confusing thing but I’ll try to also explain this as good as I can to the first chapter of this article so if you follow along you shouldn’t have any problems.

This guide has four parts:

  • A gentle introduction to how CBVs are working and to the the problems that do solve. For this we’ll implement our own simple Custom Class Based View variant and take a look at python’s inheritance model.
  • A high level overview of the real Django CBVs using CBV inspector as our guide.
  • A number of use cases where CBVs can be used to elegantly solve real world problems
  • Describing the usage of some of the previous use cases to a real Django application

I’ve implemented an accompanying project to this article which you can find at https://github.com/spapas/cbv-tutorial. This project has two separate parts. One that is the implementation of the Custom Class Based View variant to see how it is working and the other is the application that contains the usage of the various CBV use cases.

A gentle introduction to CBVs

In this part of the guide we’ll do a gentle introduction to how CBVs work by implementing our own class based views variant - along with it we’ll introduce and try to understand some concepts of python (multiple) inheritance and how it applies to CBVs.

Before continuing, let’s talk about the concept of the “view” in Django: Django is considered an MVT (Model View Template) framework - the View as conceived by Django is not the same as the MVC-View. A Django View is more or less a way to define the data that the Template (which is cloased to the MVC-View) will display, so the Django View (with the help of the Django Framework) is similar to the MVC-Controller.

In any case, traditionally a view in Django is a normal python function that takes a single parameter, the request object and must return a response object (notice that if the view uses request parameters for example the id of an object to be edited they will also be passed to the function). The responsibility of the view function is to properly parse the request parameters and construct the response object - as can be understood there is a lot of work that need to be done for each view (for example check if the method is GET or POST, if the user has access to that page, retrieve objects from the database, crate a context dict and pass it to the template to be rendered etc).

Now, since functional views are simple python functions it is not easy to override, reuse or extend their behaviour. There are more or less two methods for this: Use function decorators or pass extra parameters when adding the view to your urls. I’d like to point out here that there’s a third method for code-reuse: Extracting functionality to common and re-usable functions or classes that will be called from the functional views but this is not something specific to Django views but a general concept of good programming style which you should follow anyway.

The first one uses python decorators to create a functional view that wraps the initial one. The new view is called before the initial one, adds some functionality (for example check if the current user has access, modify request parameters etc), calls the initial one which will return a response object, modify the response if needed and then return that. This is how login_required works. Notice that by using decorators you can change things before and after the original view runs but you can’t do anything about the way the original view works.

For the second one (adding extra view parameters) you must write your function view in a way which allows it to be reused, for example instead of hard-coding the template name allow it to be passed as a parameter or instead of using a specific form class for a form make it configurable through a parameter. Then, when you add this function to your urls you will pass different parameters depending on how you want to configure your view. Using this method you can override the original function behaviour however there’s a limit to the number of parameters you can allow your function views to have and notice that these function views cannot be further overridden. The login authentication view (which is now deprecated in favour of a CBV one) is using this technique, for example you can pass it the template name that will be used, a custom authentication form etc.

It should be obvious that both these methods have severe limitations and do not allow you to be as DRY as you should be. When using the wrapped views you can’t actually change the functionality of the original view (since that original function needs to be called) but only do things before and after calling it. Also, using the parameters will lead to spaghetti code with multiple if / else conditions in order to take into account the various cases that may arise. All the above lead to very reduced re-usability and DRYness of functional views - usually the best thing you can do is to gather the common things in external normal python functions (not view functions) that could be re-used from other functional views as already discussed.

Class based views solve the above problem of non-DRY-ness by using the well known concept of OO inheritance: The view is defined from a class which has methods for implementing the view functionality - you inherit from that class and override the parts you want so the inherited class based view will use the overridden methods instead of the original ones. You can also create re-usable classes (mixins) that offer a specific functionality to your class based view by implementing some of the methods of the original class. Each one of your class based views can inherit its functionality from multiple mixins thus allowing you to define a single class for each thing you need and re-using it everywhere. Notice of course that this is possible only if the CBVs are properly implemented to allow overriding their functionality. We’ll see how this is possible in the next section.

Hand-made CBVs

To make things more clear we’ll start implementing our own class based views hierarchy. Here’s a rather naive first try:

class CustomClassView:
    context = []
    header = ''

    def __init__(self, **kwargs):
        self.kwargs = kwargs
        for (k,v) in kwargs.items():
            setattr(self, k, v)

    def render(self):
        return """
                header=self.header, body='<br />'.join(self.context),

    def as_view(cls, *args, **kwargs):
        def view(request, ):
            instance = cls(**kwargs)
            return HttpResponse(instance.render())

        return view

This class can be used to render a simple HTML template with a custom header and a list of items in the body (named context). There are two things to notice here: The __init__ method (which will be called as the object’s constructor) will assign all the keyword arguments (kwargs) it receives as instance attributes (for example CustomClassView(header='hello') will create an instance with 'hello' as its header attribute). The as_view is a class method (i.e it can be called directly on the class without the need to instantiate an object for example you can call CustomClassView.as_view() ) that defines and returns a traditional functional view (named view) that will be used to actually serve the view. The returned functional view is very simple - it just instantiates a new instance (object) of CustomClassView passing the kwargs it got in the constructor and then returns a normal HttpResponse with the instance’s render() result. This render() method will just output some html using the instance’s header and context to fill it.

Notice that the instance of the CustomClassView inside the as_view class method is not created using CustomClassView(**kwargs) but using cls(**kwargs) - cls is the name of the class that as_view was called on and is actually passed as a parameter for class methods (in a similar manner to how self is passed to instance methods). This is important to instantiate an object instance of the proper class.

For example, if you created a class that inherited from CustomClassView and called its as_view method then when you use the cls parameter to instantiate the object it will correctly create an object of the inherited class and not the base one (if on the other hand you had used CustomClassView(**kwargs) to instantiate the instance then the as_view method of the inheriting classes would instantiate instances of CustomClassView so inheritance wouldn’t really work!).

To add the above class method in your urls, just use its as_view() as you’d normally use a functional view:

from django.conf.urls import include, url
from . import views

urlpatterns = [
    url(r'^ccv-empty/$', views.CustomClassView.as_view(), name='ccv-empty'),
    # ... other urls

This doesn’t actually render anything since both header and context are empty on the created instance — remember that as_view returns a functional view that instantiates a CustomClassView objet and returns an HttpResponse filling it with the object’s render() reuslts. To add some output we can either create another class that inherits from CustomClassView or initialize the attributes from the constructor of the class (using the kwargs functionality described above).

The inherited class can just override the values of the attributes:

class InheritsCustomClassView(CustomClassView, ):
    header = "Hi"
    context = ['test', 'test2' ]

And then just add the inherited class to your urls as before:

url(r'^ccv-inherits/$', views.InheritsCustomClassView.as_view(), name='ccv-inherits'),

The as_view() method will create an instance of InheritsCustomClassView that has the values configured in the class as attributes and return its render() output as response.

The other way to configure the attributes of the class is to pass them to the as_view class method (which in turn will pass them to the instances constructor which will set the attributes in the instance). Here’s an example:

url(r'^ccv-with-values/$', views.CustomClassView.as_view(header='Hello', context=['hello', 'world', ], footer='Bye', ), name='ccv-with-values'),

The above will create a CustomClassView instance with the provided values as its attributes.

Although this method of configuration is used in normal django CBVs (for example setting the template_name in a TemplateView) I recommend you avoid using it because passing parameters to the as_view method pollutes the urls.py with configuration that (at least in my opinion) should not be there (and there’s no reason to have to take a look at both your urls.py and your views.py to understand the behavior of your views) and also, even for very simple views I know that after some time I’ll need to add some functionality that cannot be implemented by passing the parameters so I prefer to bite the bullet and define all my views as inherited classes so it will be easy for me to further customize them later (we’ll see how this is done in a second). Thus, even if you have

In any case, I won’t discuss passing parameters to the as_view method any more, so from now on any class based views I define will be added to urls py using ClassName.as_view() without any parameters to the as_view() class method.

Is this really DRY ?

Let’s now suppose that we wanted to allow our class based view to print something on the header even if no header is provided when you configure it. The only way to do it would be to re-define the render method like this:

def render(self):
    header=self.header if self.header else "DEFAULT HEADER"
    return """
            header=header, body='<br />'.join(self.context),

This is definitely not the DRY way to do it because you would need to re-define the whole render method. Think what would happen if you wanted to print "ANOTHER DEFAULT HEADER" as a default header for some other view - once again re-defining render! In fact, the above CustomClassView is naively implemented because it does not allow proper customization through inheritance. The same problems for the header arise also when you need modify the body; for example, if you wanted to add an index number before displaying the items of the list then you’d need to again re-implement the whole render method.

If that was our only option then we could just stick to functional views. However, we can do much better if we define the class based view in such a way that allows inherited classes to override methods that define specific parts of the functionality. To do this the class-based-view must be properly implemented so each part of its functionality is implemented by a different method.

Here’s how we could improve the CustomClassView to make it more DRY:

class BetterCustomClassView(CustomClassView, ):
    def get_header(self, ):
        print ("Better Custom Class View")
        return self.header if self.header else ""

    def get_context(self , ):
        return self.context if self.context else []

    def render_context(self):
        context = self.get_context()
        if context:
            return '<br />'.join(context)
        return ""

    def render(self):
        return """
                header=self.get_header(), body=self.render_context(),

So what happens here? First of all we inherit from ClassClassView to keep the as_view method which doesn’t need changing. Beyond this, the render uses methods (get_header and render_context) to retrieve the values from the header and the body - this means that we could re-define these methods to an inherited class in order to override what these methods will return. Beyond get_header and render_contex I’ve added a get_context method that is used by render_context to make this CBV even more re-usable. For example I may need to configure the context (add/remove items from the context i.e have a CBV that adds a last item with the number of list items to the list to be displayed). Of course this could be done from render_context but this means that I would need to define my new functionality (modifying the context items) and re-defining the context list formatting. It is much better (in my opinion always) to keep properly separated these things.

Now, the above is a first try that I created to mainly fulfil my requirement of having a default header and some more examples I will discuss later (and keep everything simple enough). You could extract more functionality as methods-for-overriding, for example the render method could be written like this:

def render(self):
    return self.get_template().format(
            header=self.get_header(), body=self.render_context(),

and add a get_template method that will return the actual html template. There’s no hard rules here on what functionality should be extracted to a method (so it could be overridden) however I recommend to follow the YAGNI rule (i.e implement everything as normal and when you see that some functionality needs to be overridden then refactor your code to extract it to a separate method).

Let’s see an example of adding the default header functionality by overriding get_header:

class DefaultHeaderBetterCustomClassView(BetterCustomClassView, ):
    def get_header(self, ):
        return self.header if self.header else "DEFAULT HEADER"

Classes inheriting from DefaultHeaderBetterCustomClassView can choose to not actually define a header attribute so "DEFAULT HEADER" will be printed instead. Keep in mind that for DefaultHeaderBetterCustomClassView to be actually useful you’ll need to have more than one classes that need this default-header functionality (or else you could just set the header attribute of your class to "DEFAULT HEADER" - this is not user generated input, this is your source code!).

Re-using view functionality

We have come now to a crucial point in this chapter, so please stick with me. Let’s say that you have more than one class based views that contain a header attribute. You want to include the default header functionality on all of them so that if any view instantiated from these class based views doesn’t define a header the default string will be output (I know that this may be a rather trivial example but I want to keep everything simple to make following easy - instead of the default header the functionality you want to override may be adding stuff to the context or filtering the objects you’ll retrieve from the database).

To re-use this default header functionality from multiple classes you have two options: Either inherit all classes that need this functionality from DefaultHeaderBetterCustomClassView or extract the custom get_header method to a mixin and inherit from the mixin. A mixin is a class not related to the class based view hierarchy we are using - the mixin inherits from object (or from another mixin) and just defines the methods and attributes that need to be overridden. When the mixin is mixed with the ancestors of a class its functionality will be used by that class (we’ll see how shortly). So the mixin will only define get_header and not all other methods like render, get_context etc. Using the DefaultHeaderBetterCustomClassView is enough for some cases but for the general case of re-using the functionality you’ll need to create the mixin. Let’s see why:

Suppose that you have a base class that renders the header and context as JSON instead of the HTML template, something like this:

class JsonCustomClassView:
    def get_header(self, ):
        return self.header if self.header else ""

    def get_context(self, ):
        return self.context if self.context else []

    def as_view(cls, *args, **kwargs):
        def view(request, ):
            instance = cls(**kwargs)
            return HttpResponse(json.dumps({
                'header': instance.get_header(),
                'context': instance.get_context(),

        return view

Notice that this class does not inherit from our previous hierarchy (i.e does not inherit from BetterCustomClassView) but from object since it provides its own as_view method. How could we re-use default header functionality in this class (without having to re-implement it)? One solution would be to create a class that inherits from both JsonCustomClassView and DefaultHeaderBetterCustomClassView using something like

class DefaultHeaderJsonCustomClassView(DefaultHeaderBetterCustomClassView, JsonCustomClassView):

# OR
class JsonDefaultHeaderCustomClassView(JsonCustomClassView, DefaultHeaderBetterCustomClassView):

What will happen here? Notice that the methods get_header and as_view exist in both ancestor classes! So which one will be used in each case? Actually, there’s a (rather complex) rule for that called MRO (Method Resolution Order). The MRO is also what can used to know which get_header and as_view will be used in each case in the previous example.

Interlude: An MRO primer

What is MRO? For every class that Python sees, it tries to create a list (MRO list) of ancestor classes containing that class as the first element and its ancestors in a specific order I’ll discuss in the next paragraph. When a method of an object of that specific class needs to be called, then the method will be searched in the MRO list (from the first element of the MRO list i.e. starting with the class it self) - when a class is found in the list that defines the method then that method instance (i.e. the method defined in this class) will be called and the search will stop (careful readers: I haven’t yet talked about super so please be patient).

Now, how is the MRO list created? As I explained, the first element is the class itself. The second element is the MRO of the leftmost ancestor of that object (so MRO will run recursively on each ancestor), the third element will be the MRO of the ancestor right next to the leftmost ancestor etc. There is one extra and important rule: When a class is found multiple times in the MRO list (for example if some elements have a common ancestor) then only the last occurrence in the list will be kept - so each class will exist only once in the MRO list. The above rule implies that the rightmost element in every MRO list will always be object - please make sure you understand why before continuing.

Thus, the MRO list for DefaultHeaderJsonCustomClassView defined in the previous section is (remember, start with the class to the left and add the MRO of each of its ancestors starting from the leftmost one): [DefaultHeaderJsonCustomClassView, DefaultHeaderBetterCustomClassView, BetterCustomClassView, CustomClassView, JsonCustomClassView, object], while for JsonDefaultHeaderCustomClassView is [JsonDefaultHeaderCustomClassView, JsonCustomClassView, DefaultHeaderBetterCustomClassView, BetterCustomClassView, CustomClassView, object]. What this means is that for DefaultHeaderJsonCustomClassView the CustomClassView.as_view() and DefaultHeaderBetterCustomClassView.get_header() (thus we will not get the JSON output) and for JsonDefaultHeaderCustomClassView the JsonCustomClassView.as_view() and JsonCustomClassView.get_header() will be used (so we won’t get the default header functionality) - i.e none of those two options will result to the desired behaviour.

Let’s try an example that has the same base class twice in the hierarchy (actually the previous examples also had a class twice in the hierarchy - object but let’s be more explicit). For this, we’ll create a DefaultContextBetterCustomClassView that returns a default context if the context is empty (similar to the default header functionality).

class DefaultContextBetterCustomClassView(BetterCustomClassView, ):
    def get_context(self, ):
        return self.context if self.context else ["DEFAULT CONTEXT"]

Now we’ll create a class that inherits from both DefaultHeaderBetterCustomClassView and DefaultContextBetterCustomClassView:

class DefaultHeaderContextCustomClassView(DefaultHeaderBetterCustomClassView, DefaultContextBetterCustomClassView):

Let’s do the MRO for the DefaultHeaderContextCustomClassView class:

Initially, the MRO will be the following:

Starting with the initial class
1. DefaultHeaderContextCustomClassView
Follows the leftmost class (DefaultHeaderBetterCustomClassView) MRO
2. DefaultHeaderContextCustomClassView, 3. BetterCustomClassView, 4. CustomClassView, 5. object
And finally the next class (DefaultContextBetterCustomClassView) MRO
6. DefaultContextBetterCustomClassView, 7. BetterCustomClassView, 8. CustomClassView, 9. object

Notice that classes BetterCustomClassView, CustomClassView and object are repeated two times (on place 3,4,5 and 7,8,9) thus only their last (rightmost) occurrences will be kept in the list. So the resulting MRO is the following (3,4,5 are removed):

[DefaultHeaderContextCustomClassView, DefaultHeaderBetterCustomClassView, DefaultContextBetterCustomClassView, BetterCustomClassView, CustomClassView, object]

One funny thing here is that the DefaultHeaderContextCustomClassView will actually work properly because the get_header will be found in DefaultHeaderBetterCustomClassView and the get_context will be found in DefaultContextBetterCustomClassView so this result to the correct functionality.

Yes it does work but at what cost? Do you really want to do the mental exercise of finding out the MRO for each class you define to see which method will be actually used? Also, what would happen if the DefaultHeaderContextCustomClassView class also had a get_context method defined (hint: that get_context would be used and the get_context of DefaultContextBetterCustomClassView would be ignored).

Before finishing this interlude, I’d like to make a confession: The Python MRO algorithm is not as simple as than the procedure I described. It uses an algorithm called C3 linearization which seems way too complex to start explaining or understanding if you not a CS student. What you’ll need to remember is that the procedure I described works fine in normal cases when you don’t try to do something stupid. Here’s a post that explains the theory more. However if you follow along my recommendations below you won’t have any problems with MRO, actually you won’t really need to use the MRO that much to understand the method calling hierarchy.

Using mixins for code-reuse

The above explanation of MRO should convince you that you should avoid mixing hierarchies of classes - if you are not convinced then wait until I introduce super() in the next section and I guarantee that you’ll be!

So, that’s why I propose implementing common functionality that needs to be re-used between classes only with mixins (hint: that’s also what Django does). Each re-usable functionality will be implemented in its own mixin; class views that need to implement that functionality will just inherit from the mixin along with the base class view. Each one of the view classes you define should inherit from one and only one other class view and any number of mixins you want. Make sure that the view class is rightmost in the ancestors list and the mixins are to the left of it (so that they will properly override its behaviour; remember that the methods of the ancestors to the left are searched first in the MRO list — and the methods of the defined class have of course the highest priority since it goes first in the MRO list).

Let’s try implementing the proposed mixins for a default header and context:

class DefaultHeaderMixin:
    def get_header(self, ):
        return self.header if self.header else "DEFAULT HEADER"

class DefaultContextMixin:
    def get_context(self, ):
        return self.context if self.context else ["DEFAULT CONTEXT"]

and all the proposed use cases using the base class view and the mixins:

class DefaultHeaderMixinBetterCustomClassView(mixins.DefaultHeaderMixin, BetterCustomClassView):

class DefaultContextMixinBetterCustomClassView(mixins.DefaultContextMixin, BetterCustomClassView):

class DefaultHeaderContextMixinBetterCustomClassView(mixins.DefaultHeaderMixin, mixins.DefaultContextMixin, BetterCustomClassView):

class JsonDefaultHeaderMixinCustomClassView(mixins.DefaultHeaderMixin, JsonCustomClassView):

I believe that the above definitions are self-documented and it is very easy to know which method of the resulting class will be called each time: Start from the main class and if the method is not found there continue from left to right to the ancestor list; since the mixins do only one thing and do it well you’ll know what each class does simply by looking at its definition.

The super situation

The final (and most complex) thing and extension I’d like to discuss for our custom class based views is the case where you want to use the functionality of more than one mixins for the same thing. For example, let’s suppose that we had a mixin that added some data to the context and a different mixing that added some different data to the context. Both would use the get_context method and you’d like to have the context data of both of them to your context. But this is not possible using the implementations above because when a get_context is found in the MRO list it will be called and the MRO search will finish there!

So how could we add the functionality of both these mixins to a class based view? This is the same problem as if we wanted to inherit from a mixin (or a class view) and override one of its methods but also call its parent (overridden) method for example to get its output and use it as the base of the output for the overridden method. Both these situations (re-use functionality of two mixins with the same method or re-use functionality from a parent method you override) are the same because what stays in the end is the MRO list. For example say we we had the following base class

class V:pass

and we wanted to override it either using mixins or by using normal inheritance.

When using mixins for example like this:

class M1:pass
class M2:pass
class MIXIN(M2, M1, V):pass

we’ll have the following MRO:

# MIXIN.mro()
# [MIXIN, M2, M1, V, object, ]

while when using inheritance like this:

class M1V(V):pass
class M2M1V(M1V):pass
class INHERITANCE(M2M1V):pass

we’ll have the following MRO:

# INHERITANCE.mro() # [INHERITANCE, M2M1V, M1V, V, object ]

As we can see in both cases the base class V is the last one (just next to object) and between this class and the one that needs the functionality (MIXIN in the first case and INHERITANCE in the second case) there are the classes that will define the extra functionality that needs to be re-used: M2 and M1 (start from left to right) in the first case and M2M1V and M1V (follow the inheritance hierarchy) in the second case. So in both cases when calling a method they will be searched the same way using the MRO list and when the method is found it will be executed and the search will stop.

But what if we needed to re-use some method from V (or from some other ancestor) and a class on the left of the MRO list has the same method? The answer, as you should have guessed by now if you have some Python knowledge is super().

The super method can be used by a class method to call a method of its ancestors respecting the MRO. Thus, running super().x() from a method instance will try to find method x() on the MRO ancestors of this instance even if the instance defines the “x()“ method i.e it will not search the first element of the MRO list. Notice that if the x() method does not exist in the headless-MRO chain you’ll get an attribute error. So, usually, you’ll can super().x() from inside the x() method to call your parent’s (as specified by the MRO list) same-named method and retrieve its output.

Let’s take a closer look at how super() works using a simple example. For this, we’ll define a method calld x() on all classes of the previous example:

class V:
    def x(self):
        print ("From V")

class M1:
    def x(self):
        print ("From M1")

class M2:
    def x(self):
        print ("From M2")

class MIXIN(M2, M1, V):
    def x(self):
        print ("From MIXIN")

class M1V(V):
    def x(self):
        print ("From M1V")

class M2M1V(M1V):
    def x(self):
        print ("From M2M1V")

    def x(self):
        print ("From INHERITANCE")

print ("MIXIN OUTPUT")


Here’s the output:

From V
From M1
From M2
From V
From M1V
From M2M1V

Notice when each message is printed: Because x() first calls its super() method and then it prints the message in both cases first the From V message is printed from the base class and then from the following classes in the hierarchy (as per the MRO) ending with the class of the instance (either MIXIN or INHERITANCE). Also the print order is the same in both cases as we’ve already explained. Please make sure you understand why the output is like this before continuing.

Using super in our hierarchy

Using super and mixins it is easy to mix and match functionality to create new classes. Of course, super can be used without mixins when overriding a method from a class you inherit from and want to also call your ancestor’s method.

Here’s how we could add a prefix to the header:

class HeaderPrefixMixin:
    def get_header(self, ):
        return "PREFIX: " + super().get_header()

and here’s how it could be used:

class HeaderPrefixBetterCustomClassView(mixins.HeaderPrefixMixin, BetterCustomClassView):

This will retrieve the header from the ancestor and properly print the header displaying both PREFIX and Hello. What if we wanted to re-use the default header mixin? First let’s change DefaultHeaderMixin to properly use super():

class DefaultHeaderSuperMixin:
    def get_header(self, ):
        return super().get_header() if super().get_header() else "DEFAULT HEADER"
class HeaderPrefixDefaultBetterCustomClassView(mixins.HeaderPrefixMixin, mixins.DefaultHeaderSuperMixin, BetterCustomClassView):

Notice the order of the ancestor classes. The get_header() of HeaderPrefixMixin will be called which will call the get_header() of DefaultHeaderSuperMixin (which will call the get_header() of BetterCustomClassView returning None). So the result will be "PREFIX: DEFAULT HEADER". However if instead we had defined this class like

class HeaderPrefixDefaultBetterCustomClassView(mixins.DefaultHeaderSuperMixin, mixins.HeaderPrefixMixin, BetterCustomClassView):

the result would be "PREFIX: " (DEFAULT HEADER won’t be printed). Can you understand why?

One thing to keep in mind is that most probably you’ll need to call super() and return its output when you override a method. Even if you think that you don’t need to call it for this view or mixin, you may need it later from some other view or mixin that inherits from this view. Also notice that super() may not return anything but may have some side-effects in your class (for example set a self attribute) which you won’t get if you don’t call it!

For another example of super, let’s define a couple of mixins that add things to the context:

class ExtraContext1Mixin:
    def get_context(self, ):
        ctx = super().get_context()
        return ctx

class ExtraContext2Mixin:
    def get_context(self, ):
        ctx = super().get_context()
        ctx.insert(0, 'data2')
        return ctx

The first one retrieves the ancestor context list and appends 'data1' to the it while the second one will insert 'data2' to the start of the list. To use these mixins just add them to the ancestor list of your class hierarchy as usually. One interesting thing to notice here is that because of how get_context is defined we’ll get the same output no matter the order of the mixins in the hierarchy since ExtraContext1Mixin will append data1 to the end of the context list and the ExtraContext2Mixin will insert data2 to the start of the context list.

class ExtraContext12BetterCustomClassView(mixins.ExtraContext1Mixin, mixins.ExtraContext2Mixin, BetterCustomClassView):

class ExtraContext21BetterCustomClassView(mixins.ExtraContext2Mixin, mixins.ExtraContext1Mixin, BetterCustomClassView):

If instead both of these mixins appended the item to the end of the list, then the output would be different depending on the ancestor order. Of course, since we’ve already defined HeaderPrefixMixin and DefaultHeaderSuperMixin nothing stops us from using all those mixins together!

class AllTogetherNowBetterCustomClassView(

This will have the desired behaviour of adding a prefix to the header, having a default header if not one was defined and adding the extra context from both mixins!

Testing all this

In the accompanying project at https://github.com/spapas/cbv-tutorial you can take a look at how this custom CBV hierarchy works by running it and taking a look at the core project (visit There you can take a look at the views.py and mixins.py to see all the views and mixins we’ve discussed in this chapter.

A high level overview of CBVs

After the previous rather long (but I hope gentle enough) introduction to implementing our own class based view hierarchy using inheritance, mixins, MRO, method overriding and super we can now start talking about the Django Class Based Views (CBVs). Our guide will be the CBV inspector application which displays all classes and mixins that Django CBVs are using along with their methods and attributes. Using this application and after reading this article you should be able to quickly and definitely know which method or attribute you need to define to each one of your mixins or views.

To use CBV inspector, just click on a class name (for example CreateView); you will immediately see its MRO ancestors, its list of attributes (and the ancestor class that defines each one) and finally a list of methods that this class and all its ancestors define. Of course when a method is defined by multiple classes the MRO ordering will be used - super is used when the functionality of the ancestor classes is also used. The CBV inspector (and our project) has Python 3 syntax. If you want to follow along with Python 2 (I don’t recommend it though since Django 2.0 only supports Python 3.x) use the following syntax to call super for method x():

super(ClassName, self).x()

this is the same as calling


in Python 3.x.

Taking a look at the View

In any case, our travel starts from the central CBV class which is (intuitively) called … View!

This class is used as the base in Django’s CBV hierarchy (similar to how CustomClassView was used in our own hierarchy). It has only one attribute (http_method_names) and a very small number of methods. The most important method is the as_view class method (which is similar to the one we defined in the previous section). The as_view will instantiate an instance object of the View class (actually the class that inherits from View) and use this object to properly generate a functional view.

The View class cannot be used as it is but it must be inherited by a child class. The child class needs to define a method that has the same name as each http method that is supported - for example if only HTTP GET and HTTP POST are supported then the inherited class must define a get and a post method; these methods are called from the functional view through a method called dispatch and need to return a proper response object. So, we have two central methods here: The as_view class method that creates the object instance and returns its view function and dispatch that will call the proper named class method depending on the HTTP method (i.e post, get, put etc). One thing to keep from this discussion is that you shouldn’t ever need to mess with as_view but, because dispatch is the only instance method that is guaranteed to run every time the class based view will run, you will frequently need to override it especially to control access control.

As an example, we can implemented the BetterCustomClassView from the first section using View as its ancestor:

class DjangoBetterCustomClassView(View, ):
    header = ''
    context =''

    def get_header(self, ):
        return self.header if self.header else ""

    def get_context(self , ):
        return self.context if self.context else []

    def render_context(self):
        context = self.get_context()
        if context:
            return '<br />'.join(context)
        return ""

    def get(self, *args, **kwargs):
        resp = """
                header=self.get_header(), body=self.render_context(),
        return HttpResponse(resp)

This method won’t print anything but of course it could use the mixins from before to have some default values:

class DefaultHeaderContextDjangoBetterCustomClassView(DefaultHeaderMixin, DefaultContextMixin, DjangoBetterCustomClassView):

Of course instead of using our mixins and render methods it would be much better to use the proper ones defined by Django - that’s what we’re going to do from now on I just wanted to make clear that there’s nothing special in Django’s CBV hierarchy and can be overridden as we’d like.

RedirectView and TemplateView

Continuing our tour of Django CBVs I’d like to talk a little about the classes that the CBV Inspector puts in the same level as View (GENERIC BASE): RedirectView and TemplateView. Both inherit directly from View and, the first one defines a get method that returns a redirect to another page while the latter one renders and returns a Django template in the get method.

The RedirectView inherits directly from view and has attributes like url (to use a static url) or pattern_name (to use one of the patterns define in your urls.py) to define where it should redirect. These attributes are used by the get_redirect_url which will generate the actual url to redirect to and can be overriden for example to redirect to a different location depending on the current user.

The TemplateView on the other hand inherits from View and two more classes (actually these are mixins) beyond View: TemplateResponseMixin and ContextMixin. If you take a look at them you’ll see that the TemplateResponseMixin defines some template-related attributes (most important being the template_name) and two methods: One that retrieves the template that will be used to render this View (get_template_names) and one that actually renders the template (render_to_response) using a TemplateResponse instance. The ContextMixin provides the get_context_data that is passed to the template to be rendered and should be overridden if you want to pass more context variables.

We can already see many opportunities of reusing and overriding functionality and improving our DRY score, for example: Create a catch all RedirectView that depending on the remainder of the url it will redirect to a different page, create a mixin that appends some things to the context of all CBVs using it, use dynamic templates based on some other condition (that’s actually what Detail/List/UpdateView are doing), render a template to a different output than Html (for example a text file) etc. I’ll try to present examples for these in the next section.

The FormView

The next view we’re going to talk about is FormView. This is a view that can be used whenever we want to display a form (not a form related to a Model i.e for Create/Update/Delete, for these cases there are specific CBVs we’ll see later). It is interesting to take a look at the list of its ancestors: TemplateResponseMixin, BaseFormView, FormMixin, ContextMixin, ProcessFormView and View. We are familiar with TemplateResponseMixin, ContextMixin and View but not with the others. Before discussing these classes let’s take a look at the FormView hierarchy, courtesy of http://ccbv.cco.uk and http://yuml.me:


The above diagram should make everything easier: The FormMixin inherits from ContextMixin and overrides its get_context_data method to add the form to the view. Beyond this, it adds some attributes and methods for proper form handling, for example the form_class (attribute when the form class will be the same always) and get_form_class() (method when the form class will be dynamic for example depending on the logged in user), initial and get_initial() (same logic as before for the form’s initial values), form_valid() and form_invalid() to define what should happen when the form is valid or invalid, get_form_kwargs to pass some keyword arguments to the form’s constructor etc. Notice that FormMixin does not define any form handling logic (i.e check if the form is valid and call its form_valid() method) — this logic is defined in the ProcessFormView which inherits from View and defines proper get() (just render the form) and post() (check if the form is valid and call form_valid else call form_invalid) methods.

One interesting here is to notice here is that Django defines both the FormMixin and ProcessFormView. The FormMixin offers the basic Form elements (the form class, initial data etc) and could be re-used in a different flow beyond the one offered by ProcessFormView (for example display the form as a JSON object instead of a Django template). On the other hand, ProcessFormView is required in order to define the get and post methods that are needed from the View. These methods can’t be overridden in the FormMixin since that would mean that the mixin would behave as a view!

Finally, the BaseFormView class is used to inherit from ProcessFormView and FormMixin. It does not do anything more than providing a base class that other classes that want to use the form functionality (i.e both the ProcessFormView and FormMixin) will inherit from.

The ListView and DetailView

Next in our Django CBV tour is the ListView. The ListView is used to render multiple objects in a template, for example in a list or table. Here’s a diagram of the class hierarchy (courtesy of http://ccbv.cco.uk and http://yuml.me):


The MultipleObjectMixin is used make a query to the database (either using a model or a queryset) and pass the results to the context. It also supports custom ordering (get_ordering()) and pagination (paginate_queryset()). However, the most important method of this mixin is get_queryset(). This method checks to see if the queryset or model attribute are defined (queryset will be checked first so it has priority if both are defined) and returns a queryset result (taking into account the ordering). This queryset result will be used by the get_context_data() method of this mixin to actually put it to the context by saving to a context variable named object_list. Notice that you can set the context_object_name attribute to add and extra another variable to the context with the queryset beyond object_list (for example if you have an ArticleLsitView you can set context_object_name = articles to be able to do {% for article in articles %} in your context instead of {% for article in object_list %}).

The MultipleObjectMixin can be used and overridden when we need to put multiple objects in a View. This mixin is inherited (along with View) from BaseListView that adds a proper get method to call get_context_data and pass the result to the template.

As we can also see, Django uses the MultipleObjectTemplateResponseMixin that inherits from TemplateResponseMixin to render the template. This mixin does some magic with the queryset or model to define a template name (so you won’t need to define it yourself) - that’s from where the app_label/app_model_list.html default template name is created.

Similar to the ListView is the DetailView which has the same class hierarchy as the ListView with two differences: It uses SingleObjectMixin instead of MultipleOjbectMixin, SingleObjectTemplateResponseMixin instead of MultipleObjectTemplateResponseMixin and BaseDetailView instead of BaseListView. The SingleObjectMixin will use the get_queryset() (in a similar manner to the get_queryset() of MultipleObjectMixin) method to return a single object (so all attributes and methods concerning ordering or pagination are missing) but instead has the get_object() method which will pick and return a single object from that queryset (using a pk or slug parameter). This object will be put to the context of this view by the get_context_data. The BaseDetailView just defines a proper get to call the get_context_data (of SingleObjectMixin) and finally the SingleObjectTemplateResponseMixin will automatically generate the template name (i.e generate app_label/app_model_detail.html).

The CreateView

The next Django CBV we’ll talk about is CreateView. This class is used to create a new instance of a model. It has a rather complex hierarchy diagram but we’ve already discussed most of these classes:

As we can see the CreateView inherits from BaseCreateView and SingleObjectTemplateResponseMixin. The SingleObjectTemplateResponseMixin is mainly used to define the template names that will be searched for (i.e app_label/app_model_form.html), while the BaseCreateView is used to combine the functionality of ProcessFormView (that handles the basic form workflow as we have already discussed) and ModelFormMixin. The ModelFormMixin is a rather complex mixin that inherits from both SingleObjectMixin and FormMixin. The SingleObjectMixin functionality is not really used by CreateView (since no object will need to be retrieved for the CreateView) however the ModelFormMixin is also used by UpdateView that’s why ModelFormMixin also inherits from it (to retrieve the object that will be updated).

ModelFormMixin mixin adds functionality for handling forms related to models and object instances. More specifically it adds functionality for: * creating a form class (if one is not provided) by the configured model / queryset. If you don’t provide the form class (by using the form_class attribute) then you need to configure the fields that the generated form will display by passing an array of field names through the fields attribute * overrides the form_valid in order to save the object instance of the form * fixes get_success_url to redirect to the saved object’s absolute_url when the object is saved * pass the current object to be updated (that was retrieving through the SingleObjectMixin) -if there is a current object- to the form as the instance attribute

The UpdateView and DeleteView

The UpdateView class is almost identical to the CreateView - the only difference is that UpdateView inherits from BaseUpdateView (and SingleObjectTemplateResponseMixin) instead of BaseCreateView. The BaseUpdateView overrides the get and post methods of ProcessFormView to retrieve the object (using SingleObjectMixin‘s get_object()) and assign it to an instance variable - this will then be picked up by the ModelFormMixin and used properly in the form as explained before. One thing I notice here is that it seems that the hierarchy would be better if the ModelFormMixin inherited only from FormMixin (instead of both from FormMixin and SingleObjectMixin) and BaseUpdateView inheriting from ProcessFormView, ModelForMixin and SingleObjectMixin. This way the BaseCreateView wouldn’t get the non-needed SingleObjectMixin functionality. I am not sure why Django is implemented this way (i.e the ModelFormMixin also inheriting from SingleObjectMixin thus passing this non-needed functionality to BaseCreateView) — if a reader has a clue I’d like to know it.

In any way, I’d like to also present the DeleteView which is more or less the same as the DetailView with the addition of the DeleteMixin in the mix. The DeleteMixin adds a post() method that will delete the object when called and makes success_url required (since there would be no object to redirect to after this view is posted).

Access control mixins

Another small hierarchy of class based views (actually these are all mixins) are the authentication ones which can be used to control access to a view. These are AcessMixin, LoginRequiredMixin, PermissionRequiredMixin and UserPassesTestMixin. The AccessMixin provides some basic functionality (i.e what to do when the user does not have access to the view, find out the login url to redirect him etc) and is used as a base for the other three. These three override the dispatch() method of View to check if the user has the specific rights (i.e if he has logged in for LoginRequiredMixin, if he has the defined permissions for PermissionRequiredMixin or if he passes the provided test in UserPassesTextMixin). If the user has the rights the view will proceed as normally (call super’s dispatch) else the access denied functionality from AccessMixin will be implemented.

Some other CBVs

Beyond the class based views I discussed in this section, Django also has a bunch of CBVs related to account views (LoginView, LogoutView, PasswordChangeView etc) and Dates (DateDetailView, YearArchiveView etc). I won’t go into detail about these since they follow the same concepts and use most of the mixins we’ve discussed before. Using the CBV Inspector you should be able to follow along and decide the methods you need to override for your needs.

Also, most well written Django packages will define their own CBVs that inherit from the Django CBVs - with the knowledge you acquired here you will be able to follow along on their source code to understand how everything works.

Real world use cases

In this section I am going to present a number of use cases demonstrating the usefulness of Django CBVs. In most of these examples I am going to override one of the methods of the mixins I discussed in the previous section. There are two methods you can use for integrating the following use cases to your application.

Create your own class inheriting from one of the Django CBVs and add to it directly the method to override. For example, if you wanted to override the get_queryset() method a ListView you would do a:

class GetQuerysetOverrideListView(ListView):
    def get_queryset(self):
        qs = super().get_queryset()
        return qs.filter(status='PUBLISHED')

This is useful if you know that you aren’t going to need the overriden get_queryset functionality to a different method and following the YAGNI principle (or if you know that even if you need it you could inherit from GetQuerysetOverrideListView i.e in another ListView). However, if you know that there may be more CBVs that would need their queryset filtered by status='PUBLISHED' then you should add a mixin that would be used by your CBVs:

class GetQuerysetOverrideMixin:
    def get_queryset(self):
        qs = super().get_queryset()
        return qs.filter(status='PUBLISHED')

class GetQuerysetOverrideListView(GetQuerysetOverrideMixin, ListView):

Now, one thing that needs some discussion here is that the method get_queryset is provided by a mixin (in fact it is provided by two mixins: MultipleObjectMixin for ListView and SingleObjectMixin for DetailView, UpdateView and DeleteView). Because of how MRO works, I won’t need to inherit GetQuerysetOverrideMixin from MultipleObjectMixin (or SingleObjectMixin but let’s ignore that for now) but I can just inherit from object and make sure that, as already discussed, put the mixin before (to the left) of the CBV. Notice that even if I had defined GetQuerysetOverrideMixin as GetQuerysetOverrideMixin(MultipleObjectMixin) the MultipleObjectMixin class would be found twice in the MRO list so only the rightmost instance would remain. So the MRO for both GetQuerysetOverrideMixin(object, ) and GetQuerysetOverrideMixin(MultipleObjectMixin) would be the same! Also, inheriting directly from object makes our GetQuerysetOverrideMixin more DRY since if it inherited from MultipleObjectMixin we’d need to create another version of it that would inherit from SingleObjectMixin; this is because get_queryset exists in both these mixins.

For some of the following use cases I am also going to use the following models for user generated content (articles and uploaded files):

    ('DRAFT', 'Draft', ),
    ('PUBLISHED', 'Published', ),
    ('REMOVED', 'Removed', ),

class Category(models.Model):
    name = models.CharField(max_length=128, )

    def __str__(self):
        return self.name

    class Meta:
        permissions = (
            ("publisher_access", "Publisher Access"),
            ("admin_access", "Admin Access"),

class AbstractGeneralInfo(models.Model):
    status = models.CharField(max_length=16, choices=STATUS_CHOICES, )
    category = models.ForeignKey('category', on_delete=models.PROTECT, )
    created_on = models.DateTimeField(auto_now_add=True, )
    created_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, related_name='%(class)s_created_by', )
    modified_on = models.DateTimeField(auto_now=True, )
    modified_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, related_name='%(class)s_modified_by', )

    owned_by = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.PROTECT, related_name='%(class)s_owned_by', )
    published_on = models.DateTimeField(blank=True, null=True)

    class Meta:
        abstract = True

class Article(AbstractGeneralInfo):
    title = models.CharField(max_length=128, )
    content = models.TextField()

class Document(AbstractGeneralInfo):
    description = models.CharField(max_length=128, )
    file = models.FileField()

All this can be found on the accompanying project https://github.com/spapas/cbv-tutorial on the djangocbv app (visit

Do something when a valid form is submitted

When a form is submitted and the form is valid the form_valid method of ModelForMixin (and FormMixin) will be called. This method can be overridden to do various things before (or after) the form is saved. For example, you may want have a field whose value is calculated from other fields in the form or you want to create an extra object. Let’s see a generic example of overriding a CreateView or UpdateView with comments:

def form_valid(self, form, ):

# let’s calculate a field value form.instance.calculated_field = form.cleaned_data[‘data1’] + form.cleaned_data[‘data2’]

# save the form by calling super().form_valid(); keep the return value - it is the value of get_success_url redirect_to = super().form_valid(form)

# For Create or UpdateView, the just-saved object will be assigned to self.object logger.log(“Created an object with id {0}”.format(self.object.id)

# return the redirect return redirect_to

This is rather complex so I’ll also explain it: The form_valid gets the actual form which, since is validated has a cleaned_data dictionary of values. This form also has an instance attribute which is the object that this form is bound to - notice that a normal Form won’t have an instance only a ModelForm. This can be used to modify the instance of this form as needed - before saving it. When you want to actually save the instance you call super().form_valid() passing it the modified form (and instance). This method does three things

  • It saves the instance to the database
  • It assigns the saved object to the object instance attribute (so you can refer to it by self.instance)
  • It uses get_redirect_url to retrieve the location where you should redirect after the form is submitted

Thus in this example we save redirect_to to return it also from our method also and then can use self.object.id to log the id of the current object.

On a more specific example, notice the Article and Document models which both inherit (abstract) from AbstractGeneralInfo have a created_by and a modified_by field. These fields have to be filled automatically from the current logged in user. Now, there are various options to do that but what I vote for is using an AuditableMixin as I have already described in my Django model auditing article.

To replicate the functionality we’ll create an AuditableMixin like this:

class AuditableMixin(object,):
    def form_valid(self, form, ):
        if not form.instance.created_by:
            form.instance.created_by = self.request.user
        form.instance.modified_by = self.request.user
        return super().form_valid(form)

This mixin can be used by both the create and update view of both Article and Document. So all four of these classes will share the same functionality. Notice that the form_valid method is overridden - the created_by of the form’s instance (which is the object that was edited, remember how ModelFormMixin works) will by set to the current user if it is null (so it will be only set once) while the modified_by will be set always to the current user. Finally we call super().form_valid and return its response so that the form will be actually saved and the redirect will go to the proper success url. To use it for example for the Article, CreateView should be defined like this:

class ArticleCreateView(AuditableMixin, CreateView):
    class Meta:
        model = Article

Change the queryset of the CBV

All CBVs that inherit from SingleObjectMixin or MultipleObjectMixin (ListView, DetailView, UpdateView and DeleteView) have a model and a queryset property that can be used (either one or the other) to define the queryset that will be used for querying the database for that CBVs results. This queryset can be further dynamically refined by overriding the get_queryset() method. What I usually do is that I define the model attribute and then override get_querset in order to dynamically modify the queryset.

For example, let’s say that I wanted to add a count of articles and documents per each category. Here’s how the CategoryListView could be done:

class CategoryListView(ExportCsvMixin, AdminOrPublisherPermissionRequiredMixin, ListView):
    model = Category
    context_object_name = 'categories'

    def get_queryset(self):
        qs = super().get_queryset()
        return qs.annotate(article_cnt=Count('article'), document_cnt=Count('document'))

Notice that I also use some more mixins for this ListView (they’ll be explained later). The get_queryset adds the annotation to the super() queryset (which will be Category.objects.all()). One final comment is that instead of this, I could have more or less the same functionality by implementing CategoryListView:

class CategoryListView(ExportCsvMixin, AdminOrPublisherPermissionRequiredMixin, ListView):
    context_object_name = 'categories'
    query = Category.objects.all().annotate(article_cnt=Count('article'), document_cnt=Count('document'))

This has the same functionality (return all categories with the number of articles and documents for each one) and saves some typing from overriding the get_queryset method. However as I said most of the time I use the model attribute and override the get_queryset method because it seems more explicit and descriptive to me and most of the time I’ll need to add some more filtering (based on the current user, based on some query parameter etc) that can only be implemented on the get_queryset.

Allow each user to list/view/edit/delete only his own items

Continuing from the previous example of modifying the queryset, let’s suppose that we want to allow each user to be able to list the items (articles and documents) he has created and view/edit/delete them. We also want to allow admins and publishers to view/edit everything.

Since the Article and Document models both have an owned_by element we can use use this to filter the results returned by get_queryset(). For example, here’s a mixin that checks if the current user is admin or publisher. If he is a publisher then he will just return the super() queryset. If however he is a simple user it will return only the results that are owned by him with qs.filter(owned_by=self.request.user).

class LimitAccessMixin:
    def get_queryset(self):
        qs = super().get_queryset()
        if self.request.user.has_perm('djangocbv.admin_access') or self.request.user.has_perm('djangocbv.publisher_access') :
            return qs
        return qs.filter(owned_by=self.request.user)

Another similar mixin that is used is the HideRemovedMixin that, for simple users, excludes from the queryset the objects that are removed:

class HideRemovedMixin:
    def get_queryset(self):
        qs = super().get_queryset()
        if self.request.user.has_perm('djangocbv.admin_access') or self.request.user.has_perm('djangocbv.publisher_access'):
            return qs
        return qs.exclude(status='REMOVED')

One thing that needs a little discussion is that for both of these mixins I am using get_queryset to implement access control to allow using the same mixin for views that inherit from both SingleObjectMixin and MultipleObjectMixin (since the get_queryset is used in both of them). This means that when a user tries to access an object that has not access to he’ll get a nice 404 error.

Beyond this, instead of filtering the queryset, for views inheriting from SingleObjectMixin (i.e DetailView, UpdateView and DeleteView) we could have overridden the get_object method to raise an access denied exception. Here’s how get_object could be overridden to raise a 403 Forbidden status when a user tries to access an object that does not belong to him:

from django.core.exceptions import PermissionDenied

def get_object(self, queryset=None):
    obj = super().get_object()
    if obj.owned_by=self.request.user:
        raise PermissionDenied
    return obj

Configure the form’s initial values from GET parameters

Sometimes we want to have a CreateView with some fields already filled. I usually implement this by passing the proper parameters to the URL (i.e by calling it as /create_view?category_id=2) and then using the following mixin to override the FormMixin get_initial method in order to return the form’s initial data from it:

class SetInitialMixin(object,):
    def get_initial(self):
        initial = super(SetInitialMixin, self).get_initial()
        return initial

So if the /article_create url can be used to initialte the CreateView for the article, using /article_create?category_id=3 will show the CreateView with the Category with id=3 pre-selected in the category field!

Pass extra kwargs to the FormView form

This is a very common requirement. The form may need to be modified by an external condition, for example the current user or something that can be calculated from the view. Here’s a sample mixin that passes the current request (which also includes the user) to the form:

class RequestArgMixin:
    def get_form_kwargs(self):
        kwargs = super(RequestArgMixin, self).get_form_kwargs()
        kwargs.update({'request': self.request})
        return kwargs

Please notice that the form has to properly handle the extra kwarg in its constructor, before calling the super’s constructor. For example, here’s how a form that can accept the request could be implemented:

class RequestForm(forms.Form):
    def __init__(self, *args, **kwargs):
        self.request = kwargs.pop('request', None)
        super().__init__(*args, **kwargs)

We use pop to remove the request from the received kwargs and only then we call the parent constructor.

Add values to the context

To add values to the context of a CBV we override the get_context_data() method. Here’s a mixin that adds a list of categories to all CBVs using it:

class CategoriesContextMixin:
    def get_context_data(self, **kwargs):
        ctx = super().get_context_data(**kwargs)
        ctx['categories'] = Category.objects.all()
        return ctx

Notice that the mixin calls super to get the context data of its ancestors and appends to it. This mean that if we also had a mixin that f.e added the current logged in user to the context (this isn’t really needed since there’s a context processor for this but anyway) then when a CBV inherited from both of them then the data of both of them would be added to the context.

As a general comment there are three other methods the same functionality could be achieved:

  • Just override the get_context_data of the CBV you want to add extra data to its context
  • Add a template tag that will bring the needed data to the template
  • Use a context processor to bring the data to all templates

As can be understood, each of the above methods has certain advantages and disadvantages. For example, if the extra data will query the database then the context processor method will add one extra query for all page loads (even if the data is not needed). On the other hand, the template tag will query the database only on specific views but it makes debugging and reasoning about your template more difficult since if you have a lot of template tags you’ll have various context variables appearing from thing air!

One final comment is that overriding the get_context_data method will probably be the most common thing you’re going to do when using CBVs (you’ll definitely need to add things to the context) so try to remember the following 3 needed lines:

def get_context_data(self, **kwargs):
    ctx = super().get_context_data(**kwargs)
    # ... here we add stuff to the ctx
    return ctx

Add a simple filter to a ListView

For filtering I recommend using the excellent django-filter package as I’ve already presented in my essential Django package list. Here’s how a mixin can be created that adds a filter to the context:

class AddFilterMixin:
    filter_class = None

    def get_context_data(self, **kwargs):
        ctx = super().get_context_data(**kwargs)
        if not self.filter_class:
            raise NotImplementedError("Please define filter_class when using AddFilterMixin")
        filter = self.filter_class(self.request.GET, queryset=self.get_queryset())
        ctx['filter'] = filter
        if self.context_object_name:
            ctx[self.context_object_name] = filter.qs
        return ctx

Notice that the get_context_data checks to see if the filter_class attribute has been defined (if not it will raise a useful explanation). It will then instantiate the filter class passing it the self.request.GET and the current queryset (self.get_queryset()) - so for example any extra filtering you are doing to the queryset (for example only show content owned by the current user) will be also used. Finally, pass the filter to the context and assign the contect_object_name to the filtered queryset.

Here’s for example how this mixin is used for ArticleListView:

class ArticleListView(AddFilterMixin, ListView):
    model = Article
    context_object_name = 'articles'
    filter_class = ArticleFilter

And then just add the following to the article_list.html template:

<form method='GET'>
    {{ filter.form }}
    <input type='submit' value='Filter' />
{% for article in articles %}
    Display article info - only filtered articles will be here
{% endfor %}

Support for success messages

Django has a very useful messages framework which can be used to add flash messages to a view. A flash message is a message that persists in the sesion until it is viewed by the user. So, for example when a user edits an object and saves it, he’ll be redirected to the success page - if you have configured a flash message to inform the user that the save was ok then he’ll see this message once and then if he reloads the page it will be removed.

Here’s a mixin that can be used to support flash messages using Django’s message framework:

class SuccessMessageMixin:
    success_message = ''

    def get_success_message(self):
        return self.success_message

    def form_valid(self, form):
        messages.success(self.request, self.get_success_message())
        return super().form_valid(form)

This mixin overrides the form_valid and adds the message using get_success_message - this can be overriden if you want to have a dynamic message or just set the success_message attribute for a static message, for example something like this:

class SuccesMessageArticleCreateView(SuccessMessageMixin, CreateView):
    success_message = 'Object was created!'

    class Meta:
        model = Article

I’d like to once again point out here that since the super().form_valid(form) method is properly used then if a CBV uses multiple mixins that override form_valid (for example if your CBV overrides both SuccessMessageMixin and AuditableMixin then the form_valid of both will be called so you’ll get both the created_by/modified_by values set to the current user and the success message!

Notice that Django actually provides an implementation of a message mixin which can be used instead of the proposed implementation here (I didn’t know it until recently that’s why I am using this to some projects and I also present it here).

Implement a quick moderation

It is easy to implement some moderation to our model publishing. For example, let’s suppose that we only allow publishers to publish a model. Here’s how it can be done:

def form_valid(self, form):
    if form.instance.status != 'REMOVED':
        if self.request.user.has_perm('djangocbv.publisher_access'):
            form.instance.status = 'PUBLISHED'
            form.instance.status = 'DRAFT'

    return super().form_valid(form)

So, first of all we make sure that the object is not REMOVED (if it is remove it we don’t do anything else). Next we check if the current user has publisher_access if yes we change the object’s status to PUBLISHED - on any other case we change its status to DRAFT. Notice that this means that whenever a publisher saves the object it will be published and whenever a non-publisher saves it it will be made a draft. We then call our ancestor’s form_valid to save the object and return to success url.

I’d like to repeat here that this mixin, since it calls super, can work concurrently with any other mixins that override form_valid (and also call their super method of course), for example it can be used together with the audit (auto-fill created_by and moderated_by) and the success mixin we defined previously!

Allow access to a view if a user has one out of a group of permissions

For this we’ll need to use the authentication mixins functionality. We could implement this by overriding PermissionRequiredMixin or by overriding UserPassesTestMixin.

Using PermissionRequiredMixin is not very easy because the way it works it will allow access if the user has all permissions from the group (not only one as is the requirement). Of course you could override its has_permission method to change the way it checks if the user has the permissions (i.e make sure it has one permission instead of all):

class AnyPermissionRequiredMixin(PermissionRequiredMixin, ):
    def has_permission(self):
        perms = self.get_permission_required()
        return any(self.request.user.has_perm(perm) for perm in perms)

Also we could implement our mixin using UserPassesTestMixin as its base:

class AnyPermissionRequiredAlternativeMixin(UserPassesTestMixin):
    permissions = []

    def test_func(self):
        return any(self.request.user.has_perm(perm) for perm in self.permissions)

The functionality is very simple: If the user has one of the list of the configured permissions then the test will pass (so he’ll have access to the view). If instead the user has none of the permissions then he won’t be able to access the view.

Notice that for the above implementations we inherited from PermissionRequiredMixin or UserPassesTextMixin to keep their functionality - if we had inherited these mixins from object then we’d need to inherit our CBVs from both AnyPermissionRequiredMixin and PermissionRequiredMixin or AnyPermissionRequiredAlternativeMixin and UserPassesTestMixin (with the correct MRO order of course).

Now, the whole permission cheking functionality can be even more DRY. Let’s suppose that we know that there are a couple of views which should only be visible to users having either the app.admin or app.curator permission. Instead of inheriting all these views from AnyPermissionRequiredMixin and configuring the permissions list to each one, the DRY way to implement this is to add yet another mixin from which the CBVs will actually inhert:

class AdminOrPublisherPermissionRequiredMixin(AnyPermissionRequiredMixin):
    permissions = ['djangocbv.admin_access', 'djangocbv.publisher_access']

Disable a view based on some condition

There are times you want to disable a view based on an arbitrary condition - for example example make the view disabled before a specific date. Here’s a simple mixin that overrides dispatch to do this:

class DisabledDateMixin(object, ):
    the_date = datetime.date(2018,1,1)

    def dispatch(self, request, *args, **kwargs):
        if datetime.date.today() < the_date:
            raise PermissionDenied
        return super().dispatch(request, *args, **kwargs)

You can even disable a view completely in case you want to keep it in your urls.py using this mixin:

class DisabledDateMixin(object, ):
    def dispatch(self, request, *args, **kwargs):
        raise PermissionDenied

Output non-html views

I’ve written a whole article about this, please take a look at my Django non-HTML responses article.

Also, notice that is very easy to create a mixin that will output a view to PDF - I have already written an essential guide for outputting PDFs in Django so I am just going to refer you to this article for (much more) information!

Finally, let’s take a look at a generic Mixin that you can use to add CSV exporting capabilities to a ListView:

class ExportCsvMixin:
    def render_to_response(self, context, **response_kwargs):
        if self.request.GET.get('csv'):
            response = HttpResponse(content_type='text/csv')
            response['Content-Disposition'] = 'attachment; filename="export.csv"'

            writer = csv.writer(response)
            for idx, o in enumerate(context['object_list']):
                if idx == 0: # Write headers
                    writer.writerow(k for (k,v) in o.__dict__.items() if not k.startswith('_'))
                writer.writerow(v for (k,v) in o.__dict__.items() if not k.startswith('_'))

            return response
        return super().render_to_response(context, **response_kwargs)

As you can see this mixin overrides the render_to_response method. It will check if there’s a csv key to the GET queryset dictionary, thus the url must be called with ?csv=true or something similar. You can just add this link to your template:

<a class='button' href='?csv=true'>Export csv</a>

So if the view needs to be exported to CSV, it will create a new HttpResponse object with the correct content type. The next line will add a header that (Content-Disposition) will mark the response as an attachment and give it a default file name. We then crate a new csv.writer passing the just-created response as the place to write the csv. The for loop that follows enumerates the object_list value of the context (remember that this is added by the MultipleObjectMixin and contains the result of the ListView). It will then use the object’s __dict__ attribute to write the headers (for the first time) and then write the values of all objects.

As another simple example, let’s create a quick JSON output mixin for our DetailViews:

class JsonDetailMixin:
    def render_to_response(self, context, **response_kwargs):
        if self.request.GET.get('json'):
            response = HttpResponse(content_type='application/json')
            response.write(json.dumps(dict( (k,str(v)) for k,v in self.object.__dict__.items() )))
            return response
        return super().render_to_response(context, **response_kwargs)

If you add this to a view inheriting from DetailView and pass it the ?json=true query parameter you’ll get a JSON response!

Use one TemplateView for multiple html templates

Using a TemplateView you could display an html template without much problem just by settings the template attribute of your class. What if you wanted to have a single TemplateView that would display many templates based on the query path? Simple, just override get_template_names to return a different template based on the path. For example, using this view:

class DynamicTemplateView(TemplateView):
    def get_template_names(self):
        what = self.kwargs['what']
        return '{0}.html'.format(what)

You can render any template you have depending on the value of the what kwarg. To allow only specific template names you can either add a check to the above implementation (i.e that what is help or about) or you may do it to the urls.py if you use a regular expression. Thus, to only allow help.html and about.html to be rendered with this method add it to your urls like this:

re_path(r'^show/(?P<what>help|about)/', views.DynamicTemplateView.as_view(), name='template-show'),

Finally, to use it to render the help.html you’ll just call it like <a href=’{% url “template-show” “help” %}’>Help</a>

Notice that of course instead of creating the DynamicTemplateView you could just dump these html files in your static folder and return them using the static files functionality. However the extra thing that the DynamicTemplateView brings to you is that this is a full Django template thus you can use template tags, filters, your context variables, inherit from your site-base and even override get_context_data to add extra info to the template! All this is not possible with static files!

Implement a partial Ajax view

Overriding get_template_names can also be used to create a DRY Ajax view of your data! For example, let’s say that you have a DetailView for one of your models that has overriden the get_template_names like this:

def get_template_names(self):
    if self.request.is_ajax() or self.request.GET.get('ajax_partial'):
        return 'core/partial/data_ajax.html'
    return super().get_template_names()

and you have also defined a normal template for classic request response viewing and an ajax template that contains only the specific data for this instances (i.e it does not containg html body, headers, footers etc, only a <div> with the instance’s data). Notice I’m using either the is_ajax method or I directly passed GET value (ajax_partial) - this is needed because sometimes is_ajax is not working as expected (depending on how you’re going to do the request), also this way you can easily test the partial ajax view through your browser by passing it ?ajax_partia=true.

Using this technique you can create an Ajax view of your data just by requesting the DetailView through an Ajax call and dumping the response you get to a modal dialog (for example) - no need for fancy REST APIs. Also as a bonus, the classic DetailView will work normally, so you can have the Ajax view to give a summary of the instance’s data (i.e have a subset of the info on the Ajax template) and the normal view to display everything.

Add a dynamic filter and/or table to the context

If you have a lot of similar models you can add a mixin that dynamically creates tables and a filters for these models - take a look at my dynamic tables and filters for similar models article!

Configure forms for your views

As I’ve already explained if you are using a FormView you’ll need to set a form_class for your view (needed by FormMixin) while, for an Update or CreateView which use the ModelFormMixin you can either set the form-class or directly configure the instance’s fields that will be displayed to the form using the fields attribute.

For example, let’s say that you have a rather generic FormView that will display a different form depending on the user permissions. Here’s how you could do this to return a SuperForm if the current user is a superuser and a SimpleForm in other cases:

def get_form_class(self):
    if self.request.user.is_superuser:
        return SuperForm
    return SimpleForm

Display a different form for Create and Update

There are various ways you can do this (for example you can just declare a different form_class for your Create and UpdateView) but I think that the most DRY one, especially if the create and update form are similar is to pass an is_create argument to the form which it will then be used to properly configure the form.

Thus, on your CreateView you’ll add this get_form_kwargs:

def get_form_kwargs(self):
    kwargs = super(MyCreateView, self).get_form_kwargs()
    kwargs.update({'is_create': True})
    return kwargs

while on your UpdateView you’ll add this:

def get_form_kwargs(self):
    kwargs = super(MyUpdateView, self).get_form_kwargs()
    kwargs.update({'is_create': False})
    return kwargs

Please notice that the form has to properly handle the extra kwarg in its constructor as I’ve already explained previously.

Only allow specific HTTP methods for a view

Let’s say that you want to create an UnpublishView i.e a view that will change the status of your content to DRAFT. Since this view will change your model instance it must be called through POST, however you may not want to display an individual form for this view, just a button that when called will display a client-side (Javascript) prompt and if the user clicks it it will immediately do a POST request by submitting the form. The best way to create this is to just implement an UpdateView for your model and change its form valid to change the status to DRAFT, something like this:

def form_valid(self, form, ):
    form.instance.status = 'DRAFT'
    return super().form_valid(form)

Beyond this, you’ll need to add a fields = [] attribute to your UpdateView to denote that you won’t need to update any fields from the model (since you’ll update the status directly) and finally, to only allow this view to be called through an http POST method add the following attribute:

http_method_names = ['post',]

Create an umbrella View for multiple models

Let’s say that you have a couple of models (called Type1 and Type2 that are more or less the same and you want to quickly create a ListView for both of them but you’d like to create just one ListView and separate them by their url. Here’s how it could be done:

class UmbrellaListView(ListView):

    def dispatch(self, request, *args, **kwargs):
        self.kind = kwargs['kind']
        if self.kind == 'type1':
            self.queryset = models.Type1.objects.all()
        elif self.kind == 'type2':
            self.queryset = models.Type2.objects.all()
        return super(UmbrellaListView, self).dispatch(request, *args, **kwargs)

Notice that for this to work properly you must setup your urls like this:

url(r'^list/(?P<kind>type1|type2)/$', UmbrellaListView.as_view() ) , name='umbrella_list' ),

A heavy CBV user project

In this small chapter I’d like to present a bunch of mixins and views that I’ve defined to the accompanying project (https://github.com/spapas/cbv-tutorial).

Let’s start with the mixins (I won’t show the mixins I’ve already talked about in the previous chapter):

class SetOwnerIfNeeded:
    def form_valid(self, form, ):
        if not form.instance.owned_by_id:
            form.instance.owned_by = self.request.user
        return super().form_valid(form)

class ChangeStatusMixin:
    new_status = None

    def form_valid(self, form, ):
        if not self.new_status:
            raise NotImplementedError("Please define new_status when using ChangeStatusMixin")
        form.instance.status = new_status
        return super().form_valid(form)

class ContentCreateMixin(SuccessMessageMixin,
    success_message = 'Object successfully created!'

class ContentUpdateMixin(SuccessMessageMixin,
    success_message = 'Object successfully updated!'

class ContentListMixin(ExportCsvMixin, AddFilterMixin, HideRemovedMixin, ):

class ContentRemoveMixin(SuccessMessageMixin
    http_method_names = ['post',]
    new_status = 'REMOVED'
    fields = []
    success_message = 'Object successfully removed!'

class ContentUnpublishMixin(SuccessMessageMixin
    http_method_names = ['post',]
    new_status = 'DRAFT'
    fields = []
    success_message = 'Object successfully unpublished!'

The SetOwnerIfNeeded and ChangeStatusMixin are simple mixins that override form_valid to introduce some functionality before saving the object).

The mixins that follow are used to group functionality of other mixins together and will be inherited by the views. Thus, ContentCreateMixin has the mixin functionality needed to create something (for example an Article or a Document) i.e show a success message, add auditing information, set the object’s owner, pass the request to the form, set the form’s initial values, do some moderation and only allow logged in users. On a similar fashion, the ContentUpdateMixin collects the functionality needed to update something and is similar to ContentCreateMixin (with the difference that it also as the LimitAccessMixin to only allow simple users to edit their own content). The ContentListMixin adds functionality for export to CSV, simple filter and hiding removed things.

Finally, the ContentRemoveMixin and ContentUnpublishMixin are used to implement Views for removing and unpublishing an object. Both of them inherit from ChangeStatusMixin - one setting the new_status to REMOVED the other to DRAFT.

Notice that they share much functionality so I could remove both ContentRemoveMixin and ContentUnpublishMixin and add a single ContentChangeStatusMixin like this:

class ContentChangeStatusMixin(AdminOrPublisherPermissionRequiredMixin,
    http_method_names = ['post',]
    fields = []

Thus the new_status attribute wouldn’t be there so the views inheriting from this ContentChangeStatusMixin (i.e *RemoveView and *UnpublishView) would need to define the new_status field themselves. This is definitely valid (and more DRY) but less explicit than the way I’ve implemented this - i.e you may wanted to not allow publishers to remove objects, only admins (so you could implement that differently in the get_queryset or dispatch method of ContentRemoveMixin and ContentUpdateMixin) this is easier if you have both the ContentRemoveMixin and ContentUnpublishMixin.

Now let’s take a look at the views:

class CategoryListView(ExportCsvMixin, AdminOrPublisherPermissionRequiredMixin, ListView):
    model = Category
    context_object_name = 'categories'

    def get_queryset(self):
        qs = super().get_queryset()
        return qs.annotate(article_cnt=Count('article'), document_cnt=Count('document'))

class CategoryCreateView(SuccessMessageMixin, AdminOrPublisherPermissionRequiredMixin, CreateView):
    model = Category
    fields = ['name']
    success_message = 'Category created!'
    success_url = reverse_lazy('category-list')

class CategoryUpdateView(SuccessMessageMixin, AdminOrPublisherPermissionRequiredMixin, UpdateView):
    model = Category
    fields = ['name']
    success_message = 'Category updated!'
    success_url = reverse_lazy('category-list')

class CategoryDetailView(CategoriesContextMixin, DetailView):
    model = Category
    context_object_name = 'category'

    def get_context_data(self, **kwargs):
        ctx = super().get_context_data(**kwargs)
        ctx['article_number'] = Article.objects.filter(category=self.object).count()
        ctx['document_number'] = Document.objects.filter(category=self.object).count()
        return ctx

class ArticleListView(ContentListMixin, ListView):
    model = Article
    context_object_name = 'articles'
    filter_class = ArticleFilter

class ArticleCreateView(ContentCreateMixin, CreateView):
    model = Article
    form_class = ArticleForm
    success_url = reverse_lazy('article-list')

class ArticleUpdateView(ContentUpdateMixin, UpdateView):
    model = Article
    form_class = ArticleForm
    success_url = reverse_lazy('article-list')

class ArticleDetailView(HideRemovedMixin, JsonDetailMixin, DetailView):
    model = Article
    context_object_name = 'article'

    def get_template_names(self):
        if self.request.is_ajax() or self.request.GET.get('partial'):
            return 'djangocbv/_article_content_partial.html'
        return super().get_template_names()

class ArticleRemoveView(ContentRemoveMixin, UpdateView):
    model = Article
    success_url = reverse_lazy('article-list')

class ArticleUnpublishView(ContentUnpublishMixin, UpdateView):
    model = Article
    success_url = reverse_lazy('article-list')

class DocumentListView(ContentListMixin, ListView):
    model = Document
    context_object_name = 'documents'
    filter_class = DocumentFilter

class DocumentCreateView(ContentCreateMixin, CreateView):
    model = Document
    form_class = DocumentForm
    success_url = reverse_lazy('document-list')

class DocumentUpdateView(ContentUpdateMixin, UpdateView):
    model = Document
    form_class = DocumentForm
    success_url = reverse_lazy('document-list')

class DocumentDetailView(HideRemovedMixin, JsonDetailMixin, DetailView):
    model = Document
    context_object_name = 'document'

class DocumentRemoveView(ContentRemoveMixin, UpdateView):
    model = Document
    success_url = reverse_lazy('document-list')

class DocumentUnpublishView(ContentUnpublishMixin, UpdateView):
    model = Document
    success_url = reverse_lazy('document-list')

class DynamicTemplateView(TemplateView):
    def get_template_names(self):
        what = self.kwargs['what']
        return '{0}.html'.format(what)

As you will see there are

  • 4 views related to categories (Create, Detail, Update and List)
  • 6 views related to articles (Create, Detail, Update, List, Unpublish and Remove) and
  • another 6 views related to documents (same as articles).

The views for Article and Document are more or less the same: They inherit from the corresponding mixin we defined previously (CreateContentMixin, UpdateContentMixin etc) add a redirect to their corresponding list view (I am using redirect_lazy there because redirect wouldn’t work because it will lead to a cyclic dependency between urls and views) and define their corresponding form and model. For the the DetailViews I don’t use a group mixin like the others but I just add the HideRemovedMixin and JsonDetailMixin directly to their ancestor list. This is to make clear that the group mixins (ContentCreateMixin etc) are optional and I could for example define ArticleCreateView like this:

class ArticleCreateView(SuccessMessageMixin,
    model = Article
    form_class = ArticleForm
    success_url = reverse_lazy('article-list')
    success_message = 'Object successfully created!'

What’s the advantage of using the ContentCreateMixin then? Well since the same mixins are used from DocumentCreateView I won’t need to re-define this list there. Also if for example I want to allow users that have a specific permission to create articles and document I will remove the LoginRequireMixin and add the AllowCreateContentMixin only to the ContentCreateMixin and not to both ArticleCreateView and DocumentCreateView (and I won’t be in danger of forgetting to change it somewhere). Of course all this depends on your requirements and how DRY you need to be.

The category related views are simpler and override their mixins directly. Finally there’s a DynamicTemplateView to display templates based on their filename as discussed previously.

Before continuing, please try to understand how much more DRY this project is when compared to using a traditional functional one (or if not using mixins). For example, there’s a RequestArgMixin that is used by all views that create/update content. If INHERITANCE didn’t use that mixin I’d need to re-define the same functionality (pass the current request to the form’s constructor) to 4 views (Article/Document Create/Update). Or for the AuditableMixin I’d need to remember to upgrade the created/modified by to 8 views (Article/Document Create/Update/Unpublish/Remove)!


The previous discussion should convince you how much more DRY your views will be when using CBVs and how much quicker will be to create your views. Also, if you followed closely the first and second chapters you should be able to understand everything that is needed for CBVs and be able to properly understand which method or attribute you need to override to implement some specific functionality. Finally, the list of examples in the third chapter should help you get started in all your CBV needs - if you have some specific question about CBVs or you’d like another use case added to the list feel free to ask and I’ll try to add it!

Easy downloading youtube videos and mp3s using youtube-dl and python

In this article I am going to present you with an easy (and advertisement/malware free) way to download videos from youtube, converting them to mp3 if needed. Also, I will give some more useful hints, for example how to download multiple mp3s using a script, how to break a long mp3 to same-length parts so you could quickly skip tracks when you play it in your stereo etc.

I am going to give specific instructions for Windows users - however everything I’ll propose should also be applicable to OSX or Linux users with minor modifications.

The tools we are going to use are:

  • youtube-dl which is a python library for downloading videos from youtube (and some other sites)
  • ffmpeg which is a passepartout video/audio editing library


To install youtube-dl I recommend installing it in your global python (2 or 3) package list using pip. Please read my previous article

to see how you should install and use Python 2 or 3 on Windows. Following the instructions from there, you can do the following to install youtube-dl in your global Python 3 packages:

py -3 -m pip install youtube-dl

To run youtube-dl you’ll write something like py -3 -m youtube_dl url_or_video_id (notice the underscore instead of dash since dashes are not supported in module names in python). For example try something like this py -3 -m youtube_dl YgtL4S7Hrwo and you’ll be rewarded with the 2016 Pycon talk from Guido van Rossum! If you find this a little too much too type don’t be afraid, I will give some hints later about how to improve this.

To upgrade your youtube-dl installation you should do something like this:

py -3 -m pip install -U youtube-dl

Notice that you must frequently upgrade your youtube-dl installation because sometimes youtube changes the requirements for viewing / downloading videos and your old version will not work. So if for some reason something is not correct when you use youtube-dl always try updating first.

If you wanted you could also create a virtual environment (see instructions on previously mentioned article) and install youtube-dl locally there using pip install youtube-dl however I prefer to install it on the global packages to be really easy for me to call it from a command prompt I open. Notice that if you install youtube-dl in a virtualenv, after you activate that virtualenv you’ll be able to run it just by typing youtube-dl.

Finally, If for some reason you don’t want want to mess with python and all this (or it just seems greek to you) then you may go on and directly download a youtube-dl windows executable. Just put it in your path and you should be good to go.

To install ffmpeg, I recommend downloading the Windows build from here (select the correct Windows architecture of your system and always static linking - the version doesn’t matter). This will get you a zip - we are mainly interested to the three files under the bin folder of that zip which should be copied to a directory under your path:

  • ffmpeg is the passepartout video converting toot that we are going to use
  • ffprobe will print some information about a file (about its container and video/audio streams)
  • ffplay will play the file — not really recommended there are better tools but it is invaluable for testing; if it can be played be ffplay then ffmpeg will be able to properly read your file

Notice I recommend copying things to a directory in your path. This is recommended and will save you from repeatedly typing the same things over and over. Also, later I will propose a bunch of DOS batch (.bat) files that can also be copied to that directory and help you even more in you youtube video downloading. To add a directory to the PATH, just press Windows+Pause Break, Advanced System Settings, Advanced, Environment Variables, edit the “Path” User variable (for your user) and append the directory there.

Using youtube-dl

As I’ve already explained before, to run youtube-dl you’ll either write something like py -3 -m youtube_dl (if you’ve installed it to your global python packages) or run youtube-dl if you’ve downloaded the pre-built exe or have installed it in a virtualenv. To save you from some keystrokes, you can create a batch file that will run and pass any more parameters to it, something like this:

py -3 -m youtube_dl %*

(the %* will capture the remaining command line) so to get the previous video just run getvideo YgtL4S7Hrwo (or getvideo https://www.youtube.com/watch?v=YgtL4S7Hrwo - works the same with the video id or the complete url).

One thing I’d like to mention here is that youtube-dl works fine with playlists and even channels. For example, to download all videos from PyCon 2017 just do this:

getvideo https://www.youtube.com/channel/UCrJhliKNQ8g0qoE_zvL8eVg/feed and you should see something like:

E:\>py -3 -m youtube_dl https://www.youtube.com/channel/UCrJhliKNQ8g0qoE_zvL8eVg/feed
[youtube:channel] UCrJhliKNQ8g0qoE_zvL8eVg: Downloading channel page
[youtube:playlist] UUrJhliKNQ8g0qoE_zvL8eVg: Downloading webpage
[download] Downloading playlist: Uploads from PyCon 2017
[youtube:playlist] UUrJhliKNQ8g0qoE_zvL8eVg: Downloading page #1
[youtube:playlist] playlist Uploads from PyCon 2017: Downloading 143 videos
[download] Downloading video 1 of 143
[youtube] AjFfsOA7AQI: Downloading webpage
[youtube] AjFfsOA7AQI: Downloading video info webpage
[youtube] AjFfsOA7AQI: Extracting video information
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[download] Destination: Final remarks and conference close  - Pycon 2017-AjFfsOA7AQI.f137.mp4
[download]   2.9% of 34.49MiB at 940.52KiB/s ETA 00:36

This is gonna take some time …

Now, youtube-dl has many options and can be configured with default values depending on your requirements. I won’t go into detail about these except on some things I usually use, if you need some help feel free to ask me.

When you download a video, youtube-dl will try to download the best quality possible for that video, however a video may have various different formats that can be queries by passing the option --list-formats to ffmpeg, for example here’s the output from the previously mentioned video:

E:\>getvideo YgtL4S7Hrwo --list-formats
[youtube] YgtL4S7Hrwo: Downloading webpage
[youtube] YgtL4S7Hrwo: Downloading video info webpage
[youtube] YgtL4S7Hrwo: Extracting video information
[info] Available formats for YgtL4S7Hrwo:
format code  extension  resolution note
249          webm       audio only DASH audio   53k , opus @ 50k, 15.14MiB
250          webm       audio only DASH audio   72k , opus @ 70k, 20.29MiB
171          webm       audio only DASH audio  111k , vorbis@128k, 29.42MiB
140          m4a        audio only DASH audio  130k , m4a_dash container, mp4a.40.2@128k, 38.38MiB
251          webm       audio only DASH audio  130k , opus @160k, 36.94MiB
278          webm       256x144    144p   58k , webm container, vp9, 30fps, video only, 11.01MiB
242          webm       426x240    240p   88k , vp9, 30fps, video only, 12.40MiB
160          mp4        256x144    144p  120k , avc1.4d400c, 30fps, video only, 33.64MiB
243          webm       640x360    360p  153k , vp9, 30fps, video only, 23.48MiB
134          mp4        640x360    360p  230k , avc1.4d401e, 30fps, video only, 28.91MiB
133          mp4        426x240    240p  260k , avc1.4d4015, 30fps, video only, 74.75MiB
244          webm       854x480    480p  289k , vp9, 30fps, video only, 39.31MiB
135          mp4        854x480    480p  488k , avc1.4d401f, 30fps, video only, 56.43MiB
247          webm       1280x720   720p  945k , vp9, 30fps, video only, 102.45MiB
136          mp4        1280x720   720p 1074k , avc1.4d401f, 30fps, video only, 116.72MiB
17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k
36           3gp        320x180    small , mp4v.20.3, mp4a.40.2
43           webm       640x360    medium , vp8.0, vorbis@128k
18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k
22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)

As you can see, each has an id and defines an extension (container) and info about its video and audio stream. You can download a specific format by using the -f command line otpion. For example , to download the audio-only format with the worst audio quality use C:\Users\serafeim>getvideo YgtL4S7Hrwo -f 249. Notice that there are formats with audio ony and other formats with vide only. To download the worst format possible (resulting in the smallest file size of course ) you can pass the -f worst command line (there’s also a -f best command line which is used by default).

Another thing I’d like to point out here is that you can define an output template using the -o option that will format the name of the output file of your video using the provided options. There are many examples in the docs so I won’t go into any more details here.

Another cool option is the -a that will help you download all videos from a file. For example, if you have a file named videos.txt with the following contsnts:


running getvideo -a videos.txt -f worst

will get you all three videos in their worst quality. If you don’t want to create files then you can use something like this:

for %i in (AjFfsOA7AQI 3dDtACSYVx0 G17E4Muylis) do getvideo %i -f worst

and it will run getvideo for all three files.

Some more options I’d like to recommend using are:

  • --restrict-filenames to avoid strange filenames
  • --ignore-errors to ignore errors when download multiple files (from a playlist or a channel) - this is really useful because if you have a play with missing items youtube-dl will stop downloading the remaining files when it encounters the missing one

If you want to always use these options you may add them to your configuration file (C:\Users\<user name>\youtube-dl.conf) or to the getvideo.bat defined above i.e getvideo.bat will be:

py -3 -m youtube_dl --restrict-filenames --ignore-errors %*

Extracting mp3s

The next step in this trip is to understand how to extract mp3s from videos that are downloaded from youtube. If you’ve payed attention you’d know that by now you can download audio-only formats from youtube - however they are in a format called DASH which most probably is not playable by your car stereo (DASH is specialized for streaming audio through HTTP).

Thus, the proper way to get mp3s is to post-process the downloaded file using ffmpeg to convert it to mp3. This could be done manually (by doing something ffmpeg -i input out.mp3 — ffmpeg is smart enough to know how to convert per extension) but thankfully youtube-dl offers the -x (and friend) parameters to make this automatic. Using -x tells youtube-dl to extract the audio from the video (notice that youtube-dl is smart enough to download one of the audio-only formats so you don’t have ). Using -x alone may result in a different audio format (for example .ogg) so to force conversion to mp3 you should also add the --audio-format mp3 parameter. Thus, to download an mp3 you can use the following command line (continuing from the previous examples):

py -3 -m youtube_dl --restrict-filenames --ignore-errors -x --audio-format mp3  AjFfsOA7AQI

or even better, create a getmp3.bat batch file that will be used to retrieve an mp3:

py -3 -m youtube_dl --restrict-filenames --ignore-errors -x --audio-format mp3 %1

Please notice that also in this case youtube-dl is smart enough to download an audio-only format thus you won’t need to select it by hand using -f to save bandwith.

Splitting the mp3 file to parts

Some people would like to split their large mp3 files to same-length segments. Of course it would be better for the file to be split by silence to individual songs (if the file contains songs) but these methods usually don’t work that good so I prefer the same length segments. To do that using ffmpeg you just need to add the following parameters:

ffmpeg -i input.mp3 -segment_time 180 -f segment out.%03d.mp3"

The segment time is in seconds (so each segment will be 3 minutes) while the output files will have a name like out.001.mp3, out.002.mp3 etc.

What if you’d like to make the segmentation automatic? For this, I recommend writing a batch file with two commands - one to download the mp3 and a second one to call ffmpeg to segment the file. Notice that you could use the --postprocessor-args ARGS command line parameter to pass the required arguments to youtube-dl so it will be done in one command however I’d like to have a little more control thus I prefer two commands (if you decide to use --postprocessor-args ARGS keep in mind that args must be inside double quotes “”).

Since we are going to use two commands, we need to feed the output file of youtube-dl to ffmpeg and specify a name for the ffmpeg output file-segments. The easiest way to do that is to just pass two parameters to the batch file - one for the video to download and one for its name. Copy the following to a file named getmp3seg.bat:

py -3 -m youtube_dl %1 -x --audio-format mp3 --audio-quality 128k -o %2.%%(ext)s"
ffmpeg -i %2.mp3 -segment_time 180 -f segment %2.%%03d.mp3
del %2.mp3

You can then call it like this: getmp3seg AjFfsOA7AQI test. The first line will download and covert the video to mp3 and put it in a file named test.mp3 (the %2 is the test, the %% is used to escape the % and the %(ext)s is the extensions - this is needed if you use something like -o %2.mp3 youtube-dl will be confused when trying to convert the file to mp3 and will not work). The 2nd line will segment the file to 180 second seconds (notice that here also we need to escape %) and the third line will delete the original mp3. This leaves us with the following 4 files (the video was around 10 minutes): test.000.mp3, test.001.mp3, test.002.mp3, test.003.mp3.

One final thing I’d like to present here is a (more complex) script that you can use to download a video and segmentize it only if it is more than 360 seconds. For this, we will also use the mp3info util which can be downloaded directly from the homepage and copied to the path. So copy the following to a script named getmp3seg2.bat:

@echo off

IF "%2"=="" GOTO HAVE_1

py -3 -m youtube_dl %1 -x --audio-format mp3 -o %2.%%(ext)s"

FOR /f %%i IN ('mp3info -p "%%S" %2.mp3') DO SET koko=%%i

IF %koko%  GTR 360 (
        ECHO greater than or equal to 360
        ffmpeg -i %2.mp3 -segment_time 180 -f segment %2.%%03d.mp3
        del %2.mp3
)  else (
   ECHO less than 360

GOTO :eof

ECHO Please call this file with video id and title

This is a little more complex - I’ll explain it quickly: @echo off is used to suppress non needed output. The IF following makes sure that you have two parameters. The next line downloads the file and converts it to mp3. The FOR loop is a little strange but it’s result will be to retrieve the output of mp3info -o "%S" title.mp3 (which is the duration in seconds of that mp3) and assign it to the koko variable. The next IF checks if koko is greater than (GTR) 360 seconds and if yes will run the conversion code we discussed before - else it will just output that it is less than 360 seconds.

Finally, there’s a GOTO: eof line to skip printing the error message when the batch is called with less than two parameters.

Using youtube-dl from python

Integrating with youtube-dl from python is easy. Of course, you could just go on and directly call the command line however you can have more control. The most important class is youtube_dl.YoutubeDL.YoutubeDL. You instantiate an object of this class class passing the parameters you’d like and call its download() instance method passing a list of urls. Here’s a small script that downloads the input video ids:

import sys
from youtube_dl import YoutubeDL

if __name__ == '__main__':
    if len(sys.argv) > 1:
        ydl_opts = {}
        ydl = YoutubeDL(ydl_opts)
        print("Enter list of urls to download")

Save it in a file named getvideo.py and run it like py -3 getvideo.py AjFfsOA7AQI 3dDtACSYVx0 G17E4Muylis to download all three videos!

Fixing your unicode names

The last thing I’d like to talk about concerns people that want to download videos with Unicode characters in their titles (for example Greek).

Let’s suppose that you want to download the file vFVNOaUPRow which is piano music from a well-know greek composer. If you get it without parameters (for example py -3 -m youtube_dl -x --audio-format mp3 vFVNOaUPRow) you’ll get the following output file: Ο Μάνος Χατζιδάκις. παίζει 11 κομμάτια στο πιάνο-vFVNOaUPRow.f247.mp3 (notice the greek characters) while, if you add the --restrict-filenames I mentioned before you’ll get _11-vFVNOaUPRow.f247.mp3 (notice that the greek characters have been removed since they are not safe).

So if you use the --restrict-filenames parameter you’ll get an output that contains only the video id (and any safe characters it may find) while if you don’t use it you’ll get the normal title of the video. However, most stereos do not display unicode characters properly so if I get this file to my car I’ll see garbage and I won’t be able to identify it — I will be able to listen it but not see its name!

To fix that, I propose transliterating the unicode characters using the unidecode library. Just install it using pip. Then you can the following script to rename all mp3 files in a directory to using english characters only:

import os, unidecode

if __name__ == '__main__':
    for file in os.listdir('.'):
        if file.endswith('mp3'):
            print("Renaming {0} to {1}".format(file, unidecode.unidecode(file)))
            os.rename(file, unidecode.unidecode(file))

Copy this to a file named transliterate.py and run it in a directory containing mp3 files (py -3 transliterate.py) to rename them to non-unicode characters.

Authentication for django-rest-framework with django-rest-auth


Most of the times I need authentication with any REST APIs defined through django-rest-framework I will use SessionAuthentication method. This method uses the session cookie (which is set through the normal Django login and logout views) to check out if there’s an authenticated user and get his username. This method works only in the same session (browser window) as the one that actually did the login but this should be enough for most cases.

However, sometimes instead of using the normal Django login/logout views, you’ll want to authentication through REST end-points, for example for using them with SPAs (where you don’t want to use the traditional views for authentication but through REST end-points) or because you have implemented a mobile (or desktop) application that needs to authenticate with your script.

There are various ways this could be done but one of the simplest is using django-rest-auth. This project adds a number of REST end-points to your project that can be used for user login and registration (and even social login when combined with django-allauth). In the following I am going to write a simple tutorial on how to actually use django-rest-auth to authenticate with django-rest-framework using the provided REST end points and how to call a REST API as an authenticated user.

Before continuing with the tutorial, let’s take a look at what we’ll build here:

Our project

This is a single html page (styled with spectre.css) that checks if the user is logged in and either displays the login or logout button (using javascript). When you click the login you’ll get a modal in which you can enter your credentials which will be submitted through REST to the django-rest-auth endpoint and depending on the response will set a javascript variable (and a corresponding session/local storage key). Then you can use the “Test auth” button that works only on authenticated users and returns their username. Finally, notice that after you log out the “test auth” button returns a 403 access denied.

If you want to play with this project yourself, you can clone it here https://github.com/spapas/rest_authenticate. Just create a venv, install requirements, create a superuser and you should be good to go!

Some theory

After you log in with Django, your authentication information is saved to the “session”_. The session is a bucket of information that the Django application saves about your visit — to distinguish between different visitors a cookie with a unique value named sessionid will be used. So, your web browser will send this cookie with each page request thus allowing Django to know which bucket of information is yours (and if you’ve authenticated know who are you). This is not a Django related concept but a general one (supported by most if not all HTTP frameworks) and is used to add state to an otherwise stateless medium (HTTP).

Since the sessionid cookie is sent not only with traditional but also with Ajax request it can be used to authenticate REST requests after you’ve logged in. This is what is used by default in django-rest-framework and as I said in the introduction it is a very good solution for most use cases: You login to django and you can go ahead and call the REST API through Ajax; the sessionid cookie will be sent along with the request and you’ll be authenticated.

Now, although the session authentication is nice for using in browsers, you may need to access your API through a desktop or a mobile application where, setting the cookies yourself is not the optimal solution. Also, you may have an SPA that needs to access an API in a different domain; using using cookies for this is not easy - if possible at all.

For such cases, django-rest-framework offers a different authentication method called TokenAuthentication_. Using this method, each user of the Django application is correlated with a random string (Token) which is passed along with the request at its header thus the Django app can authenticate the user using this token! One thing that may seem strange is that since both the session cookie and a token are set through HTTP Headers why all the fuss about tokens? Why not just use the session cookie and be done with it. Well, there are various reasons - here’s a rather extensive article explaining some. Some of the reasons are that a token can be valid forever while the session is something ephemeral - beyond authorization information, sessions may keep various other data for a web application and are expired after some time to save space. Also, since tokens are used for exactly this (authentication) they are much easier to use and reason about. Finally, as I’ve already explained, sharing cookies by multiple sites is not something you’d like to do.

Installation & configuration

To install django-rest-auth just follow the instructions here i.e just add 'rest_framework', 'rest_framework.authtoken' and 'rest_auth' to your INSTALLED_APPS in settings.py and run migrate.

Since I won’t be adding any other apps to this project (no models are actually needed), I’ve added two directories static and templates to put static files and templates there. This is configured by adding the 'DIRS' attribte to TEMPLATES, like this:

        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            os.path.join(BASE_DIR, 'templates'),
        // ...

and adding the STATICFILES_DIRS setting:

    os.path.join(BASE_DIR, "static"),

The remaining setting are the default as were created by django-admin startproject.

I have included the the following urls to urls.py:

urlpatterns = [
    path('admin/', admin.site.urls),
    path('test_auth/', TestAuthView.as_view(), name='test_auth', ),
    path('rest-auth/logout/', LogoutViewEx.as_view(), name='rest_logout', ),
    path('rest-auth/login/', LoginView.as_view(), name='rest_login', ),
    path('', HomeTemplateView.as_view(), name='home', ),
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

These are: The django-admin, a test_auth view (that works only for authenticated users and returns their username), a view (LogoutViewEx) that overrides the rest-auth REST logout-view (I’ll explain why this is needed in a minute), the rest-auth REST login-view, the home template view (which is the only view implemented) and finally a mapping of your static files to the STATIC_URL.

The views

There are three views in this application - the HomeTemplateView, the TestAuthView and the LogoutViewEx view that overrides the normal LogoutView of django-rest-auth. The first one is a simple TemplateView that just displays an html page and loads the client side code - we’ll talk about it later in the front-side section.

The TestAuthView is implemented like this:

class TestAuthView(APIView):
    authentication_classes = (authentication.TokenAuthentication,)
    permission_classes = (permissions.IsAuthenticated,)

    def get(self, request, format=None):
        return Response("Hello {0}!".format(request.user))

This is very simple however I’d like to make a few comments about the above. First of all you see that I’ve defined authentication_classes and permission_classes. These options define

  • which method will be used for authenticating access to the REST view i.e finding out if the user requesting access has logged in and if yes what’s his username (in our case the TokenAuthentication will be used)
  • if the user is authorized (has permission) to call this REST view (in our case only authenticated users will be allowed)

The authentication and permission clases can be set globally in your settings.py using REST_FRAMEWORK['DEFAULT_AUTHENTICATION_CLASSES'] and REST_FRAMEWORK['DEFAULT_PERMISSION_CLASSES'] or defined per-class like this. If I wanted to have the same authentication and permission classes defined in my settings.py so I wouldn’t need to set these options per-class I’d add the following to my settings.py:


Finally, keep in mind that you haven’t defined these in your views or your settings, they will have the following default values:


The above mean that if you don’t define authentication and permission classes anywhere then the REST views will use either session authentication (i.e the user has logged in normally using the Django login views as explained before) or basic authentication (the request provides the credentials in the header using traditional HTTP Basic authentication) and also that all users (logged in or not) will be allowed to call all APIs (this is probably not something you want).

The TokenAuthentication that we are using instead means that for every user there must be a valid token which will be provided for each request he does. The tokens are normal object instances of rest_framework.authtoken.models.Token and you can take a look at them (or even add one) through the Django admin (auth token - tokens). You can also even do whatever you normally would do to an object instance, for example:

>>> [ (x.user, x.key) for x in Token.objects.all()]
[(<User: root>, 'db4dcc1b9d00d1af74fb3cb41e1f9e673208485b')]

To authenticate with a token (using TokenAuthentication), you must add an extra header to your request with the format Authorization: Token token for example in the previous case root would add Authorization: Token db4dcc1b9d00d1af74fb3cb41e1f9e673208485b. To do this you’ll need something client-side code which we’ll see in the next section.

To do it with curl you can just do something like this:

curl -H "Authorization:Token db4dcc1b9d00d1af74fb3cb41e1f9e673208485b"

Try it with a valid and invalid token and without providing a token at all and see the response each time.

So, django-rest-framework provides the model (Token) and the mechanism (add the extra Authentication header) for authentication with Tokens. What it does not provide is a simple way to create/remove tokens for users: This is where django-rest-auth comes to the rescue! Its login and logout REST views will automatically create (and delete) tokens for the users that are logging in. They will also authenticate the user normally (using sessions) - this means that if a user logs in using the login REST endpoint he’ll then be logged in normally to the site and be able to access non-REST parts of the site (for example the django-admin). Also, if the user logs in through the django-rest-auth REST end point and if you have are using SessionAuthentication to one of your views then he’ll be able to authenticate to these views without the need to pass the token (can you understand why?).

Finally, let’s take a look at the LogoutViewEx:

class LogoutViewEx(LogoutView):
    authentication_classes = (authentication.TokenAuthentication,)

This class only defines the authentication_classes attribute. Is this really needed? Well, it depends on you project. If you take a look at the source code of LogoutView (https://github.com/Tivix/django-rest-auth/blob/master/rest_auth/views.py#L99) you’ll see that it does not define authentication_classes. This, as we’ve already discussed, means that it will fall-back to whatever you have defined in the settings (or the defaults of django-rest-framework). So, if you haven’t defined anything in the settings then you’ll get the by default the SessionAuthentication and BasicAuthentication methods (hint: not the TokenAuthentication). This means that you won’t be able to logout when you pass the token (but will be able to logout from the web-app after you login - why?). So to make everything crystal and be able to reason better about the behavior I specifically define the LogoutViewEx to use the TokenAuthentication - that’s what you’d use if you developed a mobile or desktop app anyway.

The client side scripts

I’ve included all client-side code to a home.html template that is loaded from the HomeTemplateView. The client-side code has been implemented only with jQuery because I think this is the library that most people are familiar with - and is really easy to be understood even if you are not familiar with it. It more or less consists of four sections in html:

  • A user-is-logged-in section that displays the username and the logout button
  • A user-is-not-logged-in section that displays a message and the login button
  • A test-auth section that displays a button for calling the TestAuthView defined previously and outputs its response
  • The login modal

Here’s the html (using spectre.css for styling):

<div class="container grid-lg">
    <div class="columns" id="non-logged-in">
        <div class='column col-3'>
            You have to log-in!
        <div class='column col-3'>
            <button class="btn btn-primary"  id='loginButton'>Login</button>
    <div class="columns" id="logged-in">
        <div class='column col-3'>
            Welcome <span id='span-username'></span>!
        <div class='column col-3'>
            <button class="btn btn-primary"  id='logoutButton'>Logout</button>
    <hr />
    <div class="columns" id="test">
        <div class='column col-3'>
            <button class="btn btn-primary"  id='testAuthButton'>Test auth</button>
        <div class='column col-9'>
            <div id='test-auth-response' ></div>

<div class="modal" id="login-modal">
    <a href="#close" class="modal-overlay close-modal" aria-label="Close"></a>
    <div class="modal-container">
        <div class="modal-header">
            <a href="#close" class="btn btn-clear float-right close-modal" aria-label="Close"></a>
            <div class="modal-title h5">Please login</div>
        <div class="modal-body">
            <div class="content">
                    {% csrf_token %}
                    <div class="form-group">
                        <label class="form-label" for="input-username">Username</label>
                        <input class="form-input" type="text" id="input-username" placeholder="Name">
                    <div class="form-group">
                        <label class="form-label" for="input-password">Password</label>
                        <input class="form-input" type="password" id="input-password" placeholder="Password">
                    <div class="form-group">
                        <label class="form-checkbox" for="input-local-storage">
                            <input type="checkbox" id="input-local-storage" /> <i class="form-icon"></i>  Use local storage (remember me)
                <div class='label label-error mt-1 d-invisible' id='modal-error'>
                    Unable to login!
        <div class="modal-footer">

            <button class="btn btn-primary" id='loginOkButton' >Ok</button>
            <a href="#close" class="btn close-modal" >Close</a>

The html is very simple and I don’t think I need to explain much - notice that the #logged-in and #non-logged-in sections are mutually exclusive (I use $.show() and $.hide() to show and hide them) but the #test section is always displayed so you’ll be able to call the test REST API when you are and are not authenticated. For the modal to be displayed you need to add an active class to its #modal container.

For the javascript, let’s take a look at some initialization stuff:

var g_urls = {
    'login': '{% url "rest_login" %}',
    'logout': '{% url "rest_logout" %}',
    'test_auth': '{% url "test_auth" %}',
var g_auth = localStorage.getItem("auth");
if(g_auth == null) {
    g_auth = sessionStorage.getItem("auth");

if(g_auth) {
    try {
        g_auth = JSON.parse(g_auth);
    } catch(error) {
        g_auth = null;

var getCookie = function(name) {
    var cookieValue = null;
    if (document.cookie && document.cookie !== '') {
        var cookies = document.cookie.split(';');
        for (var i = 0; i < cookies.length; i++) {
            var cookie = jQuery.trim(cookies[i]);
            // Does this cookie string begin with the name we want?
            if (cookie.substring(0, name.length + 1) === (name + '=')) {
                cookieValue = decodeURIComponent(cookie.substring(name.length + 1));
    return cookieValue;
var g_csrftoken = getCookie('csrftoken');

var initLogin = function() {
    if(g_auth) {
        if(g_auth.remember_me) {
            localStorage.setItem("auth", JSON.stringify(g_auth));
        } else {
            sessionStorage.setItem("auth", JSON.stringify(g_auth));
    } else {

First of all, I define a g_urls window/global object that will keep the required REST URLS (login/logout and test auth). These are retrieved from Django using the {% url %} template tag and are not hard-coded. After that, I check to see if the user has authenticated before. Notice that because this is client-side code, I need to do that every time the page loads or else the JS won’t be initialized properly! The user login information is stored to an object named g_auth and contains two attributes: username, key (token) and remember_me.

To keep the login information I use either a key named auth to either the localStorage or the sessionStorage. The sessionStorage is used to save info for the current browser tab (not window) while the localStorage saves info for ever (until somebody deletes it). Thus, localStorage can be used for implementing a “remember me” functionality. Notice that instead of using the session/local storage I could instead integrate the user login information with the Django back-end. To do this I’d need to see if the current user has a session login and if yes pass his username and token to Javascript. These values would then be read by the login initialization code. I’m leaving this as an exercise for attentive readers.

Getting the login information from the session probably is a better solution for web-apps however I think that using the local or session storage emulate better a more general (and completely stateless) behaviour especially considering that the API may be used for mobible/desktop apps.

In any case, after you’ve initialized the g_auth object you’ll need to read the CSRF cookie. By default Django requires CSRF protection for all POST requests (we do a POST request for login and logout). What happens here is that for pages that may need to do a POST request, Django will set a cookie (CSRF cookie) in its initial response. You’ll need to read that cookie and submit its value along with the rest of your form fields when you do the POST. So the getCookie function is just used to set the g_csrftoken with the value of the CSRF cookie.

The final function we define here (which is called a little later) checks to see if there is login information and hides/displays the correct things in html. It will also set the local or session storage (depending on remember me value).

After that, we have some client side code that is inside the $() function which will be called after the page has completely loaded:

$(function () {

    $('#loginButton').click(function() {

    $('.close-modal').click(function() {

    $('#testAuthButton').click(function() {
            url: g_urls.test_auth,
            method: "GET",
            beforeSend: function(request) {
                if(g_auth) {
                    request.setRequestHeader("Authorization", "Token " + g_auth.key);
        }).done(function(data) {
            $('#test-auth-response').html("<span class='label label-success'>Ok! Response: " + data);
        }).fail(function(data) {
            $('#test-auth-response').html("<span class='label label-error'>Fail! Response: " + data.responseText + " (status: " + data.status+")</span>");

    // continuing below ...

The first thing happening here is to call the initLogin function to properly intiialize the page and then we add a couple of handlers to the click buttons of the #loginButton (which just displays the modal by adding the active class ), .close-modal class (there are multiple ways to close the modal thus I use a class which just removes that active class) and finally to the #testAuthButton. This button will do a GET request to the g_urls.test_auth we defined before. The important thing to notice here is that we add a beforeSend attribute to the $.ajax request which, if g_auth is defined adds an Authorization header with the token in the form that django-rest-framework TokenAuthentication expects and as we’ve already discussed above:

beforeSend: function(request) {
    if(g_auth) {
        request.setRequestHeader("Authorization", "Token " + g_auth.key);

If this ajax call returns ok (done part) we just add the Ok to a green label else if there’s an error (fail part) we add the response text and status to a red label. You can try clicking the button and you see that only if you’ve logged in you will succeed in this call.

Let’s now take a look at the #loginOkbutton click handler (inside the modal):

$('#loginOkButton').click(function() {
    var username = $('#input-username').val();
    var password = $('#input-password').val();
    var remember_me = $('#input-local-storage').prop('checked');
    if(username && password) {
        console.log("Will try to login with ", username, password);
            url: g_urls.login,
            method: "POST",
            data: {
                username: username,
                password: password,
                csrfmiddlewaretoken: g_csrftoken
        }).done(function(data) {
            console.log("DONE: ", username, data.key);
            g_auth = {
                username: username,
                key: data.key,
                remember_me: remember_me
            // CAREFUL! csrf token is rotated after login: https://docs.djangoproject.com/en/1.7/releases/1.5.2/#bugfixes
            g_csrftoken = getCookie('csrftoken');
        }).fail(function(data) {
            console.log("FAIL", data);
    } else {

All three user inputs (username, password, remember_me) are read from the form and if both username and password have been defined an Ajax request will be done to the g_urls.login url. We pass username, password and g_csrftoken (as discussed before) as the request data. Now, if there’s an error (fail) I just display a generic message (by removing it’s d-invisible class) while, if the request was Ok I retrieve the key (token) from the response, initialize the g_auth object with the username, key and remember_me values and call initLogin to show the correct divs and save to the session/local storage.

It is important to keep in mind that with the line g_csrftoken = getCookie('csrftoken') we re-read the CSRF cookie. This is needed because, as you can see in the mentioned link in the comment, after Django logs in, the csrf cookie value is rotated for security reasons so it must be re-read here (or else the logout that is also a POST request will not work).

Finally, here’s the code for logout (still inside the $(function () {):

    $('#logoutButton').click(function() {
        console.log("Trying to logout");
            url: g_urls.logout,
            method: "POST",
            beforeSend: function(request) {
                request.setRequestHeader("Authorization", "Token " + g_auth.key);
            data: {
                csrfmiddlewaretoken: g_csrftoken
        }).done(function(data) {
            console.log("DONE: ", data);
            g_auth = null;
        }).fail(function(data) {
            console.log("FAIL: ", data);

}); // End of $(function () {

The code here is very simple - just do a POST to the g_urls.logout and if everything is ok delete the g_auth values and call initLogin() to show the correct divs and remove the auth key from local/session storage. Notice that when you POST to the logout REST end-point, you need to also add the Authorization header with the token or else (since we’ve defined only TokenAuthentication for the authentication_classes for the LogoutViewEx class) there won’t be any way to correlate the request with the user and log him out!


Using the info presented on this article you should be able to properly login and logout to Django using REST and also call REST end-points as an authenticated used. I recommend using the curl utility to try to call the rest end point with various parameters to see the response. Also, you change the LogoutViewEx with the default django-rest-auth LogoutView and then try logging out through the web-app and through curl and see what happens when you try to access the test-auth end-point.

Finally, the above project can be easily modified to use SessionAuthentication instead of TokenAuthentication (so you won’t need django-rest-auth at all) - I’m leaving it as an exercise to the reader.

Using both Python 2 and 3 in Windows

The release of Django 2.0 was a milestone in the history of Python since it completely dropped support for Python 2.x. I am a long time user of Django and, loyal to the philosophy of “if it is working don’t change” I was always using Python 2.x.

Of course right now this needs to change since new applications will need to be developed to the latest version of Django (and Python). I am using Windows for development (almost exclusively) and what I wanted was to be able to create and use virtual environments for both my old projects (using Python 2) and new projects (using Python 3).

The above requirement is not as straight-forward as I’d like - actually it is if you know what you need to do and which tools you must use. That’s why I decided to write a quick step by step tutorial on how to use both versions of Python in your Windows environment. Notice that I am using Windows 10, Python 2.7.14 an Python 3.6.4.

First of all, let’s download both versions of Python from the Python download page. I downloaded the files python-2.7.14.msi and python-3.6.4.exe (not sure why the one is .msi and the other is .exe it doesn’t matter anyway).

Firstly I am installing Python 2.7.14 and selecting:

  • Install for all users
  • Install to default folder
  • Press next to following screen (install default customizations)
  • This won’t add the python.exe of Python 2.7 to path

Next I am install Python 3.6.4:

  • Make sure to click “Install launcher for all users (recommended)”
  • I also check “Add Python 3.6to PATH” (to add the Python 3.6 executable to path)
  • I then just click “Install Now” (this will put Python 3.6 to c:)

Right now if you open a terminal window (Windows+r, cmd.exe) and run Python you will initiate the Python 3.6 interpreter. This is useful for just dropping in a Python interpreter.

Remember the “Install launcher for all users (recommended)” we clicked before? This installs the python launcher which is used to select between Python versions. If you run it without parameters the Python 3.6 interpreter will by started. You can pass the -2 parameter to start the python 2.x interpreter or -3 to explicitly declare the python 3.x interpreter:

C:\Users\Serafeim>py -2
Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:19:30) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^Z

C:\Users\Serafeim>py -3
Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> ^Z

With py we can easily start the Python interpreter we want. This is not enough though - we need to use virtualenv and create proper virtual environments for our projects. To do this you can add the -m option to py to run a module with the proper python version. For example, to start an http server with Python 2 you would use the module SimpleHTTPServer while for python 3 you would use http.server (as per this):

C:\progr\py\pelican\spapas.github.io>py -2 -m SimpleHTTPServer
Serving HTTP on port 8000 ...

C:\progr\py\pelican\spapas.github.io>py -3 -m http.server
Serving HTTP on port 8000 ( ...

Now, to create the virtual environments we’ll use the virtualenv module which is installed by default in Python 2.x (try running py -2 -m pip freeze to see the list of installed packages for Python 2.x) and the venv module which is included in the Python 3.x core. So to create a virtualenv for Python 2 we’ll run py -2 -m virtualenv name-of-virtualenv and for Python 3 py -3 -m venv name-of-virtualenv.

C:\progr\py>py -2 -m virtualenv venv-2
New python executable in C:\progr\py\venv-2\Scripts\python.exe
Installing setuptools, pip, wheel...done.

C:\progr\py>py -3 -m venv venv-3


(venv-2) C:\progr\py>python -V
Python 2.7.14

(venv-2) C:\progr\py>deactivate


(venv-3) C:\progr\py>python -V
Python 3.6.4

That’s how easy it is to have both Python 2.7 and Python 3.6 in your Windows!

My essential django package list

In this article I’d like to present a list of django packages (add-ons) that I use in most of my projects. I am using django for more than 5 years as my day to day work tool to develop applications for the public sector organization I work for. So please keep in mind that these packages are targeted to the “enterpripse” audience and some things that target public (open access) web apps may be missing.

Some of these packages are included in a django cookiecutter I am using to start new projects and also you can find example usage of some of these packages in my Mailer Server project. I’ll also be happy to answer any questions about these, preferably in stackoverflow if the questions are fit for that site.

Finally, before going to the package list, notice that most of the following are django-related and I won’t include any generic python packages (with the exception of xhtml2pdf which I discuss later). There are a bunch of python-generic packages that I usually use (xlrd, xlwt, requests, raven, unidecode, database connectors etc) but these won’t be discussed here.

Packages I use in all my projects

These packages are used more or less in all my projects - I just install them when I create a new project (or create the project through the cookiecutter I mention above)/


django-tables2, along with django-filter (following) are the two packages I always use in my projets. You create a tables.py module inside your applications where you define a bunch of tables using more or less the same methodology as with django-forms: For a Book model, Create a BookTable class, relate it with the model and, if needed override some of the columns it automatically generates. For example you can configure a column that will behave as a link to a detail view of the book, a column that will behave as a date (with the proper formatting) or even a column that will display a custom django template snippet.

You’ll then be able to configure the data (queryset) of this table and pass it your views context. Finally use {% render_table table %} to actually display your table. What you’ll get? The actual html table (with a nice style that can be configured if needed), free pagination and free column-header sorting. After you get the hang of it, you’ll be able to add tables for your models in a couple of minutes - this is DRY for me. It also offers some class based views and mixins for even more DRY adding tables to your views.

Finally, please notice that since version 1.8, django-tables2 supports exporting data using tablib, an old requirement of most users of this library.


django-filter is used to create filters for your data. It plays well with django-tables2 but you can use it to create filters for any kind of listing you want. It works similar to django-tables2: Create a filters.py module in your application and define the a BookFilter class there by relating it with the Book model, specifying the fields of the model you want to filter with and maybe override some of the default options. New versions have some great built-in configuration functionality by customizing which method will be used for filtering or even adding multiple filter methods - for example you could define your BookFilter like this:

class BookFilter(django_filters.FilterSet):
    class Meta:
        model = Book
        fields = {
            'name': ['icontains'],
            'author__last_name': ['icontains', 'startswith'],
            'publication_date': ['year', 'month', ],

which will give you the following filter form fields:

  • an ilike ‘%value%’ for the book name
  • an ilike ‘%value%’ for the author name
  • a like ‘value%’ for the author name
  • a year(publication_date) = value for the publication_date
  • a month(publication_date) = value for the publication_date

and their (AND) combinations!

The BookFilter can be used to create a filter form in your template and then in your views pass to it the initial queryset along with request.GET (which will contain the filter values) to return the filtered data (and usually pass it to the table). I’ve created a sample project that uses both django-tables2 and django-filters for you to use: https://github.com/spapas/django_table_filtering. Also, I’ve written an article which describes a technique for automatically creating a filter-table view.


The forms that are created by default by django-forms are very basic and not styled properly. To overcome this and have better styles for my forms, I always use django-crispy-forms. It actually has two modes: Using the crispy template filter and using the crispy template tag. Using the crispy template filter is very simple - just take a plain old django form and render it in your template like this {{ form|crispy }}. If the django-crispy-forms has been configured correctly (with the correct template pack) the form you’ll get will be much nicer than the django-default one. This is completely automatic, you don’t need to do anything else!

Now, if you have some special requirements from a form, for example multi-column rendering, adding tabs, accordions etc then you’ll need to use the {% crispy %} template tag. To use this you must create the layout of your form in the form’s costructor using the FormHelper django-crispy-forms API. This may seem cumbersome at first (why not just create the form’s layout in the django template) but using a class to define your form’s layout has other advantages, for example all the form layout is in the actual form (not in the template) you can control programatically the layout of the form (f.e display some fields only for administrators), you can use inheritance and virtual methods to override how a form is rendered etc.

To help you understand how a FormHelper looks like, here’s a form that is used to edit an access Card for visitors that displays all fields horizontally inside a panel (I am using Bootstrap for all styling and layout purposes):

class CardForm(ModelForm):
    class Meta:
        model = Card
        fields = ['card_number', 'card_text', 'enabled']

    def __init__(self, *args, **kwargs):
        self.helper = FormHelper()
        self.helper.form_method = 'post'
        self.helper.layout = Layout(
                HTML(u"""<div class="panel panel-primary">
                   <div class="panel-heading">
                       <h3 class="panel-title">Add card</h3>
                   <div class="panel-body">"""),
                   Div('card_number', css_class='col-md-4'),
                   Div('card_text', css_class='col-md-4'),

                Submit('submit', u'Save', css_class='btn btn-success'),
                HTML(u"<a class='btn btn-primary' href='{% url \"card_list\" %}'>Return</a>" ),

        super(CardForm, self).__init__(*args, **kwargs)

Notice that forms that will be rendered with a FormHelper actually contain their <form> tag (you don’t need to write it yourself like with plain django forms) so you have to define their method (post in this example) and submit button.


django-extensions is a swiss-army-knife of django tools. I use it always in my projects because of the runserver_plus and shell_plus commands. The first uses the Werkzeug debugger with django which makes django development an absolute joy (open a python shell wherever you want in your code and start writing commands)! The second opens a better shell (your models and a bunch of django stuff are auto-imported, a better shall will be used if found etc).

The runserver_plus and shell_plus alone will be more than enough for me to use this however it adds some more usefull management commands like: admin_generator to quickly create an admin.py for your app, graph_models to generate a graphviz dot file of your models, update_permissions to synchronize the list of permissions if you have added one to an existing model and many, many others. Take a look at them and you’ll probably find more useful things!


django-autocomplete-light is the best auto-complete library for django, especially after v3 was released (which greatly reduces the magic and uses normal CBVs for configuring the querysets). You will create an AutocompleteView for a model (similar to your other class based views) and then automatically use this view through a widget in the admin or in your own forms. It is fully configurable (both the results and the selection templates), supports many to many fields, creating new instances and even autocompleteing django-taggit tags! If for some reason it seems that it is not working please keep in mind that you need to includ jquery and {{ form.media }} to your templates or else the required client side code won’t be executed.

I think it is an essential for all cases because dropdowns with lots of choices have a very bad user experience - and the same is true with many to many fields (you could use the checkboxes widget to improve their behavior a little but you will have bad behavior when there are many choices).


django-reversion is a really important package for my projects. When it is configured properly (by adding a a reversion-middleware), it offers full auditing for all changes in the your model instances you select (and properly groups them in case of changes to multiple instances in a simple request). It saves a JSON representation of all the versions of an instance in your database. Keep in mind that this may increase your database size but if you need full auditing then is is probably the best way to do it. I have written an article about django model auditing that discusses this package and django-simple-history (following) more.


django-compressor is package that combines and minifies your css and javascript (both files and line snippets) into static files. There are other tools for this but I have never used them since django-compressor satisfies my needs. Although I’ve written about browserify and friends from the node-js world I don’t recommend using such tools in django to combine and minify your javascript and css unless you specifically require them.

It has an online and an offline mode. For the online mode, when a request is done it will check if the compressed file exist and if not it will create it. This may lead to problems with permissions if your application server user cannot write to your static folders and also your users will see exceptions if for some reason you have included a file that cannot be found. For the offline mode, you need to run a management command that will create the static files while deploying the applications - this mode is recommended because any missing files problems etc will be resolved while deploying the app.


django-debug-toolbar: This is a well known package for debugging django apps that is always included in the development configuration of my projects. It has various panels that help you debug your application but, at least for me, the most helpful is the one that displays you all SQL queries that are executed for a page load. Because of how the django orm is working it will go on and follow all relations something that will lead to hundreds of queries. For example, let’s say that you have simple Book model with a foreign key to an Author model that has N instances in your database. If you do a Book.objects.all() and want display the author name for each book in a template then you’ll always do N+1 queries to the database! This is really easy to miss because in the django you’ll just do {% for book in books %}{{ book.name}}, {{ book.author.name }}{% endif %} — however the {{ book.author.name }} will go on and do an extra SQL query!!! Such cases are easily resolved by using select_related (and prefetch_related) but you must be sure to use select_related for all your queries (and if you add some extra things to your template you must remember to also add them to your select_related clause for the query).

So, what I recommend before going to production is to visit all your pages using django-debug-toolbar and take a quick look at the number of SQL queries. If you see something that does not make sense (for example you see more than 10 queries) then you’ll need to think about the problem I just mentioned. Please notice that this, at least for me, is not premature optimization - this is not actually optimization! This is about writing correct code. Let’s suppose that you could not use the django orm anymore and you had to use plain old SQL queries. Would you write SELECT * FROM books and then for each row do another SELECT * FROM authors WHERE id=? passing the author of each book or do only select * from books b LEFT JOIN authors a on b.author_id = a.id?

Packages I use when I need their functionality

The packages following are also essential to me but only when I need their functionality. I don’t use them in all my projects but, when I need the capabilities they offer then I will use these packages (and not some others). For example, if I need to render PDFs in my applications then I will use the xhtml2pdf, if I need to login through LDAP I will use django-auth-ldap etc.


xhtml2pdf is the package I use for creating PDF’s with django as I’ve alreadly discussed in the PDFs in Django article (this is not a django-specific package like most others I discuss here but it plays really good with django). You create a normal django template, add some styling to it and dump it to html. Notice that there’s a django-xhtml2pdf project but has not been recently updated and after all as you can see in my article it is easy to just call xhtml2pdf directly. The xhtml2pdf library is actually a wrapper around the excellent reportlab library which does the low-level pdf output.

Notice that the xhtml2pdf library had some maintenance problems (that’s why some people are suggesting other PDF solutions like WeasyPrint) however they seem to have been fixed now. Also, I have found out that, at least for my needs (using Windows as my development environment), other soltuons are much inferior to xhtml2pdf. I urge you to try xhtml2pdf first and only if you find that it does not cover your needs (and have asked me about your problem) try the other solutions.


django-auth-ldap is the package you’ll want to use if your organization uses LDAP (or Active Directory) and you want to use it for logging in. Just configure your LDAP server settings, add the ldap authenticator and you’ll be ready to go. Please notice that this package is a django wrapper of the python-ldap package which actually provides the LDAP connection.


django-localflavor offers useful stuff for various countries, mainly form fields with the correct validation and lists of choices. For example, for my country (Greece) you’ll get a GRPhoneNumberField, a GRPostalCodeField and a GRTaxNumberCodeField. Use it instead of re-implementing the behavior.


django-constance is a simple package that enables you to add quick-configurable settings in your application. To change the settings.py file you need to edit the source and restart the application - for most installations this is a full re-deployment of the application. Fully re-deploying the app just to change a setting is not very good practice (depending on the setting of course but if it is a business setting it usually should be done by business users and not by administrators).

That’s where django-constance comes to help you. You can define some extra settings which can be changed through the django admin and their new value will be available immediately. Also you can configure where these settings will be saved. One option is the database but this is not recommended - instead you can use redis so that the settings values will be available much quicker!


django-rq is a django wrapper for the rq library. I use it when I need asynchronous tasks (which is on almost all of my projects). More info can be found on the two articles I have writtten about django rq (asynchronous tasks in django and django-rq redux).


One of the least known packages from those I discuss here, django-rules-light is one of the most useful when is needed. This package allows you to define complex rules for doing actions on model instances. Each rule is a function that gets the user that wants to do the action and the object that the user wants to action on. The function returns True or False to allow or not allow the action. You can then use these in both your code to programatically check if the user can do the the action and your templates to decide what buttons and options you will display. There are also various helper methods for CBVs that make everything easier.

To properly understand the value of django-rules-light you need to have some more complex than usual action rules. For example if your actions for an object are view / edit and all your users can view and edit their own objects then you don’t really need this package. However, if your administrators can view all objects and your object can be finalized so no changes are allowed unless an administrator tries to change it then you’ll greatly benefit from using it!


django-generic-scaffold is a package I have created that can be used to quickly (and DRYly) create CRUD CBVs for your models. I usually don’t want to give access to the django-admin to non-technical users however sometimes I want to quickly create the required CBVs for them (list, detail, create, edit delete). Using django-generic-scaffold you can just create a scaffold which is related with a Model and all the views will be automagically created - you only need to link them to your urls.py. The created CBVs are fully configurable by adding extra mixins or even changing the parent class of each CBV.

Notice that this package does not create any source files - instead all CBVs are created on-the-fly using type. For example, to create CRUD CBVs for a Book model you’ll do this in scaffolding.py:

class BookCrudManager(CrudManager):
    model = models.Book
    prefix = 'books'

and then in your urls.py you’ll just append the generated urls to your url list:

book_crud = BookCrudManager()
urlpatterns += book_crud.get_url_patterns()

Now you can visit the corresponding views (for example /books or /bookscreate - depends on the prefix) to add/view/edit etc your books!


django-taggit is the library you’ll want to use if you have to use tags with your models. A tag is a synonym for keyword, i.e adding some words/phrases to your instances that are used to categorise and desribe em. The relation between your to-be-tagged-model and your tags is many to many. To use it, you just add tags = TaggableManager() to your model and you are ready to go! Of course it will need some more configuration to be included in django admin and django forms but thankfully, autocomplete-lights can be integrated with django-taggit!


django-sendfile is a very important - at least to - me library. Sometimes, user uploaded files (media in django) should not be visible to all users so you’ll need to implement some access control through your django app. However, it is important to not serve these media files through your application server (uwsi, gunicorn etc) but use a web server (nginx, apache ect) for serving them. This is needed because your application server’s purpose is not serving files from the disk - keep in mind that the application server usually has a specified amount of workers (usually analogous to the number of CPUs of your server, for example 4 workers ) - think what will happen if some large media files are server through these workers to users with a slow connection! With 4 such concurrent connections your application won’t be able to serve any other content!

So this package (along with the support of X-Sendfile from the web servers) helps you fulfill the above requirements: It allowes you to check permissions to your media through your django application but then offload the serving of your media files to the web server. More info about django-sendfile can be found on this SO answer but with a few words, with django-sendfile you create a view that checks if a file is allowed to be served and, if yes, instruct the web server to actually serve that file by appending a specific header to the response.


django-reversion-compare is an addon to django-reversion. It allows you to compare two different versions of a model instance and highlights their differences using various smart algorithms (so if you have a long text field you won’t only see that these are different but you’ll also see where exactly they differ, with output similar to the one you get when using diff).


django-simple-history has similar capabilities with django-reversion - (auditing and keeping versions of models) with a very important difference: While django-reversion keeps a JSON representation of each version in the database (making querying very difficult), django-simple-history creates an extra, history table for each model instance you want to track and adds each change as a new row to that table. As can be understood this will make the history table really huge but has the advantage that you can easily query for old values. I usually use django-reversion unless I know that I will need the history querying.


django-import-export can be used to enchance your models with mass import and export capabilities (from example from/to CSV). You will add an ModelResource class that describes (and configures) how your Model should be imported/exported. The ModelResource class can then be easily used in your views and, more importantly, it is integrated to the django-admin. I have to confess that I have not used django-import-export for importing data because I prefer implementing my own views for that (to have better control over the whole process and because the data I usually need to import does not usually map 1 to 1 with a model but I need to create more model instances etc). However I am using the export capabilities of django-import-export in various projects with great success, especially the admin integration which easily fulfills the exporting data capabilities of most users.


Beyond django-forms, django supports a feature called Formsets which allows you to add/edit multiple object instances in a single view (by displaying all instances one after the other). The classic request/response cycle of django is preserved in Formsets, so your form instances will be submitted all together when you submit the form. The logic extension to the Formset is the ModelFormset i.e each form in a Formset is a ModelForm and InlineFormSet where you have a Parent model that has a bunch of children and you are editing the Parent and its children in a single Form. For example, you have a School and a Student model where each Student has a ForeignKey to School. The usual case would be to edit the Student model and select her school (through a drop down or even-better if you use django-autocomplete-light through a proper autocompelte widget). However, you may for some reason want to edit the School and display (and edit) the list of its Students — that’s where you’ll use an InlineFormSet!

The above features (Formsets, Modelformsets and Inlineformsets) are not supported natively by django CBVs — that’s where django-extra-views comes to the foreground. You can use the corresponding CBVs of django-extra-views to support the multiple-form workflows described above. Using these CBVs are more or less similar to using the good-old django FormView.


easy-thumbnails is a great library if you want to support thumbnails. Actually, thumbnails is not 100% correct - this package can be used to generate and manage various versions of your original images, for example you may have a small version of the image that will be used as a thumbnail, a larger version that will be used in a gallery-carousel view and an even larger version (but not the original one which could be huge) that will be used when the user clicks on the gallery to view a larger version of the image. To do that you define the image configurations you support in your settings.py and then you have access to your thumbnails both in your templates and in your views. Notice that a specific thumbnail congfiguration for an image will be created only once since the generated images are saved so each thumbnail will be generated on the first request it contains it and will be reused in the following such requests.


django-rest-framework is definitely the best package for creating web APIs with django. I don’t recommend using it if you want to create a quick JSON search API (take a look at django non-HTML responses) but if you want to go the SPA way or if you want to create multiple APIs for various models then this is the way to go. Integrating it to your project will need some effort (that’s why I don’t recommend it for quick and dirty APIs) because you’ll need to create a serializers.py which will define the serializers (more or less the fields) for the models you’ll want to expose through your API and then create the views (or the viewsets which are families of views for example list, detail, delete, update, create) and add them to your urls.py. You’ll also need to configure authentication, authorization and probably filtering and pagination. This may seem like a lot of work but the result are excellent - you’ll get a full REST API supporting create, list, detail, update, delete for any complex configuration of your models. You can take a look at a sample application in my React tutorial repository (yes this is a repository that has a tutorial for React and friends but the back-end is in django and django-rest-framework).

django-rest-framework integrates nicely with django-filter (mentioned above) to re-use the filters you have created for your model listings in your REST APIs - DRY at its best!


django-waffle is described as a feature flipper. What does this mean? Let’s say that you want to control at will when a specific view will be enabled - this is the library you’ll want to use. Or you have developed a new feature and you want to give access to it only on a subset of users for a pilot run of the feature - once again you should use django flipper. It offers a nice admin interface where you can configure the flags that will be used for the various feature enabling/disabling (and if they are active or not) and various template tags and functions that you can use to test if the features should be activated or not.


django-allauth should be used in two cases: When you want a better user registration workflow than the default (non-existant) one or you want to integrate your application with an external OAuth provider (i.e allow your users to login through their facebook, google, twitter, github etc accounts). I have mainly used this package for the first case and I can confirm that it works great and you can create as complex flows as you want (for example, in one of my projects I have the following user registration-activation flow: A user registers using a custom form and using his email as username, he receives an email with a confirmation link, after he has confirmed his email he receivs a custom message to wait for his account activation and the administrators of the application are notified, the administrators enable the new user’s account after checking some things and only then he’ll be able to log-in). One thing that must be noticed about django-allauth is that it (in my opition) does not have very good documentation but there are lots of answers about django-allauth in stackoverflow and the source code is very clear so you can always use the source as documentation for this package.


The django-modeltranslation library is the library I recommend for when you want to have translations to your models. To use it you add a translation.py file where you declare the models and their fields that should be translated. The, depending on which languages you have configured in your settings.py after you run makemigrations and migrate you’ll see that django-modeltranslation will have included extra fields to the database, each one with the corresponding language name (for example if you have added a field name to the translations and have english and greek as language, django-modeltranslation will add the fields name_en and name_el to your table). You can the edit the i18n fields (using forms or the django admin) and depending on the current language of your site when you use name you’ll get either name_el or name_en.


If for some reason you don’t want to do django-crispy-forms, or you have a form in which you want to do a specific layout change but without fully implementing the FormHelper then you can actually render the form in HTML and output the fields one by one. One thing that cannot be done though is passing custom options to the rendered form field. When you do a {{ form.field }} to your template django will render the form field using its default options - yes this can be overriden using custom widgets but I don’t recommend it for example if you only want to add a class to the rendered <input>!

Instead, you can use django-widget-tweaks to pass some specific class names or attributes to the rendered form fields - so if you use {{ form.field|add_class:"myclass" }} the rendered <input> will have a myclass css class.


Use django-simple-captcha to add (configurable) captchas to your django forms. This is a very simple package that does not have any requirements beyond the Pillow library for the captcha image generator. The generated captchas are simple images with some added noise so it won’t integrate reCAPTCHA with which you may be more familiar. I deliberatly propose this package for captchas so you won’t need to integrate with Google services.


wagtail is a great django CMS. I use it when I need to create a CMS or I need to add CMS like capabilities to a project. It has many capabilities, too many to be listed here. I urge you to try it if you need a CMS!


The above packages should cover most of your django needs. I have listed only packages with good documentation, that have been recently updated and work with new django versions and should be fairly easy to integrate with your projects. If you need anything more or want to take a general look at some of the packages that have are availablie I recommend starting with the django packages site.

One important thing to notice here is that some of the above packages are not really complex and their functionality can be re-implemented by you in a couple of hours. For example, you could replicate the functionality of django-constance by adding a config dict and a couple of methods (and template tags) of storing and retrieving the keys of that dict with redis. Or add some custom clean methods to your forms instead of using the form fields from django-localflavor. Also, some of these packages have similar functionality and can be used (along with a little custom code) to replicate the functionality of other packages, for exmaple instead of using django-waffle you could use django-constance to configure if the features should be enabled or disabled and django-rules-light to control if the users have access to the feature. Also, you could probably use django-waffle for access control, i.e allow only admins to access a specific views.

Please don’t do this. This violates DRY and violates being disciplined. Each package has its purpose and being DRY means that you use it for its purpose, not re-implementing it and not re-using it for other purposes. When somebody (or you after some months) sees that package in requirements or INSTALLED_APPS he will conclude that you are using it for its intented purpose and thank you because you have saved him some time - please don’t make him waste his time by needing to read your source code to understand any smart tricks or reinventing the wheel.