Advanced Django File Handling

Advanced Django File Handling

Modern Django's file handling capabilities go well beyond what's covered in the tutorial. By customizing the handlers that Django uses, you can do things pretty much any way you want.

Static versus media files

Django divides the files your web site is serving unchanged (as opposed to content delivered by your Django views) into two types.

• "Static" files are files provided by you, the website developer. For example, these could be JavaScript and CSS files, HTML files for static pages, image and font files used to make your pages look nicer, sample files for users to download, etc. Static files are often stored in your version control system alongside your code.
• "Media" files are files provided by users of the site, uploaded to and stored by the site, and possibly later served to site users. These can include uploaded pictures, avatars, user files, etc. These files don't exist until users start using the site.

Two jobs of a Django storage class

Both kinds of files are managed by code in Django storage classes. By configuring Django to use different storage classes, you can change how the files are managed.

A storage class has two jobs:

• Accept a name and a blob of data from Django and store the data under that name.
• Accept a name of a previously-stored blob of data, and return a URL that when accessed will return that blob of data.

The beauty of this system is that our static and media files don't even need to be stored as files. As long as the storage class can do those two things, it'll all work.

Django settings related to storing files

You tell Django which storage classes to use with two settings:

• STATICFILES_STORAGE manages static files.
• DEFAULT_FILE_STORAGE¹ manages media files.

The default value for STATICFILES_STORAGE is django.contrib.staticfiles.storage.StaticFilesStorage, which saves files in the local file system in the directory specified by the STATIC_ROOT setting, and generates URLs by simply concatenating STATIC_URL and the filename.

The default value for DEFAULT_FILE_STORAGE is django.core.files.storage.FileSystemStorage, which saves files in the local file system in the directory specified by the MEDIA_ROOT setting, and generates URLs by simply concatenating MEDIA_URL and the filename.

Runserver

Given all this, you'd naturally conclude that if you've changed STATICFILES_STORAGE and DEFAULT_FILE_STORAGE to storage classes that don't look at the STATIC_URL, STATIC_ROOT, MEDIA_URL, and MEDIA_ROOT settings, you don't have to set those at all.

However, if you remove them from your settings, and try to use runserver, you'll get errors. It turns out that when running with runserver, django.contrib.staticfiles.storage.StaticFilesStorage is not the only code that looks at STATIC_URL, STATIC_ROOT, MEDIA_URL, and MEDIA_ROOT.

This is rarely a problem in practice. runserver should only be used for local development, and when working locally, you'll most likely just use the default storage classes for simplicity, so you'll be configuring those settings anyway. And if you want to run locally in the exact same way as your deployed site, possibly using other storage classes, then you should be running Django the same way you do when deployed as well, and not using runserver.

But you might run into this in weird cases, or just be curious. Here's what's going on.

When staticfiles is installed, it provides its own version of the runserver command that arranges to serve static files for URLs that start with STATIC_URL, looking for those files under STATIC_ROOT. (In other words, it's bypassing the static files storage class.) Therefore, STATIC_URL and STATIC_ROOT need to be valid if you need that to work. Also, when initialized, it does some sanity checks on all four variables (STATIC_URL, STATIC_ROOT, MEDIA_URL, and MEDIA_ROOT), and the checks assume those variables' standard roles, even if the file storage classes have been changed in STATICFILES_STORAGE and/or DEFAULT_FILE_STORAGE.

If you really need to use runserver with some other static file storage class, you can either configure those four settings to something that'll make runserver happy, or use the --nostatic option with runserver to tell it not to try to serve static files, and then it won't look at those settings at startup.

Using media files in Django

Media files are typically managed in Python using FileField and ImageField fields on models. As far as your database is concerned, these are just char columns storing relative paths, but the fields wrap that with code to use the media file storage class.

In a template, you use the url attribute on the file or image field to get a URL for the underlying file.

For example, if user.avatar is an ImageField on your user model, then

<img src="{{ user.avatar.url }}">

would embed the user's avatar image in the web page.

The default storage class for media, django.core.files.storage.FileSystemStorage, saves files to a path inside the local directory named by MEDIA_ROOT, under a subdirectory named by the field's upload_to value. When the file's url attribute is accessed, it returns the value of MEDIA_URL, prepended to the file's path inside MEDIA_ROOT.

An example might help. Suppose we have these settings:

MEDIA_ROOT = '/var/media/'
MEDIA_URL = '/media/'

and this is part of our user model:

avatar = models.ImageField(upload_to='avatars')

When a user uploads an avatar image, it might be saved as /var/media/avatars/12345.png. That's MEDIA_ROOT, plus the value of upload_to for this field, plus a filename (which is typically the filename provided by the upload, but not always).

Then <img src="{{ user.avatar.url }}"> would expand to <img src="/media/avatars/12345.png">. That's MEDIA_URL plus upload_to plus the filename.

Now suppose we've changed DEFAULT_FILE_STORAGE to some other storage class. Maybe the storage class saves the media files as attachments to email messages on an IMAP server - Django doesn't care.

When 12345.png is uploaded to our ImageField, Django asks the storage class to save the contents as avatars/12345.png. If there's already something stored under that name, Django will change the name to come up with something unique. Django stores the resulting filename in the database field. And that's all Django cares about.

Now, what happens when we put <img src="{{ user.avatar.url }}"> in our template? Django will retrieve the filename from the database field, pass that filename (maybe avatars/12345.png) to the storage class, and ask it to return a URL that, when the user's browser requests it, will return the contents of avatars/12345.png. Django doesn't know what that URL will be, and doesn't have to.

For more on what happens between the user submitting a form with attached files and Django passing bits to a storage class to be saved, you can read the Django docs about File Uploads.

Using Static Files in Django

Remember that static file handling is controlled by the class specified in the settings STATICFILES_STORAGE.

Media files are loaded into storage when users upload files. Static files are provided by us, the website developers, and so they can be loaded into storage beforehand.

The collectstatic management command finds all your static files, and saves each one, using the path relative to the static directory where it was found, into the static files storage.²

By default, collectstatic looks for all the files inside static directories in the apps in INSTALLED_APPS, but where it looks is configurable - see the collectstatic docs.

So if you have a file myapp/static/js/stuff.js, collectstatic will find it when it looks in myapp/static, and save it in static files storage as js/stuff.js.

You would most commonly access static files from templates, by loading the static templatetags library and using the static template tag. For our example, you'd ask Django to give you the URL where the user's browser can access js/stuff.js by using {% static 'js/stuff.js' %} in your template. For example, you might write:

{% load 'static' %}
<script src="{% static 'js/stuff.js' %}"></script>

If you're using the default storage class and STATIC_URL is set to http://example.com/, then that would result in:

<script src="http://example.com/js/stuff.js"></script>

Maybe then you deploy it, and are using some fancy storage class that knows how to use a CDN, resulting in:

<script src="http://23487234.niftycdn.com/239487/230498234/js/stuff.js"></script>

Other neat tricks can be played here. A storage class could minimize your CSS and JavaScript, compile your LESS or SASS files to CSS, and so forth, and then provide a URL that refers to the optimized version of the static file rather than the one originally saved. That's the basis for useful packages like django-pipeline.

If you’re looking for more Django tips, we have plenty on our blog.