Deployment is usually a tedious process with lots of tinkering until everything is setup just right. We deploy quite a few Django sites on a regular basis here at Caktus and still do tinkering, but we’ve attempted to functionalize some of the core tasks to ease the process. I’ve put together a basic example that outlines local and remote environment setup. This is a simplified example and just one of many ways to deploy a Django project (I learned a lot from Jacob Kaplan-Moss’ django-deployment-workshop), so I encourage you to browse around the Django community to learn more.
The entire source for this example project can be found in the caktus-deployment Bitbucket repository.
Local Development Environment
The project directory is organized like so:
caktus_website/
__init__.py
apache/
staging.conf -- staging Apache conf
staging.wsgi -- staging wsgi file
blog/
bootstrap.py -- bootstrap local environment
fabfile.py -- manage remote environments with fabric
local_settings.py
manage.py
media/
requirements/
apps.txt -- pip requirements file
settings.py
settings_staging.py -- staging settings file
urls.pyTo setup a local development environment, we’ll create a virtual environment and run bootstrap.py, which is just a simple script that automates installing Python dependencies using pip:
1
2
3
4
5
6
7
8
| if "VIRTUAL_ENV" not in os.environ:
sys.stderr.write("$VIRTUAL_ENV not found.\n\n")
parser.print_usage()
sys.exit(-1)
virtualenv = os.environ["VIRTUAL_ENV"]
file_path = os.path.dirname(__file__)
subprocess.call(["pip", "install", "-E", virtualenv, "--requirement",
os.path.join(file_path, "requirements/apps.txt")]) |
bootstrap.py uses requirements/apps.txt (a pip requirements file), so you can source anything off of PyPI as well as mercurial, git, and SVN repositories that include setup.py files. In this example, django’s SVN is the only dependency in apps.txt:
-e svn+http://code.djangoproject.com/svn/django/branches/releases/1.1.X#egg=django
bootstrap.py must be run within virtual environment, so let’s create a new virtualenv (I recommend using virtualenvwrapper) and then run bootstrap.py to install the dependencies:
copelco@montgomery:~/caktus_website$ mkvirtualenv --distribute caktus
(caktus)copelco@montgomery:~/caktus_website$ ./bootstrap.py
Now that our environment is setup (and Django is on the python path), we can run normal Django management commands:
(caktus)copelco@montgomery:~/caktus_website$ ./manage.py syncdb --settings=caktus_website.local_settings
(caktus)copelco@montgomery:~/caktus_website$ ./manage.py runserver --settings=caktus_website.local_settings
Great! That’s it for our local setup, let’s look into deploying the project to a staging server.
Deployment and Remote Management
To help provision the remote server environment (in this case Ubuntu 9.10), we’ll use fabric. fabric allows you to streamline deployment by functionalizing common tasks in Python. I’ve created an example fabfile.py to help bootstrap and deploy the project:
(caktus)copelco@montgomery:~/caktus_website$ fab --list
Available commands:
apache_reload reload Apache on remote host
apache_restart restart Apache on remote host
bootstrap initialize remote host environment (virtualenv, dep...
configtest test Apache configuration
create_virtualenv setup virtualenv on remote host
deploy rsync code to remote host
production use production environment on remote host
staging use staging environment on remote host
symlink_django create symbolic link so Apache can serve django adm...
touch touch wsgi file to trigger reload
update_apache_conf upload apache configuration to remote host
update_requirements update external dependencies on remote hostThe fabfile splits the deployment process into discrete steps of 1) virtual environment creation, 2) code transfer, and 3) updating the Python dependencies. The bootstrap command wraps everything together, including initial directory creation, so you can setup the server quickly:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
| def bootstrap():
""" initialize remote host environment (virtualenv, deploy, update) """
require('root', provided_by=('staging', 'production'))
run('mkdir -p %(root)s' % env)
run('mkdir -p %s' % os.path.join(env.home, 'www', 'log'))
create_virtualenv()
deploy()
update_requirements()
def create_virtualenv():
""" setup virtualenv on remote host """
require('virtualenv_root', provided_by=('staging', 'production'))
args = '--clear --distribute'
run('virtualenv %s %s' % (args, env.virtualenv_root))
def deploy():
""" rsync code to remote host """
require('root', provided_by=('staging', 'production'))
if env.environment == 'production':
if not console.confirm('Are you sure you want to deploy production?',
default=False):
utils.abort('Production deployment aborted.')
extra_opts = '--omit-dir-times'
rsync_project(
env.root,
exclude=RSYNC_EXCLUDE,
delete=True,
extra_opts=extra_opts,
)
touch()
def update_requirements():
""" update external dependencies on remote host """
require('code_root', provided_by=('staging', 'production'))
requirements = os.path.join(env.code_root, 'requirements')
with cd(requirements):
cmd = ['pip install']
cmd += ['-E %(virtualenv_root)s' % env]
cmd += ['--requirement %s' % os.path.join(requirements, 'apps.txt')]
run(' '.join(cmd)) |
To bootstrap the staging environment, run:
(caktus)copelco@montgomery:~/caktus_website$ fab staging bootstrap
This will run a few commands over SSH and rsync the project directory to a specific location on the staging server. Using rsync is just one of many ways to transfer code to the server, such as pulling code from a remote repository. The “deploy” fabfile can be modified to perform almost any transfer task. Once the bootstrap process is complete, the directory structure will look like so:
home/
caktus/
www/
staging/
env/ -- virtual environment
bin/
include/
lib/ -- contains site-packages
source/ -- contains django src
caktus_website/
...
apache/
manage.py
requirements/
...Now SSH to the server and run syncdb within the newly created virtual environment:
caktus@pike:~/www/staging/caktus_website$ source ../env/bin/activate
(env)caktus@pike:~/www/staging/caktus_website$ ./manage.py syncdb --settings=caktus_website.settings_staging
The staging setting’s file is setup to use sqlite3 to simplify this deployment example. In practice we use PostgreSQL in our production environments, but database setup is for another blog post! To get Apache configured using mod_wsgi, we’ll point the apache configuration to the staging.wsgi file using the WSGIScriptAlias directive. Here’s an example Apache configuration to get a barebones Django environment up and running:
<VirtualHost *:80>
WSGIScriptReloading On
WSGIReloadMechanism Process
WSGIDaemonProcess caktus_website-staging
WSGIProcessGroup caktus_website-staging
WSGIApplicationGroup caktus_website-staging
WSGIPassAuthorization On
WSGIScriptAlias / /home/caktus/www/staging/caktus_website/apache/staging.wsgi/
<Location "/">
Order Allow,Deny
Allow from all
</Location>
<Location "/media">
SetHandler None
</Location>
Alias /media /home/caktus/www/staging/caktus_website/media
<Location "/admin-media">
SetHandler None
</Location>
Alias /admin-media /home/caktus/www/staging/caktus_website/media/admin
ErrorLog /home/caktus/www/log/error.log
LogLevel info
CustomLog /home/caktus/www/log/access.log combined
</VirtualHost>We’ll use Apache to serve static media (both local and admin media) and direct everything else to the Django instance through mod_wsgi. In order for the wsgi instance to be aware of our environment and project directory, we need to add the virtual environment’s site-packages directory, the project directory to the python path, and tell Django which settings file to use by setting the DJANGO_SETTINGS_MODULE environment variable:
1
2
3
4
5
6
7
8
9
10
11
12
| import os
import sys
import site
PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
site_packages = os.path.join(PROJECT_ROOT, 'env/lib/python2.6/site-packages')
site.addsitedir(os.path.abspath(site_packages))
sys.path.insert(0, PROJECT_ROOT)
os.environ['DJANGO_SETTINGS_MODULE'] = 'caktus_website.settings_staging'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler() |
Now just upload the staging apache configuration and reload apache:
(caktus)copelco@montgomery:~/caktus_website$ fab staging update_apache_conf
That’s it! The site should be up and running on your server’s public IP. If you run into any trouble (like a 500 Internal Server Error), just tail the Apache error.log, it’ll usually point you in the right direction.
