Caktus builds custom web applications for local and remote clients using a variety of open-source technologies. We are a small team based in the Chapel Hill/Carrboro area of North Carolina (currently residing in Carrboro Creative Coworking). We believe in face-to-face contact, both with clients and amongst ourselves, and employ agile development techniques that emphasize teamwork and collaboration. We encourage you to meet the team and learn more about what we do.

We’re looking for two experienced Python and/or Django web developers who enjoy working on a team and are excited to work on new projects. We have a preference for local candidates, but will consider all submissions. Your work will involve creating and integrating Django apps, working on existing Django projects, deployment, and database work.

You will be working in Linux (Debian-flavor) production environments with Apache and WSGI. Python and relational database experience is required. Django experience is a (big) plus. HTML/CSS and JavaScript experience are also a must, and jQuery is a plus.

If you’re interested in one of these positions, please send us your resume, some sample Python code that you wrote, and links to any open-source projects you’ve contributed to. We’re looking forward to meeting you!

DjangoCon is the annual gathering of software developers who use the open source, Python-based Django web framework. We use the framework every day here at Caktus to create custom web applications and dynamic, content-rich web sites. Additionally, starting this year, we’ve put some of that knowledge to use extending and developing applications for the RapidSMS framework. For more information about why we use Django and think it’s so great, check out our blog post titled Why Caktus Uses Django.

This year, the conference is being held again the week of September 6th in the beautiful city of Portland, Oregon. We’ve grown a little since last year at this time; it looks like 6 Caktus team members—Colin, Alex, Karen, Mark, Mike, and myself—will be attending the conference. We’re positively thrilled to be going again this year and we hope to see you there!

Caktus is a seasoned team of web developers that creates interactive, content-rich sites and applications with the Django web framework. We put a strong emphasis on best practices, employ an agile method, and also actively participate in the Django development community.

For more information about Caktus and our team, check out our newly updated team page!

In addition to Django web apps we’ve been building for years here at Caktus, we’re now offering a related service in the mobile health development. Using RapidSMS, a communications framework built on Django, we build applications that add an SMS (text message) component to the standard Django stack. This allows non-standard users, such as clinic and community health workers, to interact with the system. Coupling high-tech and low-tech in this way lets us help remedy communication problems in places where Internet (and even power) are not widely available.

Check out the new site, and please let us know if you have any questions or comments!

The entire source for this example project can be found in the caktus-deployment Bitbucket repository.

Local Development Environment

The project directory is organized like so:

caktus_website/
    __init__.py
    apache/
        staging.conf    -- staging Apache conf
        staging.wsgi    -- staging wsgi file
    blog/
    bootstrap.py        -- bootstrap local environment
    fabfile.py          -- manage remote environments with fabric
    local_settings.py
    manage.py
    media/
    requirements/
        apps.txt        -- pip requirements file
    settings.py
    settings_staging.py -- staging settings file
    urls.py

To setup a local development environment, we’ll create a virtual environment and run bootstrap.py, which is just a simple script that automates installing Python dependencies using pip:

1
2
3
4
5
6
7
8

if "VIRTUAL_ENV" not in os.environ:
    sys.stderr.write("$VIRTUAL_ENV not found.\n\n")
    parser.print_usage()
    sys.exit(-1)
virtualenv = os.environ["VIRTUAL_ENV"]
file_path = os.path.dirname(__file__)
subprocess.call(["pip", "install", "-E", virtualenv, "--requirement",
                 os.path.join(file_path, "requirements/apps.txt")])

bootstrap.py uses requirements/apps.txt (a pip requirements file), so you can source anything off of PyPI as well as mercurial, git, and SVN repositories that include setup.py files. In this example, django’s SVN is the only dependency in apps.txt:

-e svn+http://code.djangoproject.com/svn/django/branches/releases/1.1.X#egg=django

bootstrap.py must be run within virtual environment, so let’s create a new virtualenv (I recommend using virtualenvwrapper) and then run bootstrap.py to install the dependencies:

copelco@montgomery:~/caktus_website$ mkvirtualenv --distribute caktus
(caktus)copelco@montgomery:~/caktus_website$ ./bootstrap.py

Now that our environment is setup (and Django is on the python path), we can run normal Django management commands:

(caktus)copelco@montgomery:~/caktus_website$ ./manage.py syncdb --settings=caktus_website.local_settings
(caktus)copelco@montgomery:~/caktus_website$ ./manage.py runserver --settings=caktus_website.local_settings

Great! That’s it for our local setup, let’s look into deploying the project to a staging server.

Deployment and Remote Management

To help provision the remote server environment (in this case Ubuntu 9.10), we’ll use fabric. fabric allows you to streamline deployment by functionalizing common tasks in Python. I’ve created an example fabfile.py to help bootstrap and deploy the project:

(caktus)copelco@montgomery:~/caktus_website$ fab --list
Available commands:
 
    apache_reload        reload Apache on remote host
    apache_restart       restart Apache on remote host
    bootstrap            initialize remote host environment (virtualenv, dep...
    configtest           test Apache configuration
    create_virtualenv    setup virtualenv on remote host
    deploy               rsync code to remote host
    production           use production environment on remote host
    staging              use staging environment on remote host
    symlink_django       create symbolic link so Apache can serve django adm...
    touch                touch wsgi file to trigger reload
    update_apache_conf   upload apache configuration to remote host
    update_requirements  update external dependencies on remote host

The fabfile splits the deployment process into discrete steps of 1) virtual environment creation, 2) code transfer, and 3) updating the Python dependencies. The bootstrap command wraps everything together, including initial directory creation, so you can setup the server quickly:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

def bootstrap():
    """ initialize remote host environment (virtualenv, deploy, update) """
    require('root', provided_by=('staging', 'production'))
    run('mkdir -p %(root)s' % env)
    run('mkdir -p %s' % os.path.join(env.home, 'www', 'log'))
    create_virtualenv()
    deploy()
    update_requirements()
 
 
def create_virtualenv():
    """ setup virtualenv on remote host """
    require('virtualenv_root', provided_by=('staging', 'production'))
    args = '--clear --distribute'
    run('virtualenv %s %s' % (args, env.virtualenv_root))
 
 
def deploy():
    """ rsync code to remote host """
    require('root', provided_by=('staging', 'production'))
    if env.environment == 'production':
        if not console.confirm('Are you sure you want to deploy production?',
                               default=False):
            utils.abort('Production deployment aborted.')
    extra_opts = '--omit-dir-times'
    rsync_project(
        env.root,
        exclude=RSYNC_EXCLUDE,
        delete=True,
        extra_opts=extra_opts,
    )
    touch()
 
 
def update_requirements():
    """ update external dependencies on remote host """
    require('code_root', provided_by=('staging', 'production'))
    requirements = os.path.join(env.code_root, 'requirements')
    with cd(requirements):
        cmd = ['pip install']
        cmd += ['-E %(virtualenv_root)s' % env]
        cmd += ['--requirement %s' % os.path.join(requirements, 'apps.txt')]
        run(' '.join(cmd))

To bootstrap the staging environment, run:

(caktus)copelco@montgomery:~/caktus_website$ fab staging bootstrap

This will run a few commands over SSH and rsync the project directory to a specific location on the staging server. Using rsync is just one of many ways to transfer code to the server, such as pulling code from a remote repository. The “deploy” fabfile can be modified to perform almost any transfer task. Once the bootstrap process is complete, the directory structure will look like so:

home/
    caktus/
        www/
            staging/
                env/               -- virtual environment
                    bin/
                    include/
                    lib/           -- contains site-packages
                    source/        -- contains django src
                caktus_website/
                    ...
                    apache/
                    manage.py
                    requirements/
                    ...

Now SSH to the server and run syncdb within the newly created virtual environment:

caktus@pike:~/www/staging/caktus_website$ source ../env/bin/activate
(env)caktus@pike:~/www/staging/caktus_website$ ./manage.py syncdb --settings=caktus_website.settings_staging

The staging setting’s file is setup to use sqlite3 to simplify this deployment example. In practice we use PostgreSQL in our production environments, but database setup is for another blog post! To get Apache configured using mod_wsgi, we’ll point the apache configuration to the staging.wsgi file using the WSGIScriptAlias directive. Here’s an example Apache configuration to get a barebones Django environment up and running:

<VirtualHost *:80>
    WSGIScriptReloading On
    WSGIReloadMechanism Process
    WSGIDaemonProcess caktus_website-staging
    WSGIProcessGroup caktus_website-staging
    WSGIApplicationGroup caktus_website-staging
    WSGIPassAuthorization On
 
    WSGIScriptAlias / /home/caktus/www/staging/caktus_website/apache/staging.wsgi/
 
    <Location "/">
        Order Allow,Deny
        Allow from all
    </Location>
 
    <Location "/media">
        SetHandler None
    </Location>
 
    Alias /media /home/caktus/www/staging/caktus_website/media
 
    <Location "/admin-media">
        SetHandler None
    </Location>
 
    Alias /admin-media /home/caktus/www/staging/caktus_website/media/admin
 
    ErrorLog /home/caktus/www/log/error.log
    LogLevel info
    CustomLog /home/caktus/www/log/access.log combined
</VirtualHost>

We’ll use Apache to serve static media (both local and admin media) and direct everything else to the Django instance through mod_wsgi. In order for the wsgi instance to be aware of our environment and project directory, we need to add the virtual environment’s site-packages directory, the project directory to the python path, and tell Django which settings file to use by setting the DJANGO_SETTINGS_MODULE environment variable:

1
2
3
4
5
6
7
8
9
10
11
12

import os
import sys
import site
 
PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
site_packages = os.path.join(PROJECT_ROOT, 'env/lib/python2.6/site-packages')
site.addsitedir(os.path.abspath(site_packages))
sys.path.insert(0, PROJECT_ROOT)
os.environ['DJANGO_SETTINGS_MODULE'] = 'caktus_website.settings_staging'
 
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Now just upload the staging apache configuration and reload apache:

(caktus)copelco@montgomery:~/caktus_website$ fab staging update_apache_conf

That’s it! The site should be up and running on your server’s public IP. If you run into any trouble (like a 500 Internal Server Error), just tail the Apache error.log, it’ll usually point you in the right direction.

We’re proud to annouce that Caktus is hosting another local Django development sprint in the Triangle (Raleigh, Durham, and Chapel Hill/Carrboro) area of North Carolina. The sprint will be held the weekend of March 20th and 21st in Carrboro Creative Coworking, and the purpose of this sprint will be to help push out bug fixes in preparation for the upcoming Django 1.2 release.

If you’re interested in attending, no previous experience contributing to Django is necessary and the sprint will be a great opportunity to start. Work on other open source Django-based projects is welcome too. For more information, check out the corresponding wiki page.

We’ll be there to open the doors at 9am both days. Courtesy of our sponsors there will be free drinks, snacks, and lunch to go around. Hope to see you there!

We wrote a few little apps, django-pagelets, django-treenav, and django-crumbs, each of which manages different pieces of content (little snippets of content, full CMS pages, navigation, and breadcrumbs). All of the apps are available for free under an open source license on Google Code.

Decoupling was a great move for us, and the ability to plug and play any single part of the system is a huge benefit. Sometimes, however, the completely decoupled architecture was a bit of a pain: If we didn’t provide a link from the pagelets app to the treenav app, how would it be possible to edit a page’s corresponding navigation item on its change form in the Django admin interface?

Enter Generic Relations. Using Django’s content types framework, it’s possible to create admin inlines for generic relations with just a few simple lines of code.

In this case, I’ll show how we allowed users to edit a page’s corresponding navigation item in django-pagelets without requiring everyone (i.e., those who don’t need it) to install django-treenav. First, define the generic inline in the admin.py file of the app that contains the model you want to link to:

from django.contrib.contenttypes import generic
class GenericMenuItemInline(generic.GenericStackedInline):
    """
    Add this inline to your admin class to support editing related menu items
    from that model's admin page.
    """
    max_num = 1
    model = treenav.MenuItem

Then, inside the Admin class for the related model in question, dynamically import and add GenericMenuItemInline to the admin’s list of inlines based on whether or not it’s in the project’s INSTALLED_APPS:

from django.conf import settings
class PageAdmin(admin.ModelAdmin):
    # ...
    inlines = [MyOtherInline]
    if 'treenav' in settings.INSTALLED_APPS:
        from treenav.admin import GenericMenuItemInline
        inlines.insert(0, GenericMenuItemInline)

For more information, see the corresponding pagelets admin.py and treenav admin.py. Thanks for reading and don’t hesitate to post comments if you have any questions!

After attending Dr. C. Titus Brown’s “Why not run all your tests all the time? A study of continuous integration systems.” talk at Pycon and seeing Django’s Hudson setup, I figured I’d take a look at Hudson CI.

Installing Hudson and basic setup

Hudson is very easy to setup. I started with a fresh Ubuntu 9.10 install on the smallest Rackspace cloud instance and had it running after a few commands. I followed the Debian setup instructions, which basically consists of:

$ wget -O - http://hudson-ci.org/debian/hudson-ci.org.key | sudo apt-key add -
$ echo "deb http://hudson-ci.org/debian binary/" >> /etc/apt/sources.list
$ apt-get update
$ aptitude install hudson
$ apt-get upgrade

That’s it! It’s already up and running on port 8080 using it’s own web server. Go ahead and pull it up in your browser.

As a test, let’s setup django-crm (a Caktus open-source community project) as our first Hudson job. Click “New Job”, type in a job name, click “Build a free-style software project”, and hit OK. django-crm contains a sample project that we’ll use to run the test suite. On the job configuration page, check Subversion in the Source Code Management section and type in the Repository URL:

Click Save, run the job by clicking “Build Now”, and check out the Console Output:

Started by user anonymous
Checking out a fresh workspace because /var/lib/hudson/jobs/django-crm/workspace/sample_project doesn't exist
Checking out http://django-crm.googlecode.com/svn/trunk/sample_project
A         manage.py
A         site_media
A         site_media/css
A         site_media/css/jquery.autocomplete.css
A         site_media/css/django-contactinfo.css
A         site_media/js
A         site_media/js/jquery-ui-1.7.2.custom.min.js
A         site_media/js/jquery-1.3.2.min.js
A         site_media/js/django-crm.js
A         site_media/js/jquery.autocomplete.min.js
...
Finished: SUCCESS

Cool, now let’s run some tests. Too keep things simple, let’s grab Django and a few dependencies using aptitude:

$ wget http://www.djangoproject.com/download/1.1.1/tarball/
$ tar xzvf Django-1.1.1.tar.gz
$ cd Django-1.1.1
$ sudo python setup.py install
$ aptitude install python-dev python-imaging python-setuptools python-pip

To run the tests, add an “Execute shell” build step in the Build section with this command:

#!/bin/bash -ex
cd sample_project
python manage.py test crm

Run the job again and look for the test results in the console output:

[workspace] $ /bin/sh -xe /tmp/hudson6670261053226891793.sh
+ cd sample_project
+ python manage.py test crm
...
Finished: SUCCESS

XML Test output

To integrate Hudson with the Django test suite, I used unittest-xml-reporting. Just “pip install unittest-xml-reporting” and add the following lines to your settings file:

TEST_RUNNER = 'xmlrunner.extra.djangotestrunner.run_tests'
TEST_OUTPUT_VERBOSE = True
TEST_OUTPUT_DESCRIPTIONS = True
TEST_OUTPUT_DIR = 'xmlrunner'

Then check “Publish JUnit test result report” in the Post-build Actions section and add the path to the test XML output “sample_project/xmlrunner/*.xml”:

Run the job and you should see a new “Test Result” link in the navigation. Now you can view the test results right in your browser window.

Coverage

To add coverage reports, I used Ned Batchelder’s coverage.py (pip install coverage). Navigate to Hudson’s plugin manager (Hudson -> Manage Hudson -> Manage Plugins), install the Cobertura Plugin, and restart Hudson when prompted. Then modify your shell script like so:

#!/bin/bash -ex
cd sample_project
coverage run manage.py test crm
coverage xml --omit=/usr/

This will generate an XML coverage report in the working directory, so we just need to tell Hudson where to look for it. Check “Publish Cobertura Coverage Report” in the Post-build Actions section and enter the path to the report:

Run the build again and you should have access to a new “Coverage Report” link.

More to come…

This was just a simple example of getting Hudson setup with a Django project and I know a lot more can be done with Hudson (check out the large number of available plugins). The top items on my todo list are: see Hudson setup environments with virtualenv and pip, integrate more closely with the test suite (possibly using nose), check for PEP compliance, and setup build failure notifications. I hope to write more as I continue to setup our Hudson environment!

References

A few useful Hudson/Python/Django links I discovered while running through this setup:

• Django and Hudson and Bears, Oh My! (and twill, and figleaf)
• Setting up a python CI server with Hudson
• Django continuous integration with Hudson and Nose

Hope to see you there!

We’re proud to annouce that Caktus is hosting a local Django development sprint in the Triangle (Raleigh, Durham, and Chapel Hill/Carrboro) area of North Carolina. The sprint will be held the weekend of December 12th and 13th in Carrboro Creative Coworking, and the purpose of this sprint will be to help finish features and push out bug fixes in preparation for the upcoming Django 1.2 release.

If you’re interested in attending, no previous experience contributing to Django is necessary and the sprint will be a great opportunity to start. Work on other open source Django-based projects is welcome too. For more information, check out the corresponding wiki page and don’t forget to register for the event.

We’ll be there to open the doors at 9am both days. Courtesy of our sponsors there will be free drinks, snacks, and lunch to go around. Hope to see you there!