Post archive for March 2009

Migrating from django-photologue 1.x to 2.x

March 27 2009 by Tobias McNulty

We're in the process of updating a web app for a client that was built last year about this time using Django and Photologue. Needless to say, there have been a lot of changes to both over the past year!

We were somewhat dismayed to find no easy upgrade path for photologue, and there are a number of model changes that mean you can't just run svn up and be done with it. Using the JSON output from ./manage.py dumpdata, we created a little Python script that handles the database migrations for three of the photologue models (Gallery, Photo, and PhotoSize). Save this in a script called migrate-photologue.py:

#!/usr/bin/python

import sys
import simplejson

if len(sys.argv) != 2:
    print 'Usage: %s ' % sys.argv[0]
    sys.exit(1)

REMOVE_COLUMNS = {
    'photologue.photo': (
        'photographer',
        'info',
    ),
}

RENAME_COLUMNS = {
    'photologue.photo': {
        'pub_date': 'date_added',
        'slug': 'title_slug',
    },
    'photologue.gallery': {
        'pub_date': 'date_added',
        'slug': 'title_slug',
    },
}

ADD_COLUMNS = {
    'photologue.photo': {
        'view_count': 0,
    },
    'photologue.photosize': {
        'upscale': False,
        'increment_count': 0,
    },
}

data = simplejson.load(open(sys.argv[1]))

for obj in data:
    fields = obj['fields']
    model = obj['model']
    for col in REMOVE_COLUMNS.get(model, []):
        if col in fields:
            fields.pop(col)
    for old_name, new_name in RENAME_COLUMNS.get(model, {}).iteritems():
        if old_name in fields:
            fields[new_name] = fields[old_name]
            fields.pop(old_name)
    for col, default_value in ADD_COLUMNS.get(model, {}).iteritems():
        fields[col] = default_value

print simplejson.dumps(data, indent=4)

The script is fairly simple, but back up your database first, just in case. If you need support for additional models, just add the changes you need to the dicts at the top of the file.

During the upgrade, it might help to have two copies of the database running on the local machine, so you can switch back and forth between them at will. A typical migration might look like this:

./manage.py dumpdata photologue > photologue.json
./migrate-photologue.py photologue.json > photologue2.json
./manage.py sqlclear photologue | ./manage.py dbshell
svn up photologue # or however you do it
./manage.py syncdb
./manage.py loaddata photologue2.json
./manage.py sqlsequencereset photologue | ./manage.py dbshell # just in case

Of course, things will get more complicated if you have other models with foreign keys to any of the photologue models. You'll have to drop the constraints temporarily and then add them again after you finish the migration, or take the plunge and write the SQL to do the migration while keeping your database relationships intact.

Solving NFS issues on embedded machines

March 25 2009 by Tobias McNulty

As part of my work on EveryWatt, I setup an NFS-based development environment for one of the data loggers we use for energy monitoring in the Caktus office. The stock 2.4 Linux kernel in the machine seemed to have some trouble mounting the file system root I had exported from one of our servers. The symptoms included long delays for most if not all activities that used the file system and lots of messages like these in dmesg:

nsm_mon_unmon: rpc failed, status=-110
lockd: cannot monitor 192.168.x.x
lockd: failed to monitor 192.168.x.x

The issue turned out to be that statd was not running on the client embedded machine. The solution is simple: mount the file system with the -o nolock option, to get around the statd requirement, like this:

mount -o nolock 192.168.x.x:/path/to/nfsroot /mnt/root 

Now the system is zippy as ever, and I have an embedded root file system that includes Python 2.4 and sqlite, and fits in just under 3.5 MB (compressed)!