Archive for the 'Uncategorized' Category

wsgi middlewares for profiling and debugging

A while back I implemented a debugging and profiling middleware, and I’ve been using those with Pylons recently. I think their pretty useful, so I’ve wrapped them up into an installable egg that contains Paste interfaces in setup.py. This allows you to easily insert the middleware into any existing Paste project (e.g. a Pylons project). I’m basically going to use this middlewares project as a dumping ground for middleware I find useful. It currently contains a debug (via DBGP) middleware, a profiler and a csrf middleware. There is nothing that says you must use Paste with these, Paste just makes it easier. The csrf middleware is currently tied to using Beaker.

Here’s a screenshot of Komodo IDE debugging my Pylons app. Lots of other debuggers support DBGP, but for many reasons I like Komodo.

Below is a partial output of line profiling the csrf middleware call handler. The profiler can use line or call profiling. I find line profiling handy when I want to focus into a specific area. While debugging and call profiling require no code changes in your code, line profiling does require you to decorate the function(s) you want to line profile.

The profiler currently requires a patch if you want to do line profiling with Pylons. I’ve sent the patch to the maintainer of line_profiler.

Timer unit: 1e-06 s

File: ...../csrf.py
Function: __call__ at line 40
Total time: 0.004621 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
    40                                               @profile
    41                                               def __call__(self, environ, start_response):
    42         1           29     29.0      0.6          request = Request(environ)
    43         1            3      3.0      0.1          session = environ['beaker.session']
    44         1          476    476.0     10.3          csrf_token = session.get('csrf')
    45         1            4      4.0      0.1          if not csrf_token:
    46                                                       csrf_token = session['csrf'] = str(random.getrandbits(128))
    47                                                       session.save()
    48
    49         1            7      7.0      0.2          if request.method == 'POST':

database migrations for SQLAlchemy part duex

Well, as I was looking at making miruku more reliant on sqlalchemy-migrate, I discovered the expirmental command: migrate update_db_from_model! So much for an afternoons work, but at least I’m much more familiar with the migration tools. So here’s how I’ve implemented an auto upgrade for Pylons.

First, easy_install sqlalchemy-migrate

Now, in your pylons development.ini, add the following to app:main:

# SQLAlchemy migration
# if managed, the migration repository is here
migrate.repository = %(here)s/changes
# automatically do database upgrades
migrate.auto = 1

Then, in PRJNAME.config.environment, in load_environment, after the call to init_model add the following:

    
    # sqlalchemy auto migration
    if asbool(config.get('migrate.auto')):
        try:
            # managed upgrades
            cschema = schema.ControlledSchema.create(engine, config['migrate.repository'])
            cschema.update_db_from_model(meta.Base.metadata)
        except exceptions.InvalidRepositoryError, e:
            # unmanaged upgrades
            diff = schemadiff.getDiffOfModelAgainstDatabase(
                meta.Base.metadata, engine, excludeTables=None)
            genmodel.ModelGenerator(diff).applyModel()

Of course, don’t forget the imports you need:

from paste.deploy.converters import asbool
from migrate.versioning.util import load_model
from migrate.versioning import exceptions, genmodel, schemadiff, schema

Run your app: paster serve –reload development.ini

Now with most basic changes in the model, when paste reloads your database will be updated to reflect the new model. This of course can fail sometimes, such as adding a new column with nullable=False.

I’m only using the unmanaged upgrades right now, so the managed section may need some tweaking, I’ll see when I get there.

get the dirt on your communications!

I’ve been hacking on a few things lately, one is the Contacts addon for Firefox and getting it to work in Thunderbird.  A quick demo of a little feature I added to Contacts in Thunderbird…

Here I’ve received an email from some random person, and I need to know who they are and why I should care that they’ve emailed me…Give me the dirt!

And here is a Contact tab in thunderbird as the result:

I’m not sure about this email, that guy looks a little sketchy.

You can find out more about Contacts at Mozilla Labs, and follow the Thunderbird patch I’m working on in bugzilla.

getting started

Here i am, getting started.  Initially this will primarily be work content, though eventually I’m sure I’ll branch out.  I’ve written a little bit about me for those that want to know.