Book Review: The Innovator's Dilemma

Posted by Tim Freund Tue, 06 Jan 2009 02:41:00 GMT

Is it possible to stay on top once you get there? That is the central question in The Innovator's Dilemma. Although I'm far from the top of anything, I read it anyway because so many other people have said it is an amazing book. Oh, and it can be had for a song on Amazon.com.

I read The Innovator's Dilemma primarily because it is mentioned by so many other entrepreneurial people as a great book. The fact that it can be had used for a fraction of the cover price didn't hurt, either.

I was able to storm through the book in a single day, not because it was captivating, but because it was a pretty good read and I was enduring a full day's worth of air travel. The first part of the book was more interesting than Sky Mall, but less interesting than much of the reading material around my house. There were some great nuggets of wisdom in the middle and back of the book that I may have never read if it weren't for my situation.

Christensen uses the hard drive industry as his primary example throughout the book. The industry has a large body of empirical evidence available for research, and, as a bonus for those of us who code, it is an industry that most software developers are already familiar.

Despite the evidence available in the hard drive industry, I was actually more interested in the other examples used in the book, especially the story of the mini steel mill industry.

The example of mini mill steel plants eating away at big steel's market from the bottom up is a great example of why it is OK to focus on small problems. Small problems solved well create opportunities for us to turn our attention to ever larger problems. If businesses can boom and blossom with that approach in a high capital high risk business like steel, then we software developers have no excuse not to consider small problems.

The Innovator's Dilemma isn't a book that I will keep on my shelf to read again and again, but it is one that I am very glad to have read through once.

Posted in Reading List | no comments

Evolution of an Android Application: Day 2

Posted by Tim Freund Fri, 19 Dec 2008 06:31:00 GMT

I'm still working on the Rhymote project.

I sat down with the intent of immediately putting the code on my phone, but it was just so ugly that I had to spend about 30 minutes cleaning things up. Let that be a lesson: late night programming can be fun, but it isn't always productive.

Posted in Android, Open Source | no comments

Evolution of an Android Application: Day 1

Posted by Tim Freund Wed, 17 Dec 2008 03:04:00 GMT

Programmers are supposed to be lazy right? Here is the scene for one of my lazy moments: my home office contains a desk and a futon in opposite corners. I often want to change music on my desktop PC while reading on the futon. Most people would just get up to change the music without much thought, but I'd much rather use my G1 as a remote control for Rhythmbox. This will be my first Android project, and it shouldn't be too much to handle for a newbie like myself.

I started the Rhymote project yesterday, and I am writing these posts to keep track of all the hurdles I encounter along the way. Although the project could end up doing all sorts of neat stuff, all I really want to do is a handful of operations:

• Play/Pause
• Next track
• Previous track
• Volume adjustments

I have a tendency to over-engineer solutions, so I'm going to explicitly avoid anything that smacks of over ambitious design. Quick and dirty is the name of the game. The code will grow up if it holds my interest for long enough.

Here's the gear I am working with:

• Ubuntu 8.10
• Rhythmbox 0.11.6
• Eclipse 3.4
• Android SDK + Eclipse Plug-in
• T-Mobile G1

All of the code that I write throughout the series can be found in this repository.

Posted in Android, Open Source | no comments

Custom Error Pages with TurboGears 2

Posted by Tim Freund Tue, 21 Oct 2008 03:35:00 GMT

Your TurboGears 2 application is not perfect. Just for a second, let's pretend like it is perfect. Even with all of its perfection, your application will need to deal with bad incoming links and malformed data. Is it ready? It is only a matter of time before your users receive a 404 or 500 response from your most wonderful application. Whether the source of trouble is a bug in the code or a bad incoming link, why leave users lost in the dark? Custom error pages can put them back on track when something goes wrong, and they are very easy to implement. We will create one in the following few paragraphs.

We will use Turtle Goals to demonstrate the techniques in this tutorial. It is open source and fairly simple. Please feel free to follow along in the Turtle Goals source, or work along in your own TurboGears 2 project.

Look in your controllers directory. See that file named error.py? That's the key to a custom error page. The Routes package does a bit of work behind the scenes to send any request that generates an error through the ErrorController, so by customizing the ErrorController, we can customize the resulting 404 and 500 pages. The default document method produces a standard Pylons error page. It looks nice, but it probably doesn't look right compared to the rest of your project.

TurboGears projects default to the Genshi template engine, and that is the engine used by Turtle Goals. Let's create a new template, error.html, and save it in the templates directory.

So now all we need to do is add the expose decorator to the document method, and return a dictionary of appropriate values. Done, with enough time to check Reddit before your next meeting, right? Well, almost. Look closely at the ErrorController definition, and you will see that it is not a standard TurboGears controller. It extends a Pylons controller class, WSGIController, and that causes it to behave differently from our other controllers. At least it should extend WSGIController according to a post on the mailing list. Apparently there is a bug in the quickstart template, and you you may need to chage the ErrorController definition to extend WSGIController. I was easy to convince: as soon as I made the suggested change, my error page started working. Back to the point: the expose decorator will do you no good inside of the WSGIController. It is up to us to render the template to a string and return that string. Fortunately TurboGears provides a method to do just that:

from tg import render
rendered_template = render.render_genshi("error.html", {})

Of course there are also methods like render.render_mako and render.render_jinja if you prefer those other template engines. Here's the full listing of our modified error.py:

There's one other small matter to deal with: configuration. There are two relevant configuration values in your application: debug and full_stack. To use your custom 404 error but still get the interactive debug page for 500 errors, try the following:

debug = true
full_stack = true

The interactive debug page is inappropriate for production environments. When your application is deployed into production, use these settings instead:

debug = false
full_stack = true

If you have both set to false, you will get generic error pages that end users will run from, screaming. All done, for real this time. And you still have time to check Reddit, but you may want to check out these links instead:

• Pylons Error Document Details
• Google's 404 widget
• Wanted: Your 404 Error Pages
• 404 Error Pages: Reloaded

What is your strategy for designing and implementing custom 404 and 500 error pages?

Posted in Python, TurboGears, Tutorials | no comments

Cover Your Nose When You Test

Posted by Tim Freund Wed, 24 Sep 2008 06:06:00 GMT

My mom always reminded me to cover my nose when I sneezed. Now I take it a step further, and I cover my nose when I test.

Nose and the nosetest command are used to run unit tests for Pylons and TurboGears 2 applications, as well as a multitude of other Python applications and libraries. Although nose is great at running tests and reporting back issues, it doesn't natively show developers what isn't being tested. For that, we need a code coverage tool.

Just because nose doesn't handle code coverage reports natively doesn't mean this will be a difficult task. Ned Batchelder's coverage package provides exactly those reports, and nose ships with a plugin to enable it. To install coverage and invoke the reports, you could do something like this:

$ easy_install coverage
$ nosetests --with-coverage  

Name                                         Stmts   Exec  Cover   Missing
--------------------------------------------------------------------------
_strptime                                      228    149    65%   23, 80, 84-89, 155, 169-170, 175, 189, 237, 280-294, 303-304, 306, 314-323, 329, 332, 353-360, 366, 368, 374-388, 393-427, 431-432, 443-446
encodings.ascii                                 19      0     0%   9-42
ez_setup                                       103     11    10%   53-62, 80-104, 117-151, 156-190, 197-222, 226-229
fixture                                         10      9    90%   38
fixture.base                                   124     33    26%   10-19, 25-28, 48-49, 56, 60, 64, 98, 103-104, 122-217
fixture.dataset                                301    225    74%   41, 51-52, 55-59, 65, 76-77, 80, 98-110, 141, 144, 225-229, 232-239, 243-248, 277, 285-289, 323-325, 447-450, 456-457, 461, 468, 479, 483, 529-530, 560-564, 571-574, 580, 631, 720-722, 725-737, 740-744, 752-753
... lots of lines cut to keep you from going blind ...
zope.interface.ro                               22     22   100%
zope.interface.verify                           45     29    64%   46, 51, 56-61, 66, 70, 75-79, 88, 93, 104, 107, 109, 111
--------------------------------------------------------------------------
TOTAL                                        17257   7646    44%
----------------------------------------------------------------------

Good Gravy, man! That's a big coverage report!?

As cool as it is to know that we can get coverage reports for the entire Pylons/TG2 stack, how about we focus for just a minute on only our project. That's the problem with kids today, no focus.

$ nosetests --with-coverage --cover-package=YOUR_PACKAGE_NAME

OK, that's better. Now the report only shows the coverage for code inside of our package. But, seriously, do we really need to type those arguments every time?

Of course not. Sitting there in the root of your Pylons or TurboGears 2 application is a file named setup.cfg. If I were a betting man, I'd say you've never opened it. Ever. Let's knock the dust off and take a look at it in any decent text editor. We're looking for a section named [nosetests], and since you've never changed the file, it is probably at line 8 and looks just like this:

[nosetests]
with-pylons=test.ini

We can add any additional options for nose in this section. Now is your chance to spring into action. Add the following lines to

with-coverage=true
cover-package=YOUR_PACKAGE_NAME

Save setup.cfg and run your tests. You will see a code coverage report at the end of the test run, and the last two columns will be the most interesting. They show the percentage of the code that was covered and the lines of code that were not covered, respectively. Depending on the outcome of your coverage report, you may be feeling rather smug right now. Stop it, we're not Rails developers, and there is still work to do.

On the other hand, if your code coverage report leaves you feeling a little ashamed of just how much code is uncovered, don't despair. Just knowing what code coverage is and caring about the results of a coverage report already puts you into a minority of all programmers. Pick a block of untested code and write a test. All of the sudden, your numbers are better and you'll start to leave those fly-by-the-seat-of-their-pants programmers in the dust.

Remember that this setup.cfg trick will work with any python application or library that is testable with nose. We happen to be focusing on Pylons and TurboGears because that's what I've been busy using lately. How do you use nose to help you write better code?

Posted in Python, TurboGears, Tutorials | no comments

Python and Jabber: Head to Head Library Review

Posted by Tim Freund Tue, 12 Aug 2008 03:42:00 GMT

Python is known as a batteries included language, but sometimes it can feel like a home improvement project gone bad. You know, the kind of project where there seems to be too many parts for the work that remains? I had that feeling as I tried to figure out which Jabber/XMPP library would suit my application best. Like any programmer worth their salt, my first tool for research is a visit to Google. Two searches, python jabber and python xmpp, seemed to turn up three viable projects.

• jabber.py (GNU LGPL)
• PyXMPP (GNU LGPL)
• xmpppy (GNU GPL)

There’s also a reference to Twisted’s support for XMPP in the first page of search results, but we’ll save Twisted for another day. I haven’t grasped Twisted enough to feel comfortable writing about it, and the application that I am working on doesn’t use Twisted in any way. Keep an eye on these guys for more information regarding Twisted’s XMPP support.

All three of the options seem like reasonable choices, so we will review each of the packages for the following criteria:

• Installation
• Ease of Use
• Support Options

Installation

The trial begins with an empty virtualenv environment. We have a blank slate into which we can install the projects. We will first attempt easy_install $PROJECT_NAME, and then easy_install $DOWNLOAD_URL. Any library that won’t easily install with easy_install is going to be a source of grief form any team or project that depends on it.

jabber.py

(xmpp)tim@prime:~/src/xmpp$ easy_install jabber.py
...
error: Could not find suitable distribution for Requirement.parse('jabber.py')
(xmpp)tim@prime:~/src/xmpp$ easy_install "http://downloads.sourceforge.net/jabberpy/jabberpy-0.5-0.tar.gz?modtime=1075826815&big_mirror=0" 
...
Finished processing dependencies for jabber.py==0.3-1

It appears that jabber.py isn’t in the PyPI index at all, and when we attempt to download and install jabber.py version 0.5-0, we are told that we have just installed version 0.3-1. I downloaded jabberpy-0.5-0.tar.gz and confirmed that setup.py reports the distribution as version 0.3-1. One other slightly confusing issue: the README file reports jabber.py as a GPL project, but the home page and setup.py files report it as LGPL. The xmpppy project also refers to jabber.py as LGPL. Best 3 out of 4 licensing?

The good news: jabber.py installation is painless on Windows, OS X, and Linux, as long as a download URL is provided.

PyXMPP

(xmpp)tim@prime:~/src/xmpp$ easy_install PyXMPP
...
Reading http://pypi.python.org/simple/
Reading http://pypi.python.org/simple/pyxmpp/
Reading http://pyxmpp.jabberstudio.org/
No local packages or download links found for PyXMPP
error: Could not find suitable distribution for Requirement.parse('PyXMPP')
(xmpp)tim@prime:~/src/xmpp$ easy_install http://pyxmpp.jajcus.net/downloads/pyxmpp-1.0.0.tar.gz
...
Finished processing dependencies for pyxmpp==1.0.0

PyXMPP is in the PyPI index, but the download URL is out of date and the download URL is not automatically discovered from the page at http://pyxmpp.jajcus.net. The package installs without issues once the download URL is passed to the easy_install program, assuming you are not using Windows. Windows users will likely see something like “Python was build with Visual Studio version 7.1, and extensions need to be build with the same version of the compiler, but it isn’t installed.” And with that, we’ve exceeded the scope of this tutorial.

Before we go further, are you running OS X? I know you OS X users were laughing at the Windows guys in the last paragraph because their inferior system left them hanging. Sure, PyXMPP installed just fine on your Mac Book Pro, but open an iTerm session right now and try to run a PyXMPP application. You will probably see ImportError: No module named libxml2, and if you don’t, you’ve run into this problem before and fixed it yourself. Let this be a lesson, karma always finds a way. Google may hold the answer to this problem, however, I would guess that Cuil does not.

So now we have, what, maybe a handful of Linux users still chomping at the bit to try PyXMPP? Let’s get going. One more package to install, and we’re off to the races.

(xmpp)tim@prime:~/src/xmpp$ easy_install dnspython
...
Finished processing dependencies for dnspython

xmpppy

Hey, you Windows and OS X users, you can come back now. This next one is a piece of cake for everybody:

(xmpp)tim@prime:~/src/xmpp$ easy_install xmpppy
...
Installed /home/tim/pyenvs/xmpp/lib/python2.5/site-packages/xmpppy-0.4.1-py2.5.egg
Processing dependencies for xmpppy
Finished processing dependencies for xmpppy

The latest version of the xmpppy project installs automatically with easy_install.

Ease of Use

We are about to write some code, and this is where any head to head competition can turn subjective. For the sake of this comparison, I have written a small base class that does the following:

1. Create a chat bot
2. Send an arbitrary message to an arbitrary user on a supplied list every 10 seconds
3. Accept messages from users: print the message to the console, and thank the user.

Although the messages sent in this demonstration are fairly silly, the concept can easily be applied to real world applications. For instance, bug trackers could send notifications to developers, and developers could request status changes with the help of a chat bot.

We will create an implementation for each of the libraries under review. Before we begin, you may want to review the base class below. You can also check out the whole package from subversion.

jabber.py

Let’s dive in with jabber.py because it comes first alphabetically. That’s the kind of rigorous science we’re doing here, we alphabetize.

The JabberPyBot subclass proceeds in a pretty straightforward way that matches my experiences with writing Jabber client software in Java. We create a Client, connect, authenticate, and prepare to send and receive messages in the __init__ method.

I was able to send messages in very little time, but receiving messages was a little tricky. We registered a message handler when we created our Client object, and I thought that registering the handler would cause a glitter-laden jabber fairy to fly in on gossamer wings and sprinkle a little “we’ll handle these incoming messages automagically” dust over the code.

I ran the code a second time. Surely those dropped incoming messages were a fluke, but running unchanged code a second time produced exactly the same results. I was shocked. Shocked! Truthfully, I was just delaying the inevitable RTM moment. After reading through the example a second time, I saw the process method. That’s the ticket, when called it processes queued messages and fires off the appropriate handlers. We wire the process method into a receiver method that will run in its own thread, and we’re in business.

PyXMPP

The PyXMPP client was the last of the three clients written. So much for rigorous, alphabetized, science, eh? This code was a nice change of pace since the jabber.py and xmpppy clients were so similar, as you will soon find out.

The code for the PyXMPP implementation is just as short as our other two implementations, but something feels a little funny about it. Notice how we need to keep digging into our connection (JabberClient) object and grabbing its stream object to get stuff done? I don’t know about you, but that feels a little dirty. Matter of fact, there’s even a name for the “rule” that we are breaking. Take note, and during your next code review at the office when you see something similar you can say “I don’t know about that, Barry, the AbstractDatabaseManagerManagerFactory doesn’t adhere to the Law of Demeter very well, and I think it will be a maintenance nightmare down the road. Are you going to support this stuff when the API changes in the future?” But who are we kidding? Barry is such an old crank that it’s easier to fix his stuff after it’s been deployed than argue with him about his crazy design right now at 10:55. It is so close to lunch, and we can’t be late for that. Man, I’m hungry. A gyro sounds really good.

What were we talking about? Oh, right, our PyxmppBot and the code that talks to more than just its immediate friends. I feel like we’re giving the library a quick brush off just because it doesn’t adhere to the world view that we’ve adopted in the other two client examples. Let’s take another look at the PyXMPP example. We have formed the opinion that our JabberBot “has-a” connection/client object and interacts with it, but the PyXMPP programmers seem to think that a bot “is-a” client/connection object. The have been doing this stuff longer than we have, so let’s give it one more try.

That feels a little bit cleaner. If Demeter is happy, I’m happy.

xmpppy

The xmpppy client was shockingly similar to the jabber.py client, but the similarities were explained away upon reading the xmpppy home page where it mentions that some of the code and API decisions came directly from jabber.py. That just goes to show, when in doubt, read the docs.

One thing I didn’t quite understand about the xmpppy API was the choice of method names. Why are some methods given capitalized names, while others are lowercase? This is probably a silly complaint on my part, but I just know I will spend a lot of time second guessing the capitalization of method names if I use this library in my applications.

Reading the xtalk.py source provided most of the pointers that I needed to get our client up and running, with only one minor issue that stood in the way. The example calls a method named SendInitPresence which was since changed to sendInitPresence. Again, this made me wonder about the naming conventions in use.

Support and Growth Options

The example we worked through today was pretty basic. Where do these packages leave us as our skill and requirements grow?

jabber.py

Jabber.py is dead, dead, dead, and they are kind enough to tell us that right on the front page. If that’s the case, why did we look into it? Well, it works, and it isn’t licensed under the GPL. The working bit matters to everyone, and the GPL bit matters to some. If you have fairly simple needs and a list of requirements that includes “NO GPL!”, then this may just be the project for you.

PyXMPP

PyXMPP under active development, and it is a capable performer that seems to have a lot of power available for developers. For instance, as your bots grow, you can transfer your efforts into server side Jabber components, and the PyXMPP API will support your efforts. Of course, the fact that a large majority of developers will need to jump through a few hoops to install the library will be a source of frustration.

• API Docs
• Bug Tracker
• Home Page
• Mailing List

xmpppy

The xmpppy project’s last release was in December of 2007, and the mailing list remains active. It installs easily on Windows, OS X, and Linux, and it is a fairly easy to grasp library that could be quickly integrated into existing applications. The fact that it is GPL licensed could cause heartache for some projects or corporations, so proceed with caution if you are working in such a situation.

• API Docs
• Bug Tracker
• Home Page
• Mailing List

Conclusion

Sometimes the best way to pick a library is to sit down and write a little bit with each of the contenders. A little bit of code, and a little bit of reading goes a very long way toward a sound design decision. Sometimes these experiments lead us to question our initial assumptions. In this case, I am reconsidering Twisted to handle my XMPP needs since it is mature, under active development, and installs quickly on all of my potential target platforms.

What did I do right? What did I get wrong? What did I miss? Your comments are appreciated!

Submit to programming.reddit.com

Posted in Code, Python, Tutorials | no comments

Code and Philosophy

Posted by Tim Freund Thu, 05 Jun 2008 06:08:00 GMT

Remember back to your school days and you will recall at least one time when a student raised their hand and asked "why will this ever matter?". Usually this happens in the middle of a lecture on the square root of negative 1 and the plane of complex numbers. Not once did anyone ask that question during my time in PL115, Reality and the Human Existence. This was philosophy class, and we were there to learn for the sake of learning. And because it’s a required course at that particular Jesuit institution.

I just read Bob Walsh’s "Games microISVs play with themselves," and the chain of thoughts that it kicked off has given me an epiphany that I thought would never happen. Material that I learned in PL115 finally applies directly to my reality and existence.

The primary thrust of his article applies the lessons of The Panacea for Putting Things Off to his life in a microISV. The following quote from the original article made me sit up and pay attention to Bob’s entry:

Putting something off once makes it easier to put it off again, and before you know it, several weeks have past and you still haven’t done it?

That made me chuckle. I have a problem with putting things off. Heck, I even wrote a task tracking application that is tailored to track tasks in weekly increments and provide me with statistics on my weekly completion rate as well as the average number of weeks a task is put off before it is finally finished. How did I ever find the time to work on that application? I was putting off other work.

That’s right, rather than acknowledging my problem directly and put my nose to the grindstone, I created a whole new project to distract myself from more challenging tasks. Perhaps I just don’t get it.

This is where Plato enters the scene, perhaps a cave.

Starting and finishing a software project are the two hardest things for me to do, and it’s all because of Forms. Not Windows Forms or web forms, just Forms. Plato has this Theory of Forms where he hypothesizes that in the universe there are Forms that are the essences of various objects, and there may be numerous implementations of a Form. While an implementation will have at it’s root the essence of the Form, that implementation will not have the perfection that the Form is. Whew, that’s a little deeper than I usually like to go in a blog about software, so lets pull this up to a more practical level.

Back to the first and last days of a project. Day one takes forever to arrive, because I am a sucker for great looking code, and I love to learn new things. Before I actually sit down and plunk out the first lines of code in my emacs session, I think, and I just keep thinking. The idea rolls around in my head as I eat delicious barbecue or walk my two mongrel dogs. I’m contemplating design, as well as tools. How can I best implement Product X? I am distilling the essence, the Form, of what it means to be Product X. I know that I as soon as I start to code, I’m committing myself to a design. Regardless of how "agile" that design may be, I know that a certain percentage of my code will one day have the burden of being "Legacy Code" hoisted upon itself. No wonder it is easy to delay the start of a project.

And then, as the project winds down, I need to figure out when to call it done, or at least 1.0. I survey the code, and it isn’t nearly as beautiful as I originally intended. Although the essence of Product X is there, my implementation is but a mere shadow of a replica of what it means to be the perfect Product X as imagined in my mind. It may work, and customers may love it, but my code never quite matches what I imagined on day one. That makes it difficult for me to ever call a software product done.

So there you have it, Plato’s Theory of Forms as it applies to the trouble I have starting and finishing projects. Perfection doesn’t exist in software, and there is nothing wrong with a little mid course correction. Plato devised the Theory of Forms, but that didn’t stop him from continuing to write and work. I didn’t even particularly like that philosophy class, so why should I let Plato’s theory hold me back?

Well, it’s late. Should I go to bed, or actually get a little bit of real work done? ;-)

submit to programming.reddit

no comments

BarCampKC Happens May 9-10

Posted by Tim Freund Thu, 10 Apr 2008 15:41:00 GMT

A slightly modified BarCamp is happening in Kansas City this May. The diff: no actual camping, and sessions will be sketched out the night before they happen. It looks like the event will happen in an office building at 80th and Metcalf, so the location should be easy to reach for most people, and there’s plenty of decent food close by.

I would like to work up something to present, probably something Python related, but Java is always a possibility as well. FPyS, perhaps, or Transponder if I can clean it up before then. Maybe I will even dabble with AppEngine before then and sketch out a talk.

If you enjoy programming and the company of other programmers, try to block out May 9th and 10th on your calendar and join us. Full event details are available.

Posted in Events | no comments

FPyS, A Python Client for Amazon FPS

Posted by Tim Freund Mon, 10 Dec 2007 06:56:00 GMT

Earlier today I posted a note in the Amazon Developer Forums about FPyS, my attempt at a Python library for the Amazon Flexible Payment Service. It's rough, very rough, but it is a decent start at a fully featured FPS library. The library provides enough functionality to run FPeS, but not much beyond that.

For the curious, the demo application is a TurboGears application that allows people to select and purchase full sized images from a thumbnail gallery. The application is configured to process payments through the Amazon FPS Sandbox, so any completed transactions are drawn from imaginary credit cards. With that in mind, I'd like to welcome anybody to attempt a sale. The more the merrier, as it will cost us nothing and help to find bugs in the library.

Posted in Open Source, Python, TurboGears

Schema Migrations in TurboGears with Migrate

Posted by Tim Freund Mon, 06 Aug 2007 00:42:00 GMT

I learned about the importance of schema migrations the hard way. At my previous job, I helped a team upgrade a Java web application. The upgrade involved schema changes, and I had the forethought to script the upgrade and thoroughly test it on the development database. Even with that preparation, the night of the upgrade would teach me two important questions that new developers should always ask of their team. Does the development database schema match the production database schema? And do you know how to restore the database from backups should anything go wrong? The answers to both questions on that night were no and oh no. Sometime after midnight things started working.

A better way existed. I first learned of schema migrations a few months earlier when exposed to ActiveRecord::Migrations. After using them on several projects, I was itching to have the same capability in my Python and Java projects. The Pythonic answer came in the form of Migrate, a schema migration library for SQLAlchemy, and direct support for TurboGears was added with TGMigrate. Having migrate integrated with my projects greatly reduces my blood pressure on deployment days. Of course, I still make sure that my database backups are working.

Those interested in integrating Migrate with a TurboGears project might enjoy the screencast I just completed on the topic. If you use SQLAlchemy but avoid Migrate, I would be interested to hear what is holding you back.

• Add Migrate Support to TurboGears (QuickTime)
• Add Migrate Support to TurboGears (DivX)

Schema refactoring and migration was one of three topics at the last DotNext Kansas City Tech Coffee meeting. Notes on the schema migration talk were posted to Squidoo.

My apologies to those with small screens. I will record my next screencast in a smaller window. Your comments on today's screencast are appreciated.

Posted in Code, Python, TurboGears, Tutorials