There’s a large body of technical information out there about content management systems and frameworks, but not much written specifically for decision-makers. Programmers will always have preferences, but it’s the product managers and supervisors of the world who often make the final decision about what platform on which to deploy a sophisticated site. That’s tricky, because web platform decisions are more-or-less final — it’s very, very hard to change out the platform once the wheels are in motion. Meanwhile, the decision will ultimately be based on highly technical factors, while managers are often not highly technical people.

This document aims to lay out what I see as being the pros and cons of two popular web publishing platforms: The PHP-based Drupal content management system (CMS) and the Python-based Django framework. It’s impossible to discuss systems like these in a non-technical way. However, I’ve tried to lay out the main points in straightforward language, with an eye toward helping supervisors make an informed choice.

This document could have covered any of the 600+ systems listed at cmsmatrix.org. We cover only Drupal and Django in this document because those systems are highest on the radar at our organization. It simply would not be possible to cover every system out there. In a sense, this document is as much about making a decision between using a framework or using a content management system as it is between specific platforms. In a sense, the discussion about Drupal and Django below can be seen as a stand-in for that larger discussion.

Disclosure: The author is a Django developer, not a Drupal developer. I’ve tried to provide as even-handed an assessment as possible, though bias may show through. I will update this document with additional information from the Drupal community as it becomes available.

Systems Make Assumptions

Every web shop has its favorite tools and languages. Settling on just a few keeps toolchains, deployment, and development workflows sane. At both the UC Berkeley Graduate School of Journalism and the Knight Digital Media Center, we’ve settled on a mix of:

• WordPress for basic sites and “modest” online publications
• Django for sites with more complex needs or non-standard data models

For example, it makes sense to build student magazines, handbooks, FAQs, and blogs with WordPress. But it’s simply not possible to build an equipment checkout system, or a course evaluation system, or a student/faculty/staff directory with WordPress — not while trying to preserve your sanity. That’s because WordPress assumes so much about the structure of your data — it thinks every piece of content has a title, a summary, an author, etc. Start to veer away from that basic data structure and you find yourself quickly needing a platform that doesn’t make assumptions about how things should be done. For those kinds of problems, we use Django.

Meanwhile, there’s a lot of Drupal momentum on the UC Berkeley campus. In fact, there’s a lot of Drupal momentum all over the world — e.g. the recent move of whitehouse.gov to Drupal. And with that momentum comes questions from supervisors.

“Why aren’t we using Drupal?”

To be clear on terms:

• WordPress is a simple content management system.
• Drupal is an advanced content management system.
• Django is a framework.

Frameworks operate at a lower level than CMSs, and provide less of a turnkey experience for basic sites. In contrast, a framework is a box of parts from which you build your dream CMS – the CMS that precisely fits your organization’s data model and workflow. But the lines aren’t always so clear. Django comes with a self-generating administrative interface that’s so good and so user-friendly that we find it works fine as the CMS for the vast majority of projects we work on. The self-generating administrative interface can be tweaked and customized, or ignored altogether. It’s an optional component – you’re free to use it, modify it, or ignore it completely and write your own CMS on top of the framework.

On the other end of the spectrum, Drupal has some framework-like aspects to it. It’s been said that “Django is a framework with CMS-like tendencies, while Drupal is a CMS with framework-like tendencies.” So before you protest that I’m comparing apples and oranges, remember that in real life, the lines are blurry, and that apples and oranges can be compared, if what you intend is to compare types of fruit.

Every assumption made by a platform can either be a time-saver or an obstacle when it comes time for your site to grow or evolve, depending on:

• The ways you want that evolution to happen
• The skill of your developers

By their nature, frameworks are more flexible than CMSs. They make NO assumptions about the shape of your content, the Ajax system you might want to use, which rich text editor to present to your content people, etc.

Here are some of the key architectural points that matter to us when selecting a publishing platform, with pros and cons for Drupal and Django. I’ve sprinkled in some comments and feedback from members of the Django developer community (relevant comments from the Drupal development community will be included as well).

Object Orientation

Python (the language on which Django is built) is extremely object-oriented, and thus so is Django. This means that data objects, querysets, variables, templates and arguments can be passed around in the codebase easily. This makes it easier to reduce or completely eliminate redundant code (Django subscribes to the DRY principle).

Drupal is generally not object-oriented. However, Drupal developer “catch” adds:

Drupal 7’s field and entity APIs are a step towards an ORM. Drupal’s database layer and many other subsystems including contributed modules like Views are object oriented to various extents.

MVC/MTV

Django uses the model-view-controller methodology (going by the name MTV in Django-land). This means that in Django, developers have almost complete separation of logic (programming), data structure, and display logic. MVC/MTV has emerged as a core aspect of modern programming practice.

Django is MVC from the ground up.

Templating System

Django is famed for its super-clean, inheritance-based templating system, which makes it almost trivially easy to nest templates within templates, use different templates for different parts of the site, and to completely eliminate any redundancy in template code. Working with Django’s templating system is often described as “a joy.”

In contrast, a common complaint about Drupal is that templates can be difficult to customize. In fact, there are entire books about working with templates in Drupal.

Said one Django developer on working with Drupal templates: “I always felt like I was shoving a square peg in a round hole.”

Drupal developer “catch”:

Drupal’s PHPTemplate theme layer gives you a lot of freedom to do processing in themes, which can be good or bad depending on situation/skillset etc. … However, even if you do have code which you need to share between themes: since at least Drupal 5, we’ve had theme inheritance (sub themes) which means if you have some central logic which would need to be re-used between multiple themes – whether that’s templates, CSS files, preprocessing etc. (either on the same site or completely different sites) then there’s zero need to copy this between them – you just have your base theme with that stuff in, then build a subtheme on top of it, subthemes can be as light as a single CSS file.

The theme/sub-theme system does give Drupal the ability to do theme inheritance. However, the basic methodology still allows for the inclusion of business logic in themes, which breaks the separation of presentation, content, and programming logic. The Django project is very strict about this, and has been very good at saying “NO” to proposals to make Django’s templating language include more options for direct programming. The Django philosophy is, “If you feel like you need to do some programming in your templates, you’re doing it wrong.” However, it is possible to create your own custom template tags and filters to extend Django’s base template language. Developers often share these custom tags and filters on sites like djangosnippets.org.

Once you head down the slippery slope of scattering code logic between internal functions and external templates (which should be kept simple enough for a designer to work on), you lose the clean separation of logic and presentation and no longer have certainty that certain kinds of code must be in certain places. My view is that Django’s insistence on keeping as much programming out of templates as possible makes people better developers, since it enforces “best practices” in programming.

Contrast all of this to WordPress theme development, where developers have to copy chunks of logic over to the new theme manually if they ever decide to change themes.

ORM

In Django, development begins by defining the data models that describe the organization, publication, or site. For example, these might be Stories, Authors, and Presses, or Equipment, Schedules, and Reservations, or Faculty, Students, Courses and Evaluations, etc. All of these have datatypes and interdependent relationships.

From that data model, database tables are auto-generated, and the system becomes internally “aware” of data relationships. To query for various sets of data, the Django developer uses something called the Object Relational Mapper, or ORM. So, rather than writing raw SQL queries, the Django developer writes “querysets” like:

1

	     books = Book.objects.filter(author='Hacker')

In a simple example like that, there’s no real time savings over writing the equivalent SQL. But when queries become complex, SQL gets difficult and error-prone, while querysets remain straightforward and readable. When developers can save time and reduce errors, organizations save money.

Drupal does not have an ORM.

Preston Holmes: For relatively “flat” data, Drupal’s CCK is usable by non devs to create custom content types, and the Views module allow for simple generic_view style pages. When it comes to more complex objects, Django’s ORM is far superior.

Starting with the data modeling process is a key point. Django makes no assumptions about the needs of your organization or the shape of your data, which means the end product should fit your organization like a glove. Django projects are built up to fit the organization, whereas Drupal developers often talk about having to remove or work around the things they don’t need. Many developers who have spent time in both systems say the Django methodology is easier and faster.

Performance

I don’t have numbers to support this, and it would be quite hard to test without building the equivalent test site in both Django and Drupal, but the conventional understanding is that Drupal has a lot of functional “layers” through which all page requests must pass. This results in a high query count and relatively sluggish performance. Granted, this limitation is largely moot given that all high-traffic database-backed sites rely on caching to keep performance up, but it goes without saying that a system that’s as light on its feet as possible is desirable. A Drupal site is probably going to require caching to be enabled with a lower amount of traffic than an equivalent Django site.

Bruno Desthuilliers: I recently had to work on a not-so-complex (functionaly-wise) Drupal project, and the response times on my local station – all caches disabled of course – turned the development process into a sluggish nightmare. Never had this kind of problems with Django, even for far more complex apps.

Learning Curve

Drupal is sometimes criticized for having a steep learning curve. But as someone who went through the Django learning curve last year, I won’t claim that Django’s is much better. This really depends on the pre-existing skills of the developer. In my case, my move from PHP-based systems to Django was my first exposure to object-oriented programming, my first exposure to Python, and my first exposure to MVC/MTV thinking. Not to mention learning the framework itself. I think we hear this complaint more about Drupal than about Django because there are more newbie developers in the Drupal world, while Django for the most part attracts more experienced developers.

In fact, I’d go as far as to say that Drupal has an advantage here for basic sites, where little to no customization is required. However, as soon as customization is required, you’re going to need an experienced developer on hand with either system. For systems flexible enough to let you build whatever you can imagine, there are no free lunches.

Rapid Development

Be careful of the term “rapid application development.” Without good skills on-staff, there’s nothing rapid about working with either system. But with those skills, rapid application development is no myth. I can’t speak for Drupal here, but a few recent rapid-development case studies are interesting:

• michaelmoore.com, recently rebuilt in Django in five weeks.
• texastribune.org, built from the ground up in just four weeks with Django.
• Mahalo Answers, built in 35 days with Django.
• baynewsnetwork.org, an RSS mashup and aggregation site with a complex back-end that lets member sites have multiple members and single members contribute to multiple sites — built in 3-4 days with Django.
• django-treedata: Winner of the recent DataSF public dataset Hack-a-Thon, built in 8 hours with Django (and another 8 hours cleaning the public data set).

Both systems have fairly steep learning curves. But once developer skills are up to speed, it’s amazing how easy it is to get the platform to do most of the heavy lifting for you.

Note also that both systems are sometimes distributed as part of larger bundled packages (“distributions”) which extend their capabilities. For example Drupal is part of CivicSpace, which is aimed at building civic/community sites, while the Django world has the option of deploying the Pinax project, which bundles together dozens of re-usable apps to greatly reduce time spent integrating parts into a cohesive whole.

Flexibility

This is very hard to quantify. Both Django and Drupal let developers work with data models any way they please. The question is, what happens when you need to change those models, or add new ones? How easy is it to create new views or slices of the same data? What happens to the logic that ties your data together when the platform is upgraded?

Both systems value flexibility highly, though both go about things in very different ways. What I do know is that when visiting journalists tell us about their Drupal experiences, a complaint we hear a lot is “Everything was fine until we tried to change xyz — then everything fell apart and we’re still trying to recover.” Again, much of this comes down to whether you have good developers on staff more than it does to the platform. Still, the fact that Django is so intensely object-oriented gives it a leg up when it comes to moving things around.

That’s not to say that a complex Django site is infinitely flexible. Even well-designed data models have interdependencies, and a change made in one place can have real ramifications for other parts of the system, which may or may not be easy to recover from. Managers should not be given the impression that just because you’re using a “flexible system” and that “anything is possible” that the webmasters can turn the whole thing inside out on short notice. Things don’t work that way.

Drupal has developed a bit of a reputation for being difficult to upgrade, since so much changes internally between major releases. No platform is immune to that problem, but I’ve rarely seen Django apps affected by upgrades. And when they are, the fix is generally minor.

Bruno Desthuilliers: From my experience, it can take more time doing “simple customizations” with Drupal than you’d need to implement the same features from scratch in Python/Django.

A recent Django success story was the 4-week (!) launch of texastribune.org. Asked why the Tribune chose Django over other platforms, developer Chase Davis replied:

We went with Django for two reasons:

1. We decided early on that the Texas Tribune needed room to evolve. It’s a startup, and nobody has any idea what it’s going to look like in six months. That being the case, our goal was to build them a sandbox — something that could evolve as their organization evolved. We settled on Django because we thought a framework-based approach would give them maximum flexibility, while keeping the benefits of stability and rapid development. If they need a new feature — no matter what it is — we can prototype it, built it and integrate it quickly. We’re not limited by pre-existing modules or dependency issues you find with off-the-shelf systems.

2. The Tribune’s plan going forward is to integrate tremendous amounts of data into their coverage. Their lawmaker directory is one example of that, but they have much more ambitious plans for campaign finance records, financial disclosures, lobbying reports, etc. Rather than segregating that data by walling it off in its own ghetto, we thought Django offered the best chance to integrate it with other content on the site. Drupal isn’t great for building data apps; Django is. If we used Drupal for the CMS and tucked Django data apps within that, tightly integrating the two could quickly become a headache. But with everything in Django, our data apps and CMS play together seamlessly. You’ll see a lot more examples of that in the months ahead.

Community Resources

This is one area where Drupal really has the advantage. Don’t get me wrong – Django has a great community of developers who are more than ready to help each other out, respond to questions on IRC or mailing lists, and a really active blogosphere. But while there are many sites listing re-usable Django applications ready to become part of any project, Django simply does not have the immense collection of downloadable modules present in Drupal-land.

That doesn’t mean we haven’t been able to find ready-made solutions for most situations where we’ve needed them, but there are areas where we’ve really felt the pinch. Case in point: I recently had to implement a survey system on a Django site. That’s an important and not-uncommon piece of software, and you’d hope to find yourself choosing between lots of options. Unfortunately, I found exactly one re-usable survey app for Django, and it was somewhat buggy.

On the plus side, the developers of django-survey responded literally overnight to the half-dozen bug reports I filed on it the first day, but it would be nice if there were several mature survey apps to choose from. That’s the kind of thing that happens when community momentum really takes off. Django’s momentum is increasing by the day, it’s heading in that direction, but Drupal definitely wins in this arena.

Still, there are community-provided apps and resources I utilize on every Django project I build. And nothing in this department has ever been a show-stopper for me. Keep in mind that, in many cases, just building the re-usable app you need when an existing one isn’t available may be easier than expected.

Popularity / Installed Base

By some measures, particularly for decision makers, popularity counts. Is the platform popular enough that I know it’s not going to be abandoned? Can I readily find qualified developers? Is the platform proven in the marketplace? Drupal developer Zack Rosen has this to say:

1. Drupal by all measures of an open-source project is a magnitude larger than Django. Core contributors, community members, deployed sites, any way you want to compare numbers Drupal is a much more established project.
2. Drupal is more proven in the market place. It runs magnitudes more websites (500,000+), and many of note. Whitehouse.gov yes, but Viacom, Warner Brothers, Yahoo, Wikipedia and others all run highly trafficked Drupal sites and have made strategic business decisions to invest in the platform.
3. Drupal has far better commercial support in the marketplace than Django. There are many more (and more qualified) Drupal development shops and specialized service providers in the marketplace.

Excellent points, which I would qualify by noting that Django’s popularity is rising, along with the number of qualified Django developers on the scene, and the number of high-profile Django sites appearing (just tonight, the Mozilla foundation announced that it would be moving its addons site from CakePHP to Django.)

Language

After years of working with both custom and open source PHP code and applications, we felt that we had had our fill of PHP, which has a tendency to sprawl. Granted, sprawl is not much of an issue when working with pre-fab solutions like WordPress or Drupal, but Python’s syntax is clean and uncluttered, which speeds development time and encourages developers to think in more object-oriented ways. Simply put, clean code is easier to write and easier to maintain.

Mike Ramirez: Finally, there is the maturity of the language. PHP is still adding in features other, similar languages already have had. For example namespaces are just barely a year old in php, initial release in the dev versions, less than 6 months for the actual first appearance in a stable release. To quote wikipedia[3] on missing features from PHP: “PHP currently does not have native support for Unicode or multibyte strings; Unicode support will be included in PHP 6 and will allow strings as well as class, method and function names to contain non-ASCII characters.” Seriously, in todays global world and well, for the past what 10-15 years, hasn’t this been mandatory? I do not understand why web developers who use php, still do. I don’t understand why companies and schools support it.

People often work in PHP because it’s the default, and easy to gravitate towards. But that doesn’t mean life couldn’t be a lot better with a cleaner language to work with.

Security

Security is a multi-headed hydra, and affects all web applications, large or small. SQL injection, form sanitization, cross-site scripting, and many other attack vectors require web application developers to stay up to speed on current security trends and to examine old code for new vulnerabilities. It’s a constant chore.

One benefit of using a mature framework or CMS is that many of the most common security issues are handled by the system itself. This doesn’t mean we can afford to get lazy about it, but it’s good to know that hundreds of people smarter than oneself have pored over the code to identify and mitigate security issues.

That much is true of both systems. However, it’s also true that Drupal has a history of security issues, whereas similar issues in Django are extremely rare. This is in part due to the fact that Django has a lower public profile, and in part because Django makes fewer assumptions about where things live and how they’re configured. In other words, less about a site’s setup is known to a would-be attacker since sites built from a clean slate by various developers share less in common with one another.

There are other factors that may contribute to Django’s excellent security track record, such as its URL mapping system, which simply prevents access to URL patterns that haven’t been predetermined by the developer, and its exceptional form generation and handling system.

Still, I don’t want to emphasize the security point too much – all systems can be vulnerable if you do dumb things with them, and vice versa.

Drupal developer “catch” adds:

The Drupal security team regularly releases SAs for security issues which in some cases are theoretical, in many cases have never been exploited (simply reported after code review), and very often require either authenticated or some level of administrative access to a site. Additionally, the 3-4,000 contributed modules are hosted centrally on Drupal.org, not distributed around the web, and have the same rules for security announcements as Drupal core, handled by the same process. The number of security announcements for Drupal core itself is relatively low given code base and deployments.

Deployment

On the other hand, PHP-based systems have a slight leg up in deployment. Since virtually every web host offers PHP hosting. In contrast, setting up a Django system on a server will take some level of fiddling that may not be familiar to the typical web developer. Fortunately, a growing number of web hosts specialize in Django hosting, which makes this problem go away in many use cases.

Preston Holmes: “Where Drupal has its single biggest advantage, is that a technical user, a sysadmin etc, can get Drupal up and running. Someone doesn’t have to know PHP AT ALL to get drupal going. This is because Drupal has a decent admin-UI – where modules are pretty much drop in, activate and configure. Setting up even a basic Django site REQUIRES someone who has some programming skills, even if thats just a matter of setting up the settings.py file.

But the Python/Django deployment hurdle is not huge, and is a one-time problem to solve. Once Apache has been configured to work with Django, you don’t need to think about it again. This hurdle affects hobbyists, who tend to look for a simple button in their web hosting control panel, far more than serious sites for whom server configuration is just part of doing business on the web.

Developer Feedback

I asked members of the django-users mailing list who had done development work with both Django and Drupal to provide comarison-oriented feedback for this article. Here are a few of their comments:

Preston Holmes: What happens is that non-developer/technical decision makers will evaluate Drupal and think that all their needs will be met out of the box. When they find out they aren’t, they need to hire developers, and then those developers have to massage existing modules to customize and tweak. Then at some point they have this monster of hodgepodge code. But its that entry point that is the key to Drupal’s popularity. … I don’t think people appreciate how much Drupal development is spawned by this existence of a usable entry point for Drupal by non- developers. (and that it has 1-click installs on hosts, and in general uses the far more common infrastructure of PHP-MySQL).

Javier Guerra: “Many people feel they have to code less to write their own Django-based CMS than to customize an existing one. If that’s your case, you’re a lot less likely to break something when going with Django.”

Clifford Ilkay: “On balance, I’d say Django is the more flexible and powerful of the two but Drupal gives the *illusion* of productivity quicker. With Django, I build up from a base, whereas with Drupal, I seem to spend an inordinate amount of time trying to figure out how to make it *not* do something I want done differently. Django is lean and simple by comparison to Drupal, which is neither lean nor simple. Both have very active and helpful communities. If I had the choice, I’d pick Django simply because I prefer Python to PHP and find Django to be more productive. Prototyping and debugging PHP is very crude by comparison to Python. Much of Drupal’s convoluted architecture and reliance on naming convention “magic” to provide quasi-inheritance is due to the limitations of PHP. I often say, half-seriously, that Drupal is good despite PHP, not because of it.”

Alexandru Nedelcu: Development in Django is a lot easier, because the modules are more generic and the development process is more predictable because of its simpler architecture.

Mike Ramirez: Django has made making web apps almost trivial and the most time I spend now, is on the UI (both Dojo and JQuery make this almost trivial also), rather than the back-end.

Preston Holmes: Both have the problem of often complex dependencies. The biggest problem with Drupal is its relatively fast moving core – so that modules for the “current version” are hard to find, or often abandoned. Also Drupal modules almost always seem to do 80% of what you need.

Milan Andric: Drupal might get you quickly to Point A, but the problem is in customization, pushing a site to do *exactly* what you want (nothing more or less) … My main problem with Drupal has been all the time spent removing stuff. The best tool for web developers right now is a framework, not a CMS. Regardless of the language it’s written in.

catch: Drupal 6’s API was officially frozen 2.5 years ago and Drupal 7 has been in rapid development for more than 18 months, with probably 1-3,000 patches committed by this point.

Preston Holmes: (On the ease of getting the system to do highly custom tasks): Django just wins this so hands down. Here is part of the reason why. Drupals assumptions translate into your code having to jump through a lot of hoops – where as in Django, you make the assumptions and you make the hoops.

Conclusion

Drupal represents a middle ground between framework and CMS that we’ve chosen not to take. Drupal is far more capable than a CMS like WordPress, but also much less flexible than a pure framework. But more importantly, the facts that Drupal isn’t object-oriented, isn’t MVC/MTV, doesn’t have an ORM, and is generally less flexible than a pure framework, not to mention our preference for working in Python over PHP, all contribute to our decision not to use it.

In the end, a good developer can do good work with just about any system, while a bad developer can make mincemeat of even the best system. It’s not all about the platform. But modern tools and best practices in platform design go a long way toward ending up with a cleaner, faster, better-designed architecture that precisely matches the needs of your organizations, with no assumptions or historical baggage to work around.

Note: Public comments below may be incorporated into the document above. Thanks for your feedback.

Update: This post sparked an interesting thread on the WordPress Hackers mailing list.

Update: Drupal developer Nick Sergeant has posted a similar piece: Drupal v. Django

These are my notes and observations on how the J-School got into this mess, why we’ve fallen in love with Django, and how we plan to dig ourselves out.

Bailing Wire

I personally have no formal CS or programming background. Over the course of the six years I managed most of the J-School web properties, I learned just as much PHP and shell scripting as required, on an as-needed basis. Job pressures and constant deadlines never allowed time for pure training.

Even without training, I’ve been extremely productive with ad-hoc PHP. I ended up developing and/or implementing some very effective web apps, such as a course and instructor review system, events database, story submission database, application and review system, online quiz system, etc. – all more or less stitched together with bailing wire and duct tape.

But everything people say about PHP systems not scaling well eventually came around to bite us – our systems are sprawling. Login systems are disconnected, disparate databases that should be unified are spread all the place. Instead of a centralized content management system, we have dozens of loosely connected CMSs. From a certain perspective, this made sense, as much of what we do is not about running a single site, but a loose federation of barely related sites. At a journalism school, students, classes, and organizations demand one publication site after another, few of which have anything to do with one another.

But at a certain point, non-unified systems simply fail to scale. It’s not about traffic, but about consistency. You can’t require students and faculty to create new logins all over the place, and you can’t expect web developers to master dozens of different platforms that work in dozens of different ways. Over the past year, we’ve known it was time to consolidate as much as possible into a single, centralized framework or CMS. The scaling problem is organizational — too many parties, too many tools, too many different ways of doing things, too many cooks in the kitchen. A classic example of PHP spaghetti.

Quite a while ago, we realized that something had to give. We needed to rebuild everything in a single system – one that came with a foundation flexible enough to model everything the organization is and does, and to build any kind of web-based representation or tools we needed on top of that model. We needed something more than a typical CMS. We didn’t want to have to shoehorn parts into places where they don’t belong. We wanted a system to work for us, rather than against us.

Framework vs. CMS

What’s the difference between a content management system and a framework? Look at it this way: If you write in a raw language, like PHP or Perl, it’s like going to Home Depot and buying lumber and nails. You have absolute flexibility, but you’re responsible for every square centimeter of the house you build. If you deploy a content management system, it’s like buying a house – you can rearrange the furniture, but you’re pretty much stuck with the floor plan.

A framework lies somewhere between those two poles – as if you’re buying parts of a pre-fab house. “Now I need walls. Now plumbing. Now kitchen appliances. Now air conditioning.” The nitty gritty stuff is taken care of, but you have full control over which pieces you use and how they fit together. “Now I need a commenting system. Now I need RSS. Now I need search. Now I need form validation.”

CMSs have a nasty habit of becoming inflexible. Editorial staff get tired of hearing “we can’t do x, y, or z because the CMS won’t let us.” CMSs make assumptions about how you work, and about how a site is structured. As long as you work within that set of assumptions, all is well. But try to bend a CMS to do things it wasn’t meant to do, and you open the door to failure.

For example, we do a ton of work with WordPress. WordPress assumes that the basic content type is the story, and that that story has a headline, summary, article body, and maybe a bit of metadata. WordPress is an amazing tool, ideally suited to publication-oriented sites such as blogs and focused publications. But building a custom database application involving arbitrary arrays of fields, or building custom workflow tools, would be a matter of bending WordPress in ways it wasn’t meant to be bent. You may or may not be successful, but you’ll probably never call your solution graceful. You’ll always be working against, not with WordPress’ assumptions about the structure of your content. It’s true that WordPress is much more than just a blogging system – it’s a darn good CMS for a lot of site types. But you wouldn’t use it to be build an equipment checkout system, or an alumni database, or an application processing system. Its limitations come from its assumptions.

In contrast, a framework is a toolkit you use to build the CMS that matches your organization’s specific needs. No assumptions, just tools and conventions you can use to model data structures and create a workflow based on those structures. A framework takes care of much of the heavy lifting involved in the creation of highly customized web apps, so you don’t have to pour countless hours into things like fighting spam, managing fine-grained permissions, building CRUD tools, etc.

Because a framework works at a lower level than a CMS, it generally has a steeper learning curve. In exchange for that learning curve, you get more flexibility as your organization changes, grows, and scales. It’s a middle ground between (on one hand) raw code that makes no assumptions but leaves you responsible for every bit, and (on the other) a full-blown CMS that makes too many assumptions about content types, and often feels convoluted and/or limiting (not to mention bloated, which most major CMSs are).

There are web application frameworks written in many languages. PHP has Cake, Symfony, Zend and others. Ruby is famous for Rails. Perl has Catalyst and Gantry. Python has Zope, TurboGears, and, in recent years, Django. I haven’t done enough work with frameworks to be able to offer meaningful side-by-side comparisons, but I can tell you what it’s been like for a PHP guy to learn Django over the past six months.

Starting From Scratch

Here’s why PHP is so popular: You don’t have to learn much up-front. Start with an HTML page, drop in one measly function call, and you get immediate gratification. Need to do something else? Look up another function and drop it in. Over time, you get a head-full of function calls you can use in a pinch, and get to call yourself a programmer. It’s a double-edged sword. The fact that PHP takes no real study to become productive means there are millions of PHP developers who don’t have a grasp of “real” programming concepts. Like me. That doesn’t mean we aren’t productive, but it does mean we tend to build things in non-optimal ways under deadline pressure. And we tend to build things that turn around and bite us in the ass when organizations and data models grow, change, and evolve, because our code is littered with secrets and mysteries, and because we lack the foundations needed to be “real” programmers. I’m generalizing grossly (and being unnecessarily self-deprecating) to make a point.

For me, learning a framework has been another story. There is no “instant gratification” entry point. You need to have a grasp of the language the framework is written in. For Django, that means learning some Python – at the very least lists, dictionaries and tuples. You need to learn that language’s programming structures – how it handles conditionals, loops, I/O, etc. You need to have some basic grounding in object-oriented programming principles – something I’ve simply never gotten around to. In most cases – certainly in Django – you need to learn MVC (model view controller) or in Django’s case MTV (model template view) principles. And you need to learn the framework itself. How it thinks, how data moves through the system, where to look when things go wrong.

Finally, you need to know how to deploy. Because PHP is built into virtually every Apache host out there, deployment is something most PHP devs don’t have to think about. For Rails or Django, it’s a different story. Suddenly you’re compiling apache modules, building database bindings, configuring the web server… or looking around for a host who’s done all of that for you.

I signed up for an online Python course through University Extension. I spent many, many hours reading documentation, blog posts, and watching tutorial videos. I went through the official tutorials, and struggled to get development environments working both locally and on production servers.

Nothing about it was easy. Which is ironic for a system designed to make you more productive. After years of extreme productivity, I felt like I had been thrown back to Square One, and was starting my web development education all over again. Painful, but much-needed. Only after I started to assimilate and internalize the absolute elegance of the Django weltanschauung did I realize just how bad the systems I’d built over the years really were – both from data modeling and coding perspectives. I needed to completely re-think the way I was doing things. It was time for a hard reset.

After several months of diving in, experimenting, failing, getting frustrated, and finally succeeding, I realized:

• It’s all about the data model. Get this right and you win. Think long and hard about the data model your organization is built around, and build web apps around that. Get this right and you can build any view, any tool that will do right by the organization. Get it wrong and you’ll struggle forever.
• Amateur approaches will only get you so far. Learning on a catch-as-can basis will only get you halfway there. You may be a hero in the short term, but in the long run, real accomplishments come from real programmers. It takes time to learn the bigger picture, but if you do it, you win.
• As a PHP dev, I wasn’t used to having to spend long periods of time reading documentation. I read just enough to get each little job done. Django made me sit back and read everything. To grasp the gestalt of what I was dealing with before trying to move forward.
• As long as you expect it to be easy, you’ll be disappointed. As with all things in life, there’s the easy way, and there’s the right way. The right way takes extra effort, but pays off in the end. Yes, Django will make you more productive and enable you to solve hard problems quickly – but only after you’ve put in the effort to wrap your head around completely new ways of doing things. The paradox is that Django (and Rails) claim to make web development easier. And it’s true – they do. But only after slogging through a long ramping-up period. More work up front in order to do less work later.

I’ve been incredibly lucky to be allowed a big stretch of time to learn Python/Django on the boss’ clock. The key is in getting people on board, convincing them that moving the organization forward means training time is required. You can’t just change languages/systems in mid-stream and expect everything to carry on at the same pace it has been. The larger organization has to feel the pain of the current systems as well, and therefore to support the time/expense of real training.

Why Django?

Since we were already a PHP shop, why wouldn’t we choose one of the existing PHP frameworks? Yes, we could have gone that route, but none of the PHP frameworks have the credibility or momentum of Rails or Django. I don’t want to say that PHP is a bad language, but it wasn’t built as an object-oriented language from the ground up. The web framework momentum today is in cleaner, more O-O languages. And from what we could see, the PHP-based frameworks themselves didn’t look as clean as Rails or Django. We had learned the hard way, and were ready to move on.

To be fair, I’ve only experimented briefly with Ruby on Rails, and am not qualified to say a lot about it. What I do know is that there have been a lot of complaints about RoR scalability and stability over the past few years. I’m sure the Rails community would refute those claims as myths, but for whatever reason, they have been prevalent. And in my reading of half a dozen comparisons between the two frameworks, I haven’t yet found one that showed faster development times or better scalability for RoR over Django. When you don’t have the resources to compare things side-by-side for yourself (how long would it take to learn enough about every framework on the market in order to do a truly objective comparison/analysis?), you have to lean on the opinion of developers who actually have. And from what I can tell, most developers who have tried both prefer Django over Rails.

The fact that the RoR community is much larger than Django’s was certainly a point in Rails’ favor, but not enough to tip the scales for us.

One of the real “wins” for Django is in its automatically generated “admin” interface. Once your data models have been defined, a mini-CMS is built for you around that data model, with all of the form widgets properly reflecting the models’ relationships. If a field is a ForeignKey to another model, you get a picklist of that foreign model’s instances. If a field is a DateTime, you get slick little date/time pickers. If a field describes a many-to-many relationship with another model, the admin gives you a combo box. If a field is designated required, the right database constraints and field validations are put in place automatically.

The Django admin isn’t perfect and isn’t suitable for all tasks, but for the most part it “just works,” and saves you tons of time. In most cases, you can forget about ever having to write CRUD apps. In the journalism context, it means the developer can sit down with the reporter, get a good handle on the data models that describe the feature, and the journalist can start entering data immediately, while the developer continues work on the public-facing site.

People will tell you “Django is a framework, not a CMS.” That’s true, but the presence of the admin means you essentially get a CMS for free, alongside the framework. The admin is a huge win for Django. If the Admin doesn’t serve a particular purpose, no problem – build your own custom workflow on top of the model you’ve defined, outside of the Admin. Rails doesn’t have anything comparable, as far as I know.

On the flip side, Django is currently missing an equivalent of Rails’ official “migrations” system, which helps keep database schemas in sync when an application’s data models change. Right now, it’s a mostly manual affair. However, there are currently three separate Django projects in the works to address this need. Thankfully, all three developers are now working together to bring their projects into a single unified solution, eventually to be bundled in Django core (listen to Russell Keith-Magee on episode 44 of This Week in Django to learn more about why model migrations is an inherently hard problem).

Unlike Rails, no one complains about Django not scaling to very high traffic situations, or about ongoing stability problems. And no one complains about the project being run/controlled by difficult people. Django grew out of the real publishing needs of a real-world news publication trying to do difficult things on tight deadlines (hence “The web framework for perfectionists with deadlines.”) It may look like a small player, but it’s battle-hardened. According to Google Trends, interest in Django over the past year has surpassed Rails. Something’s going on.

(The search above excludes “Reinhardt” to eliminate results on jazz guitarist Django Reinhardt)

To reiterate: I am not dissing on Rails. I haven’t used it enough to have an opinion, and I know it’s chock-full of excellent code and good practices. I just have the strong impression Django has an edge in terms of development speed, scalability, quality docs, and no-bull OSS leadership.

The open sourcing of Django and the creation of the Django Software Foundation opened the doors to wider adoption, and the September release of Django 1.0 – a version guaranteed to be production-ready and stable – will give bean counters confidence.

But nothing speaks to market confidence in Django more loudly than its blessing by Google. Google engineers have been Python-centric for years (Python creator Guido van Rossum works at Google). But when Google launched AppEngine – a cloud-based web app framework – they released it with just one deployment option: A somewhat modified version of Django. In their eyes, Django was not only good enough to run in heavy production, but also deemed easy enough to learn that they were willing to ask developers to learn it if they wanted to use AppEngine. Any other company would have looked for the lowest common denominator and made AppEngine work with the most popular languages/frameworks first, in order to snare the the most possible developers. Not Google – they let their love for the language and the framework speak for itself with the offering (of course AppEngine will support more languages in the future, but it’s been all-Django so far).

And oh yeah – Django has a magical pony.

Digging In

In September, two of us spent a few days at Google headquarters in Mountain View for the first annual DjangoCon – an international conference bringing together heroes of the Django world with developers who traveled from all corners for the privilege. Heard a lot of great talks, learned heaps about Django best practices, met some great people, etc. Drank the Kool-Aid and got all fired up (video from the complete session is available on YouTube).

Shortly after, I realized that part of my Django education needed to involve creating my own pluggable Django application, so set to work on a multi-user, multi-group task list management system called django-todo (yes, I could have built a blog engine from scratch like most Django newbies do, but the truth is I have no intention of ditching WordPress any time soon, so doing that would only have been an academic exercise).

I was able to finish Django-todo in about a week, and eventually got it up onto Google code, where it’s gotten some good feedback and a few dozen downloads. Even got a mention in episode 40 of This Week in Django!

Django is renowned for its excellent documentation. It’s true that the documentation is incredibly thorough and well-presented, but from a newbie’s perspective, I often found it frustrating, since you don’t know what to search on to find answers to specific problems. Trying to figure out how to make global variables available to my templates, I found nothing, because the answer is in Django’s “context processors.” All well and good if you already know that that’s what you need to search on. Similarly, quite a few of the code examples in the docs show interactions with the Django API through Python shell examples. Those examples do a good job of showing how the API interaction is occurring, but offer no help for the newbie trying to figure out how to actually implement things.

For example, I found the docs on Django’s pagination functionality completely inscrutable. Shell examples? Why should users be submerged by that level of abstraction?  Show me a web-based example – with working snippets from model, view, and template -  applicable to common scenarios, which I can use as a starting point. Fortunately, I found an excellent blog post on Django pagination in the wild. After corresponding with the author a bit, I was able to modify the official documentation (which is bundled with every Django checkout), submit a Trac ticket, re-build the docs, submit an svn diff patch, and get it accepted into the official docs — which is why you see it there now.

Hosting on Birdhouse

If there’s one big downside to working with a framework based on a language that lacks the widespread support of PHP, it’s in deployment. When you work in PHP, you never have to wonder whether your host will support your app or framework of choice. PHP is like air or water – it’s just there.

If you want to deploy a site with Ruby on Rails, it’s a bit more difficult – you’ve either got to run/manage your own server, or find a web host that supports RoR out of the box. Luckily for Ror developers, the world’s most popular web hosting control panel cPanel supports RoR natively in recent builds.

Django folks don’t have it so good. cPanel doesn’t yet support Django (though I’ve filed a feature request). So if you’re not managing your own server, you’ve got to choose from among a fairly limited selection. Trouble is, I already run a cPanel-based web hosting service. If I was going to be able to deploy any of the sites I was already starting to build, I wasn’t going to be able to wait around for official Django support in cPanel – I’d have to roll my own.

Fortunately, this problem had already been largely solved by @mandric, who has been the hidden mentor/kick-starter behind Django activity at the J-School, and who wrote up an excellent summary on setting up Django with cPanel. Based on his notes, I was able to get Django working on a cPanel host myself, and took my own notes on the process. Today, Birdhouse Hosting officially supports Django hosting. Looks like the first official Django user on Birdhouse will be a commercial wine database with a fairly sophisticated data model. More on that as the project comes along.

A Django Shop

Long story short, we finally came to a decision: The J-School is now officially a Django shop. For better or worse, we’re embarking on the long road toward unifying all of our toolsets around Python and Django. Which means A) Finally creating that mythical Grand Unified Database that represents the entire organization and B) Rebuilding a whole lot of tools around that new database. There’s a lot more web to the J-School than meets the eye — we’ve got a pretty expansive hidden intranet and a whole lot of less-visible sites to support that most J-School site visitors never see. It’s going to be a ton of work, and there will be plenty of bumps in the road, but I’m excited — not just at the prospect of getting all of our stuff into one bucket for a change, but at the learning opportunities this decision represents.

Despite the many frustrations I’ve had with the early stages of the learning process, Django development is fun. There’s nothing like watching things “just work,” the feeling that “web development was meant to be this way.” I look forward to not having to struggle against the expectations and limitations of a monolithic content management system, or of the ghostly chains of bad historical decisions. The process is going to rock as much as the final result.

Wish us luck.

George DeMet Owner, Palantir.net
Jeff Eaton Lullabot
Tiffany Farriss Pres, Palantir.net
Mike Essl Owner Operator, mike.essl.com
Matthew McDermott Principal Consultant, Catapult Systems

The perennial question on every web dev mailing list: What CMS should I choose? Expression Engine made a huge splash at this year’s SXSW, but the Drupalites were out in force as well. This panel basically boiled down to MS Sharepoint (missed this, but not interested), EE, Drupal, and observations on a smattering of other systems. In a software category that offers around 600 choices, it’s impossible ever to represent the whole picture with anything approaching accuracy, but the conversation was still useful.

DRUPAL

Built by the people who USE it (which has upsides and downsides). Dmitri – 12-year-old who does amazing and scary things with Javascript. Sony BMG, Lifetime, etc. contributing code back in. Oh, and Belgians (we got a lot of them).

Major sites/launches: Fast Company had to roll in around 100,000 person social network, and to port in all their existing content. Lifetime Television. Lots of artists sites. MTV UK with lots of social networking. Inc. Mag. Us Mag. DC Comics + IBM recently launched a site. The Onion – The guy who built their Drupal site got hired by The Times.

3000 plugin modules.

Under the hood – six layers

Theme layer
Views of content
Content
Users
Modules
Drupal core

Content layer: news, blogs, wikis, images, etc.

Building a drupal site is a process of identifying: 1) Who’s going to be using, 2) What kind of content will they be using

User roles, new content types. Start layering in capabilities and it all comes together nicely.

Drupal Sucks if “I just want to make a page” or “I want to make the next Twitter” “I just need another static site” or “Just another blog”

Drupal rocks if Lots of user-generated content, communities, many kinds of content, many views of content, Open APIs, web standards

Sony artist sites: Massive traffic boost, rolling out new artists fast, supporting open source, playing “It’s Britney, Bitch” on page load.

EXPRESSION ENGINE

Mr. T site in 2004 – built with not a spec of custom PHP – everything needed was built in.

You can define all kinds of custom fields for every content type. You can move custom fields around, define different sets for different content types.

Paid support staff in the forums. That’s huge. You know you’re going to get help when you need it.

See my EE notes for more.

EVOLUTION OF CMS CHOICES
Art Institute of Chicago Case Study – they’ll use whatever CMS makes sense for any give project.

Round 1: Dreamweaver + custom CMS

Round 2: Serena Collage – Requires you to set up “contribution layouts” (content entry views). Collage is very metadata-driven. Spits out static HTML like Movable Type. Allows custom workflows. Includes version control, breadcrumbs, links as assets. Downsides: Expensive licensing, excruciatingly slow interface, not Mac-friendly, navigation, training developers, interferred with PHP code, no support for dynamic frameworks, end of life product.

Round 3: Drupal – They needed to ramp up to handle many thousands of images with smooth back-end. Lots of meta-data for each image. Fell in love with Drupal – solved all problems. Needed ability to create lots of individualized artist sites with completely different themes (themed sub-sites)

Drupal pros: Powerful templating, remote data handling, great usre management, JQuery integration, solid, flexible framework, ability to write own modules. Drupal cons: Reverse proxy difficult in D5. Not great at handling edge cases (one-off exceptions).

Drupal is like getting a garbage truck full of legos – to really master it you need to learn what all the pieces are.

Every time someone would ask how to do something fancy in Drupal, they’d get a long answer, then the EE guy would say “EE has that built in.”

Is there a trend toward abstracting away the developer role.

Chirp – like Tumblr on your own server.

Sharepoint: License is $25,000 !!! (but they’re working on cheaper licenses)

Now powered by code igniter — fully objeect-oriented OSS PHP framework. The ingredients of EE left uncooked. So now EE is built on a framework. This is a big deal. Code has merged, communities are merging. Faster, easier development for Ellis Lab and for 3rd party devs. Instant increase in capabilities of both systems.

Database: Abstracted querying. Tools, such as the database forge class. YOu can manipulate your database at the code level.

New libraries: Session, HTML generation, form generation, unit testing. JQuery and javascript libraries.

Obsessive about making CodeIgniter the fastest in its class. No bloat! Super efficient. Performance is highest priority.

Code Igniter is fully MVC.

EE 2.0 control panel just rocks. Super well designed, fast, smartly laid out. I want this! Includes a wiki engine. Capable of building arbitrarily complex tools on the back-end. Control panel is completely redesigned and super-slick. Infinitely customizable.

Want to talk to people who have worked with both Drupal and EE. How do they stack up? I think you get a lot more “out of the box” with EE. And the support is supposed to be fanatical.

Timing is great – we’ve been debating on frameworks (Django) vs. CMSs (primarily EE) for the past week at a “web summit” the three of us J-School web devs were holding. By letting us drop all of our existing PHP tools into place without modification, EE could provide a migration path that would be very painful if we were to move everything to Django.

For the past year or so, I’ve virtually never found a case where anything I wanted to do with WP hadn’t already been solved by an existing plugin or tweak to template logic. But amazingly, I couldn’t find anything to override the default future post behavior. Posted on WP-Hackers about the problem and got a few solutions volunteered within a few hours (there’s nothing like a vibrant open source community). By far the most elegant was this one from the magical Ryan Boren (same guy who planted the semi-secret WordPress t-shirt geocache):

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<?php
function setup_future_hook() {
 // Replace native future_post function with replacement
 remove_action('future_post', '_future_post_hook');
 add_action('future_post', 'publish_future_post_now');
}
 
function publish_future_post_now($id) {
 // Set new post's post_status to "publish" rather than "future."
 wp_publish_post($id);
}
 
add_action('init', 'setup_future_hook');
?>

Stick this in a php document in your plugins folder (remember not to include any whitespace after the closing php tag!), activate it, and create a post with a future timestamp. The post’s status field in wp_posts will be set to “publish” rather than “future” and it’ll go live on the site immediately.

You can also download this as a ready-to-go plugin.

Ryan’s too busy to host this trivial but super-useful plugin himself, but invited me to. I’ve submitted it to WP-Plugins and am awaiting a response – should be available there as well before long.

Created an ID for myself at MyOpenID (though you could use any OpenID provider). Doing so gave me an identity URL through that provider. But here’s the dirty little OpenID secret that shouldn’t be a secret: The protocol supports “delegation” — by adding a couple of meta lines to the header of any URL you control (the birdhouse.org homepage, in my case), that URL can stand in as your identity URL. So when I typed “birdhouse.org” into a blog that supported OpenID earlier today, it fetched that URI and read its delegation headers. It then knew my “real” identity URL at the provider. The provider was able to determine that I was already logged into their service and pass “true” back to the blog I was trying to access. If I hadn’t been logged into MyOpenID at the time, I would have been prompted to log in there first, as a middle step in a seamless process.

Once authenticated to the blog, which had the WordPress OpenID plugin installed, a user-level account in that blog was created automatically for me. The admin could then escalate my privileges to admin or whatever, and I’d still only need to type “birdhouse.org” to log in there as admin. And you can’t. So there.

Distributed single sign-on works. Totally elegant.

A while back, Six Apart launched TypeKey, a single sign-on mechanism first made available for Movable Type blogs. TK never really took off, for a couple of reasons. First, most blog owners had already discovered that requiring any kind of sign-on had a chilling effect on blog conversation — any barrier to commenting was too high, and tended to stop casual “stopper-by” conversation dead. Second, a lot of people didn’t want to put all their identity eggs in the Six Apart basket, didn’t feel comfortable having a corporation behind the critical task of identity maintenance. That assumption was bogus – TypeKey was always an open API – but a lot of people didn’t feel comfortable with it. TypeKey isn’t dead, but there aren’t many sites using it.

Lots of identity conversation at SXSW this year, with OpenID emerging as the “final” solution to the distributed identity problem. Ended up not attending that panel, but did get to eat sushi with Kaliya “identity is a commons that no one can own” Hamlin, who (by some accounts) is single-handedly responsible for wrangling the monolithic corporate gargoyles (who all wanted to sell the world on their own proprietary silo identity systems and end up falling into the same hole that swallowed TypeKey), tying them up in a room and making them take mushrooms and hug until they agreed to adopt OpenID. Now even AOL is an OpenID provider.

Free love works!

Thanks Milan

Katovich Law Group assists clients in integrating sustainable, socially and environmentally responsible practices into their businesses at every level.

Katovich Law came to Birdhouse as a Plone site. The Plone CMS embeds its own server, and is therefore incompatible with Apache (without doing fancy port re-routing). Rather than go down that road (and because circumstances were going to make it very difficult to get a raw data dump or even a Plone backup from the old host), I offered to port the site to a more common/compatible CMS (Katovich had no particular attraction to any particular CMS – they were on Plone by circumstance).

Since the site had a pretty straightforward structure, decided to see if I could pull it off in WordPress (it’s what’s for breakfast). There were a few rough edges where WordPress’ blog orientation made things a bit tricky, but overall, the experience underscored my confidence in WP’s flexibility. Had never had cause to dig into the parent/child relationship of WordPress pages, but found them an incredibly easy way to organize hierarchical material and get logically nested URLs and nav sub-menus with zero effort.

Actually wanting to start mastering messing with Drupal, but this was a useful experiment.

CMSs are a funny category of software. When you go to choose a word processor, you’ve got three or four serious options to consider (but it generally comes down to Word). Image editor? Maybe a dozen (but it generally comes down to Photoshop). There are usually more options for server-side web application software. Survey package? Maybe a dozen. Blogging platform? Again, maybe a dozen (but it generally comes down to Movable Type and WordPress). But the game changes immensely when you start looking at content management systems.

OpenSourceCMS lets you try out demo installations of around 50 free CMSs. cmsmatrix.org lets you create feature comparison tables on, wait for it, 643 different content management systems, ranging in price from $0 to $200k (at the higher end, you’re paying for a team of guys in jumpsuits to come to your organization and live under the desks).

On the university web development mailing lists I’m on, “Which CMS is right for us?” is the single-most frequent topic of discussion, by a wide margin. Everyone needs one (or thinks they need one — I think they’re often deployed unnecessarily), but no one knows which way to jump. There is probably no harder software choice you can make, for several reasons:

1) Unlike, say, email clients or HTML editors, you can’t switch CMSs on a dime. Once you buy into a CMS, it’s very hard to switch to another one. You have both data lock-in and systems lock-in (this alone is a very good reason to think hard about whether you really need a CMS or just think you need one).

2) Actual needs can be very difficult to quantify correctly in advance, and difficult to correlate with product descriptions.

3) It takes a lot of time to evaluate the options – you really need to install a CMS and work with it for a good while before you know whether it’s going to work for you. But these things are not plug and play — it’s not like installing a piece of desktop software and being able to figure it out by studying the menu options. Bringing a new CMS up to the point where you can even duplicate your normal workflow could take days or weeks of research and experimentation.

4) There are simply too many to choose from. If you have to go through step #3 for each of half a dozen finalists, you’re talking about a major expenditure of resources, just to make the decision.

CMSs are complicated beasts. Once you’ve made your decision, the real development work begins, which generally involves understanding that system’s internal workflow, node hierarchy, templating and conditional languages, plugin options, data structures, etc.

So why are there so many CMSs on the market? Two factors contribute to the reason. One, because choosing from among existing systems is so damn hard, and so time-consuming. And two, because organizations’ needs differ so radically. For a CMS to encompass everything anyone might want to do, it has to emerge “out of the box” as a very generalized, rarefied system. There’s no such thing as a CMS that works the way you want it to work as soon as it’s installed. Making a CMS sit up and do the tricks you need it to do is going to require development time, period. Faced with these options, many organizations with in-house programming talent just throw up their hands and say screw it, let’s just build something. How hard can it be?

In a sense, they’re right. A basic CMS is a pretty trivial programming task. Create, update, and display content in a database. But the devil is in the details. You probably need a user management system, with authority roles. You need a workflow system to move content through draft, review, and published states. You need a system to manage incoming non-text media. You might need the ability to run surveys and polls, or to handle other kinds of form data. If your content takes public comments, you need to be able to manage those comments on the back end (and deal with comment spam as it arrives). You need it to be able to generate email to authors and editors when content states change. You might want built-in blogging, and everything that comes along with that. You want RSS feeds coming out of its ears. You need to deal with all those nasty curly quotes and other special characters being pasted out of MS Word into your forms. You need author bio pages. You need to accommodate static content sections, like “About This Site” and “Contact Us” (again, with spambot protection). You need form field validation to make sure authors are putting the right kind of data in the right places. You might need to accommodate multiple-author bylines. You need pagination for longer pieces of content. You need both a back-end and a public display system. You need “E-mail this page to a friend” and “Print this page” functionality. You need to think about security at every step of the way. You need to be able to preview content before publishing. You need context-sensitive usage documentation for all of the authors and editors. You need the ability to modify timestamps and story slugs. You need to come up with a way to create clean URLs. You might need multi-level search (site-wide and project-wide). If you’ve got multiple developers, you need a code checkin/checkout system (CVS or SVN). The list goes on, and every little piece piece needs to be built by you — hopefully with enough forethought that it scales outwards nicely as you add features.

So once a group has put in all of the time required to get the results they want, they decide they may as well release it into the wild, either as open source or commercial software. And “walla,” we now have 643 systems to choose from.

Our case called for a multiple team layout (the four participating universities), each with its own project. Some projects would have custom Flash interfaces, others would not. We needed projects attached both to schools and to semesters, so we could refresh the whole thing next time. We needed to handle multimedia content from many sources, in an elegant manner that didn’t require users to mess with paths or HTML. We needed the ability to have user-selectable layout templates nested within master templates. We need the ability to embed custom HTML blobs, to alter story order, to embed custom database applications with story bodies. And so on.

Given enough time, we probably could have found an existing CMS that could have been hammered into shape. But the clock was ticking, and our experiments with existing CMSs were showing that the research and discovery process was going to take too long. If we built it ourselves, we reasoned, we’d grok every square millimeter of the project, and be able to tweak anything without having to study docs or hang out in forums.

After several weeks, we drew a line in the sand and said “We can build this.” Almost immediately, we started getting feedback from people who had been in the same boat, and either vehemently agreed or disagreed that we had taken the right approach. Arguments in favor were pretty much in line with our own reasoning. Arguments against included things we already knew, such as the fact that we wouldn’t benefit from free security and feature enhancements and would not be able to take advantage of existing plugins and modules that could save us a lot of time. All true, but in the end, we have no regrets. We even got accused of having a bad case of Not Invented Here syndrome (which simply wasn’t true – we were just trying to do the right thing for the organization and for ourselves, and would have gone with an existing solution if we had thought it would be easier).

Ultimately, I think the two-month timeframe it took to construct our CMS from scratch was about what it would have taken to choose and implement an existing CMS, so it was probably a wash, time-wise.

The fact that major news organizations have already inquired about adopting our CMS makes us proud. Ironically, in the end, I’m still not convinced that the site we produced warranted a CMS at all. We spent months developing our system. Formatting 100 or so pages manually inside a simple templating system would have taken a fraction of that. The number of pages you ultimately have to manage, the frequency of updates etc. needs to be considered very carefully when deciding whether you need a CMS at all (don’t adopt a CMS just because everyone else is doing it, or because you think it will ipso facto make your life easier – ye olde permissions-based filesystem, favorite HTML editor, and templating system remain a great way to manage large amounts of content in many contexts — that’s what we did for the new J-School site, and we couldn’t be happier).

One thing is clear: Anyone who could really master the whole CMS minefield could make a great living as a CMS consultant. Organizations need good, educated advice from people who have been in lots of trenches. And they need it bad.

I’ve left a whole other category/approache out of this discussion: Frameworks (Ruby on Rails, Cake, Symfony, Code Igniter, Django). We did spend time talking about frameworks vs. CMSs, and in retrospect, I kind of wish we had used one rather than build completely from scratch. But like CMSs, frameworks require quite a bit of exposure developer/experience before you can really be productive, so the same time constraints we faced pretty much kept frameworks off the table. We went with plain old PHP, which is what we know best. We’re now developing some in-house Django talent, which will probably guide our next big project.

Has your organization been through the build vs. buy CMS decision? Which way did you go? Any regrets?

* No, we weren’t responsible for the visual design, just the CMS.

** When I say “buy” I’m being colloquial – if we had not done it ourselves we would have used an open source CMS such as Drupal.

Technorati Tags: cms, contentmanagement, frameworks, php

As part of the planning for the initiative, the five participating deans drafted a vision for change that seeks to renew the mission of schools of journalism much the same way that schools of business, medicine and law have renewed themselves at different junctures in history.

Online now is a starter/brochure site, and currently all of the advance reportage is happening through external blogs. But a compadre and I (yes, we have two webmasters at the jschool now!) have been hard at work building a custom content management system* to meet the project’s fancy multimedia and nested template needs — the largest pure programming job I’ve ever been involved with, and the first time I’ve done any kind of team programming — a very satisfying experience. We’ll be rolling out the “official” site on top of our CMS later this summer. For now, the reporting fellows are scattered all over the globe, gathering material.

The project was recently blogged at Dan Gillmor’s Center for Citizen Media, at Boing-Boing and also at Utterly Boring, though the interesting stuff is yet to come, once the story packages are completed by the fellows and we wrap up the CMS.

* Will have to post separately sometime on the old build vs. buy CMS question.

Interesting to learn how template inheritance works in WP – it’s not at all obvious from studying template code which parts of a site will be handled by which templates. For example, a link to a content page will be handled by the index template unless a single.php template is present, in which case the exact same link will be handled by that template instead. And so on. Unintuitive at first, but it’s part of what makes the system so flexible.

Biggest hangup was in trying to properly format content that included HTML or PHP snippets (text formatting options in Movable Type are much better at handling this kind of thing). Solved with Priyadi’s Autoescape plugin.

Anyway, an interesting learning experience. Also currently working with another WordPress site: Dan Gillmor recently launched the Center for Citizen Media out of the J-School, and I’ve been working with him on that.

Mullenweg, FWIW, is one of the sweetest, most charming guys you could hope to meet. Was very interested to learn that c|net uses WP (with a simple caching plugin) for a huge number of public publishing projects. Last I heard was that c|net had basically invented the massive Vignette CMS. Very interesting to learn they’ve basically abandoned their own baby in favor of simple, lightweight, open source tools.

Yesterday hooked up with Simon Clarke of Vara Software — I know Simon from the Adamation days (can’t believe there’s still a web site there), where he and another engineer were responsible for personalStudio (BeOS video editing application, later released for Windows). Vara is doing some really cool stuff, and I’ve decided to stop using Channel Storm’s LiveChannel for J-School webcasting and switch to Vara’s WireCast. It’s that cool, and won’t result in any loss of functionality. Also got a personal demo of Vara’s VideoCue — teleprompter software that also takes camera input, lets you mix in images and titles, output to QuickTime, and optionally post results directly to a blog. Skookum stuff.

Just emerged from a couple of sessions on building Dashboard Widgets and am totally fired up. Just need to clear a few days (yeah, right) and go for it.

The J-School has been looking forward to MT3 for a long time, hoping for new features that would make it easier to manage the 17 blogs and 260 authors (with ~50 new authors added per semester) we currently support. What we didn’t anticipate was the new licensing scheme that could not only become prohibitively expensive, but a logistical nightmare as we try to track and pay for licenses for each new author, semester to semester. diveintomark has an excellent piece on why the “free enough” approach MT takes isn’t enough. Even if there’s a free version, tie yourself to a corporation and you’re subject to all their whims, prat falls, and unfortunate licensing decisions. Unless SixApart responds soon to my query on custom licensing, we’ll either be moving on to WordPress, a homebrew PHP/MySQL solution, or all of our blogs will be integrated into whatever CMS I choose for the rest of the J-School site this summer.

The licensing issue doesn’t apply to birdhouse — SixApart still offers a free version for non-commercial purposes. Disappointingly, MT3 offers almost no new features beyond comment registration. That’s okay – I’ve seen software revved to major numbers for minor changes plenty of times, and I wanted some real solution to the comment spam problem. So I ran the upgrade tonight. A few technical misfires (apparently not uncommon) — finally succeeded by installing the full version rather than the upgrade and then running the database upgrade script. Signed up for TypeKey, and received a token to drop into the new and improved back-end. Added the new DynamicComments directive to mt.cfg. But wait — after integrating the new comment registration tag into my templates, things fell apart. Not only were existing comments hidden from view, but when users clicked on the “Sign up to comment” link, they were told that birdhouse was not registered with TypeKey (it was). Screw it. Backpedal. Restore the original MT directory; fortunately it still worked, even though the upgrade script had modified some database structures.

MT3 is almost certainly a no-go for the J-School, and I’m increasingly skeptical about using it for birdhouse. There seems to be an MT –> WordPress exodus afoot, and I’ll probably join it. Lots of content out there on migration strategies.

I had read various theories about this in the MT forums. The most common suggestion is to not use MT Modules and includes. Instead, generate include files via Index Templates, then suck the output into your pages via PHP, SSI, or whatever you like. By putting re-usable content into index templates, custom MT tags still get properly parsed, but MT won’t need to generate the same data for every page each time the database is modified. Instead MT will crunch the include file once, and the http daemon will include it in the page at request time. I replaced all of my MT includes with PHP includes. Result: almost nothing. Post and comment time was virtually unchanged. MT modules were not at fault.

Next I tried removing everything fancy and stripping the templates down to “factory install.” No significant change.

A few times in the past, I’ve modified the timestamp of entries, which resulted in their numerical order not matching their chronological order. I’ve often wondered if this fact was causing MT to do more work. The only way I could think of to test this cleanly was to export all entries, create a new test blog, import all entries (thus re-assigning the post IDs to each entry), and testing post and comment time again. Some success! The “virgin” blog took “only” 62 seconds to generate a new post, and 40 seconds to leave a comment. Better, but still unacceptable. If I were to write a blogging system myself with PHP, post and comment time would be virtually nil, even if generating static pages like MT does. I was looking for something closer to five seconds.

Thinking about what one user had said about the <MTEntryArchive> tag being slow, I went into the Archiving section of the config and disabled Monthly and Category archives. Unfortunately, I now got a build error with my Archives include, since MT is no longer maintaining these archive types. So I made the Monthly and Category archive lists into static PHP includes and tried again. Whoa! Post time decreased to 18 seconds and comment time to 15 seconds. Not quite what I was after, but it was at least within the realm of the acceptable.

However, this means that new posts are no longer added to the Monthly or Category pages automatically. My first thought was to use the MTRebuild plugin to rebuild those archives once a night via cron, but of course that won’t work since I’ve disabled archiving for those two types. They can’t be rebuilt if MT isn’t aware of them. That means I’ll have to rebuild them manually from time to time by turning archiving on for those two, rebuilding, and turning it off again.

So. I’ve achieved a significant speedup (leaving comments should be less frustrating now) but at the expense of some of the cooler automated features. I still consider this a big flaw in MT’s design, and hope it will be worked out in MT Pro.

2) Blogging software assumes that the person who publishes the story is also the author. But in our case, we’ll have a bunch of people writing but only two “techie” people publishing their stories. We need actual author bylines, and don’t care about who actually posted the piece. There’s no way to add a “byline” field without hacking the actual MT engine and modifying the database. Go down that road and you instantly cut yourself off from future updates (you just forked the code). I want to limit modifications to things I can do with templates and plugins. To get around this, we’ll simply put the byline into the story body manually. Inelegant from a database perspective, but it will get the job done.

I’m beginning to realize how flexible this could be as a generic publishing solution for lots of j-school students – it’s near perfect – MT just needs a few more generic CMS features to replace the more specific blogging features.