Building multi-sites with dbuild.py

Roman Zimmermann's picture
Roman Zimmermann

Here at more onion we're hosting most of our client's sites ourselves. This means we are hosting lots of sites which share most of their modules (ie. our standard Drupal installation). Reproducible setups, fast builds, sharing-module code among many sites -- This is where our new build-tool called dbuild.py comes in.

It started out as yet another wrapper around drush make and drush site-install but became a fully fledged download-extract-patch-symlink tool in the end.

Differences to drush make:

  • Different folder structure: All modules are extracted outside of the webroot and symlinked to all sites that should use them.
  • Built-in and explicit support for multi-sites.
  • Facilitate code-sharing among sites: There is only one copy per module and version.
  • Fast rebuilds: if you change one project only this project needs to be rebuilt.
  • The "make" files are all in JSON and there is one file per drupal tree + one file per site.
  • No dynamic versioning: Only fixed versions of projects are supported (except branches of git repositories).

Curious?

Pitch in on Github!

Update

Perhaps we'll take a look at DSLM - Drupal Symlink Manager next to see if we can re-use it for managing our symlinks.

Comments

How do you manage the "drush updatedb" calls for all those sites when a module or core needs updating? Do you keep a master list of installed sites and loop over each one? How do you use it to manage dev vs production sites, for reviewing changes prior to final deployment? Also, I presume you still keep the codebase in git it something similar?

Roman Zimmermann's picture

  • Currently we use a simple bash for-loop to loop over all sites. Something like:
    for s in */settings.php; do (cd $(dirname $s); …); done -- But indeed there is a master-list of all sites: since each site has it's own *.site.json config file I could simply loop over those too. (I really need to add example setups to the repository on github!)
  • For our custom code we have git-repositories, as well as for the build-files. Since our build-tool "knows" about all modules and versions there is no need to keep the whole drupal-tree in git though. To build a project we need only two things: A checkout of dbuild.py and a checkout of our build-files.
  • Deployment happens by checking out the right branch in git and running a dbuild.py -u install $site; drush updb. We use a 3 branch model: master (main integration branch), staging (for our staging setups) and stable (for our live-servers). Before changes are merged from staging into stable they undergo some manual and automatic testing. Naturally we do those tests with a copy of the live-database that we sync via drush db-sync.
  • I hope this answers your questions … ?

Nice work on this, i've actually been working on a similar tool, also in python.

Mine uses yaml files instead of json, and also integrates with ansible, but there are a few similiarities.

Roman Zimmermann's picture

I thought about using YAML too. Maybe I'll have a go at it at some point.

Do you have your tool online somewhere?

Not online just yet, trying to get it to a good working state first. Will post a link when i do though.

Add new comment