Add Comment
Source
News
Bookmark/Search this post with:
Delicious Delicious
Digg Digg
StumbleUpon StumbleUpon
Facebook Facebook
Google Google
Yahoo Yahoo
Technorati Technorati
9
Saturday, April 9, 2011
By admin
Building a Drupal project using git is different than building Drupal itself, and requires its own workflow. I’ve been kicking ideas back and forth Sam Boyer lately about how to make this process take advantage of all the Git power, but also be newbie-safe and as frictionless as possible. I think what we’ve come up with is pretty good: there’s even code written! The process I am going to explain allows the following:
Git-based updates for Drupal core and contrib
The ability to patch/tweak core/contrib without the complexity of vendor-branches
Portability for local development or Git-based deployment
Unrestrained custom development: feature branches, tags, multiple repos
Safe patterns that minimize conflicts and provide a clear resolution process
Pretty cool, eh? Expect a larger manifesto post from Sam in the near future, but for now here’s where we are heading.
Using Two Git Remotes: upstream and collab
In a vanilla git situation, cloning an existing repository creates a single remote called origin, allowing you to push and pull code to/from that remote. However, a single origin isn’t going to be enough to enable a best-practice Drupal on Git workflow. Why? Well, any good Drupal project should start with a clone of the canonical Drupal core project repository (or a repo that tracks core like Pressflow). This lets you pull updates for security and easily contribute back any innovations you make. Anything else is starting off on the wrong foot.
However, this immediately creates a problem because any project is going to need to add code in addition to core. Unless you’re Dries, webchick (or davidstrauss) you’re not going to be able to push back to the single origin, meaning you can’t work as part of a team or use the power of git in any deployment workflow. No bueno.
Luckily this is something git was explicitly built to handle. The answer is to take a small step beyond the vanilla git workflow, and create two remotes: one for upstream, and one for collab. As you might have guessed, you’ll use upstream as a “pull-only” source to get updates and make patches, while collab will hold your custom modules or themes, and allow you to work with a team and implement git-powered workflows.
The actual host for collab can be anything. You could use your own private repository, github (public or private), or even a drupal.org hosted sandbox if you don’t mind your work being completely public. Likewise, your upstream could be any valid source. Drupal core from drupal is always a good choice, but any repository that starts with the canonical drupal history is also valid. You might want to use Pressflow, or maybe your team maintains its own “drupal-plus” repository (e.g. a distribution or quick-start set) which you use as the upstream for projects.
Contrib Modules as Git Submodules
A best-practice workflow will follow the same pattern for Drupal contrib as we’ve described for core: allowing project builders the ability to pull upstream updates and easily contribute back their changes if they want. There’s a problem though: git.drupal.org necessarily separates every contrib module into its own repository. If your project started off as a clone of Drupal core, how can you include a separate repository for Views?
The answer is Git submodules, which are designed to handle this specific problem. However, these are an advanced feature, and it’s important for us to have a consistent pattern for using them.
Luckily the use-case for contributed modules and themes is consistent, and the commands you’ll need to add them as Git submodules — as well as updating them, — are the same every time. In the event that you need to apply a patch or make an enhancement ahead of the upstream maintainer, the same process for adding a collab remote will work just the same.
Custom Development in collab
The particular development for your project happens directly in the primary repository, and is tracked in the collab remote. This lets you work with a team, taking full advantage of feature-branching, local development, and branch/tag-based deployment workflows. With the small change of using collab where you’re used to using origin, the git workflow of git checkout, add, commit, pull and push works the same as ever.
This also means you should be able to use your favorite Git GUI or other power tools with no problems.
The only complication here is the case where you have multiple developers who are adding Git submodules as per above. In that case, in addition to pulling code from collab as usual, it is necessary to run the git submodule update command, and potentially rebase your code if you’ve added the same submodule as someone else and have a tree conflict.
Visualize It
Dog workflow
In this version we have the on a server. It’s main upstream is the official git.drupal.org/project/drupal.git and its main collab is on github at sitename/drupal.git.
Additionally, we have added views, wysiwyg and jquery_ui as submodules from drupal.org, and the tinymce library from github. We have created a collab repo for jquery_ui because we needed to update some of its libraries.
The sites custom module(s) and theme are stored in the primary collab repository.
Introducing dog
The workflow described above is safe and solid, but running all those git operations is a practical nightmare:
Repetitive stress injury is no laughing matter.
Missing a step or making a typo means you’re at risk.
Git has you all the info you need, but quickly assessing the status of a complex repository is a multi-step process.
As I’m fond of saying, human beings are really bad at repetitive rote tasks. It’s not what we evolved to do, and we’re unhappy and error-prone when subject to those conditions. Computers, on the other hand, love repetition and rote tasks. So let’s make the robots to the $&*%‘ing work!
dog = a Drush extension for “Drupal on Git”
The Drupal project already has a wonderful robot helper tool in Drush. Since the patterns we are describing are completely regular, this is a perfect use-case. Better yet, code is online here:
http://drupal.org/project/dog
Contributions are encouraged. As of right now, here’s what Dog is specced to do for you:
dog-init [—upstream] [—branch] [—collab] [<directory>]
Initializes a new local project repository for building Drupal on Git.
—upstream defaults to latest major stable branch (e.g. Drupal 7.x), accepts drush dl style shorthand for drupal.org sources, or a full git url for using non-drupal.org remotes.
—branch local branch name; defaults to master
—collab remote collab repository; defaults to upstream
<directory> where to make the repository locally; defaults to the repository name of upstream
Example: drush dog-init —upstream=6.x —collab=git://github.com/joshk/my-drupal-project my-new-drupal6-project
dog-dl [—collab] [<project>] [<destination>]
Downloads a contrib from Druapl, sets up submodule and updates the main collab repo with the new information.
—collab optional collab repo for this contrib. Necessary if you don’t have write access to the drupal source and intend on making local changes. Can be added later.
<project> project from drupal in drush dl style; also accepts a full git uri to support non-drupal remotes
<destination> destination for the module/theme; defaults to sites/all/modules or sites/all/themes
Example: drush dog-dl views-6.2.x
dog-collab [<uri>] [<directory>]
Add a new collab remote to a module, theme or main repository if one was not set up initially.
<uri> the location of the collab remote
<directory> path to the module or theme directory, or drupal rood; defaults to current working directory
Example: drush dog-collab git://github.com/joshk/my-views-patches sites/all/modules/views
dog-catchup
Pulls collab updates and automatically brings new submodules in/up to date.
Example: dog-catchup
dog-upstream-update [<directory>]
Pulls upstream updates and commits them to the collab remote if one exists.
<directory> optionally specify a directory to update; defaults to current working directory and works recursively.
Example: drush dog-upstream-update /sites/all/modules/views
dog-status
Parses main repository and submodule status and presents an overview of the entire project.
Possible alias: dog-vet
dog-remove [<directory>]
Completely removes contribs added via dog-dl and pushes that change to collab
<directory> directory of contrib to remove
Example: drush dog-remove sites/all/modules/views
Possible alias: dog-gone
Project Manifest
In order to maintain the integrity of a project and insure portability for local development and deployment, dog maintains a manifest file for the current local project. The allows us the potential to dog-rollup a project into a manifest file and then dog-rollout the same project elsewhere in a similar fashion to drush_make.
However, the dog manifest is entirely git-centric and must include the upstream and collab information. It will likely also be stored in JSON format.
In the longer-run we hope to see more convergence between drush_make, the dog manifest file and possibly the site archive format since these are all different approaches to describing a Drupal project.
Scriptability
As a tool designed to automate the low-level git workflow, dog is itself designed with scriptability in mind. Any commands which allow interaction should include a -y flag to run non-interactively, and they should all support a —backend or —json flag to do their output in script-friendly JSON.
Future Potential
We’re hoping to get many Drupal projects “on the dog sled” to help “vet” these patterns and create critical mass around a set of best practices. There are also obvious implications for Drupal distributions, as well as the update manager. The sky is the limit here.
Drupal And Git