Building Drupal applications with drush_make, provisioning sites with Aegir

Submitted by mig5 on Mon, 23/11/2009 - 15:17

Tags:

I did a screencast on how you can use Drush Make to generate 'builds' based on Drupal core, with optional components such as install profiles + module/theme/library dependencies. The example given is the Managing News news aggregation system.

I then show how you can add the build as a Platform in Aegir, and rapidly provision sites on it in a multisite structure.

Comments

Sean Bannister

Mon, 23/11/2009 - 21:20

Thanks Mig5, I've been meaning to try this out for a while but it was great to see someone else do it :)

- Sean Bannister

Pasqualle

Tue, 24/11/2009 - 00:42

It does do stuff.. and that stuff is cool.
Yes, it is! :)

JulienD

Tue, 24/11/2009 - 23:33

Nice screen-cast Mig.
Good quality (sound and video).
To understand the first part of the screen-cast, user need to know how work Drush.

Now that I've seen more about Aegir I think I'll create my own server !

Nice work !

Gregg

Wed, 25/11/2009 - 07:20

Hi Mig5

I am in my var/aegir/,
I have copied and pasted the code from your make file into a file called managingnews.build in this folder
I run php drush/drush.php make managingnews.build

A Drupal installation directory could not be found [error]
The command 'drush.php make managingnews.build' could not be found.

I guess a simple question to ask to start is if I have the latest build of aegir, do I need to download the drush make file? I was under the impression it was included already.

Any help would be appreciated

Thanks
Gregg

mig5

Wed, 25/11/2009 - 07:46

Hi Gregg,

If you're using the latest version of Aegir, then yes you should have drush_make already. It will be located in the /var/aegir/.drush directory.

I'm unsure why it thinks it needs to bootstrap Drupal at this early stage - that may indeed be a symptom of not having drush_make installed.

Hop onto IRC on #aegir at freenode if you get a chance, and I'll be happy to help you troubleshoot in real time.

Gregg

Sun, 29/11/2009 - 04:55

I see that drush make is installed in the .drush folder
I am now trying to run it from my platforms folder /var/aegir/platforms
I use this command php ../drush/drush.php make managingnews.make

A Drupal installation directory could not be found [error]
The command 'drush.php make managingnews.make' could not be found. [error]

When I run php ../drush/drush.php make

A Drupal installation directory could not be found [error]
The command 'drush.php make' could not be found. [error]

When I run php ../drush/drush.php I get the following list of commands
There does not seem to be a make command listed anywhere. Should there be?

I used the Aegir install script install.sh for my setup

Thanks for any help
Gregg
Gregg

Commands:
help Print this help message. Use --filter to limit command
list to one command file (e.g. --filter=pm)
cron Run all cron hooks.
updatedb (updb) Execute the update.php process from the command line
status (st) Provides a birds-eye view of the current Drupal
installation, if any.
script Run php script(s).
cache clear (cc) Clear all caches.
watchdog show (ws) Shows recent watchdog log messages. Optionally filter
for a specific type.
watchdog delete (wd) Delete all messages or only those of a specified type.
search status Show how many items remain to be indexed out of the
total.
search index Index the remaining search items without wiping the
index.
search reindex Force the search index to be rebuilt.
sync Rsync the Drupal tree to/from another server using ssh.
eval Evaluate arbitrary php code after bootstrapping Drupal.
installcore (ic) Install Drupal core via the specified install profile.
enable (en) Enable one or more modules.
disable (dis) Disable one or more modules.
uninstall Uninstall one or more modules.
statusmodules (sm) Show module enabled/disabled status
refresh (rf) Refresh update status information
updatecode (upc) Update your project code
update (up) Update your project code and apply any database updates
required (update.php)
info Release information for a project
download (dl) Download core Drupal and projects like CCK, Zen, etc.
test mail Run all tests and mail the results to your team.
test clean Delete leftover tables and files from prior test runs.
sql conf Print database connection details.
sql connect A string for connecting to the DB.
sql dump Exports the Drupal DB as SQL using mysqldump.
sql query (sqlq) Execute a query against the site database.
sql load Copy source database to target database.
sql cli (sqlc) Open a SQL command-line interface using Drupal's
credentials.
variable get (vget) Get a list of some or all site variables and values.
variable set (vset) Set a variable.
variable delete Delete a variable.
(vdel)

davebv

Fri, 08/01/2010 - 21:43

Hi mig5, thanks a lot for the screencast!

I am just starting with the installation profiles and drush_make, and I found the modules specified in the makefile inside the installation profile package are downloaded to the profiles/name_or_install_profile directory, instead of sites/all/modules...

I see, for example, atrium modules are organized in the sites/all/modules directory. Is that packages generated with drush_make? How can I make drush_make to put the modules I want to sites/all instead the profile/name_of_profile dir?

Thanks!

mig5

Fri, 08/01/2010 - 21:53

Hi davebv,

Your question is the same as was asked here - basically per my comment there,, you could simply avoid using install profiles and makefiles contained therein, and generate Drupal builds with core + modules + themes directly all in the one main makefile - because the default destination for modules and themes *is* actually /sites/all/ in Drush Make. It's the recursive execution of the makefile inside the install profile that 'binds' the dependencies to that install profile. And that's on purpose.

I do want to stress that putting modules, themes and libraries into the subdir of the profile is very much the point of this process: it means the entire 'application' is contained within one location, and that's what makes it easy to package. It makes /sites/all/ kind of redundant in that you are starting to treat your Drupal sites as running instances of a self-contained application that can be controlled.

Also note that sites using that install profile can see those modules and themes and no other sites can. To me it's a feature, but then again, I don't use /sites/all/ . Basically my attitude is that I simply don't think about /sites/all/ - in my 'application' workflow design, it is redundant (and yes there is duplication, lots of copies of cck and so on lying around.. but disk is cheap to me, and if I have things in /sites/all/, I feel I can't keep control of the application properly if I want to move things, and other sites depend on that stuff, etc.)

Re: your question about Atrium: the reason Atrium's components are in sites/all is because Atrium predates Drush Make, which is why Atrium is shipped with a whole copy of core rather than just an install profile + makefile like Managing News. So /sites/all here is really just a victim of the pre-paradigm shift.

However, I actually made a proof of concept OpenAtrium makefile and split out all the modules into individual repos on github, and provided the work to DevelopmentSeed. So we will soon see Atrium available simply as an install profile with makefile, and the modules in /profiles/(atrium profile)/modules/ .. the guys are working on it as we speak.

Not a very satisfying answer, but as best an answer that I can give :) think of it as 'by design'.

Santhan

Fri, 26/03/2010 - 15:20

Miguel, thank you for the tutorials on this site and the clear explanations.

Our projects are hosted on VPS accounts and run of a single Drupal core and shared common modules. Each VPS runs both the database and web servers and make use of xcache. Our multi-sites use CDN for static content, sometimes the Boost module along with other standard Drupal cache features. We run several medium traffic sites (~3-4K page loads) on VPS accounts with 512-768M of memory and get very good performance for both anonymous and logged in users. This lean setup keeps monthly expenses low without compromising the user experience.

My concern about having modules repeated, like cck and views is that the memory allocated to the opcode cache gets quickly used up. A "compiled" php script uses up quite a bit of memory. For example xcache stats show that views.module is using ~180K of memory when "compiled" (36K disk size).

Is this concern valid?

I think even if we had a huge budget (many of our projects are non-profit) I would still try to stay lean. The adage, "Less is more" keeps running through my mind.

davebv

Sun, 10/01/2010 - 21:21

Thanks a lot for the explanation. I am sorry I repeated the question. Now I think I understand the concepts of makefiles and organization of files, etc. Thanks again for the complete answer.

Phil Hersh

Sat, 19/06/2010 - 04:49

Hi Miguel,

Thanks for your excellent work. Could I ask you a question about something I saw in the video?

In the video, you had created a shortcut to run host dispatch. I can't get this to work as shortcut to an su command and I was wondering what your recipe is.

I can run this from the command line:
su aegir -c "/var/aegir/drush/drush.php hosting-dispatch --root=/var/aegir/hostmaster-0.4-alpha8 --uri=<a href="http://mydomain.com"">http://mydomain.com"
[/geshifilter-code]
so then I tried
export DISPATCH='su aegir -c "/var/aegir/drush/drush.php hosting-dispatch --root=/var/aegir/hostmaster-0.4-alpha8 --uri=<a href="http://mydomain.com"'">http://mydomain.com"'
[/geshifilter-code]
but when I run
$DISPATCH

I get this error:
su: unrecognized option `--root=/var/aegir/hostmaster-0.4-alpha8'

It seems that when bash expands the VAR, the su command as I've written it doesn't recognize that the -- options are part of the drush command.

Any thoughts or suggestions would be most helpful.

Thanks,
Phil

mig5

Sat, 19/06/2010 - 11:42

Hi Phil,

An environment variable you'd lose every time you exited a shell session unless you export it to an rc file or something.

In the aegir user's .bashrc file, set an alias like this:

alias dispatch='php /var/aegir/drush/drush.php hosting-dispatch --root=/var/aegir/hostmaster-0.4-alpha8 --uri=<a href="http://mydomain.com'

">http://mydomain.com'
[/geshifilter-code]

Consider entering a bash shell as the aegir user like this:

su -s /bin/bash aegir

Then you can just run 'dispatch'.

Alternatively you could put a 'dispatch' script in /usr/local/bin/ that just looks like this:

#!/bin/bash
php /var/aegir/drush/drush.php hosting-dispatch --root=/var/aegir/hostmaster-0.4-alpha8 --uri=<a href="http://mydomain.com

">http://mydomain.com
[/geshifilter-code]

And make it chmod 755 to execute it.

Steve

Sun, 28/08/2011 - 10:36

Perhaps you already now this, but the video is no longer available. In fact, there is no Mig5 videos on Vimeo at this time.

Add new comment