Using deploy hooks

Overview

Deploy hooks are scripts that allow you to automate actions at various points during the deployment process for your applications. This allows you to customise your deployments by, for example, installing software packages or upgrading components.

Tutorial

What you'll need

Before you start, please check you have the following:

• A Cloud 66 Account — If you don't already have one, sign up for a Cloud 66 account. Your first server is free, no credit card required.
• An existing application set up in Cloud 66 — To make the most of this tutorial you need to have an app already set up in Cloud 66. Follow our Getting Started guide if you're not sure how to do this.

Creating a deploy hook

A deploy hook needs, at a minimum:

• a hook point - where in the deployment process the hook must be invoked
• a hook type - either a command or one of the two script types
• a target - which type(s) of servers will use this hook
• a hook field - the command or script being run (or invoked)

So, to write a deploy hook you must:

1. Choose your hook point - e.g. first_thing
2. Choose your hook type - e.g. command
3. Set a target server-type for the hook - e.g. mysql
4. If your target is not any then set the run_on to either single_server or all_servers.
5. Configure the hook fields you require

While this is the bare minimum required to write a functional deploy hook, there are extensive options available for customization. Please read the rest of the guide (below) to understand all the possibilities.

Writing the YAML

The simplest kind of hook is the command. This simply executes a command in the operating system as part of the deployment process.

We're going to add the hook below to our demo application:

    first_thing: # Hook point
      - command: apt-get install nmap -y # Hook type
        target: any # Hook fields
        execute: true

This hook will install the nmap package on our server during the deployment process. We've added the execute hook field because we want the command to be executed during deployment. If you don't add this field, the code you're calling won't be executed.

Deploying and testing

Now that our hook is in place, we need to re-deploy our application to see it in action.

1. Navigate back to application
2. Click the Deploy button
3. Watch the deploy log and you will see the "first _thing" deploy hook being called as part of the process

(As always, we recommend testing your hooks in a non-production environment before using on your live application.)

The best way to check whether your change has been applied to your server is to access it directly using SSH. Cloud 66 Toolbelt is the quickest way to do this.

Once you are connected to your server, type nmap into the terminal. If your deploy hook was set up correctly, you will see the usage / help text for the nmap utility. If not, Ubuntu will complain that nmap is not installed.

Full guide to deploy hooks

Deploy hooks are YAML-formatted configuration sets that that allow you to automate actions (such as running commands) at various points during the build and deployment process for your applications. If you’ve never used hooks before, please follow our tutorial to get a grasp of the basics.

Deploy hooks are a powerful tool for customising the components on your servers, and automating the setup process whenever you add new servers. However they require careful configuration to work exactly as expected. This guide will explain all the necessary detail for you to write your own custom hooks. For am exhaustive list of all the options available, please consult our Reference guide.

Creating or editing deploy hooks

Every deploy hook needs, at a minimum:

• a hook point - where in the deployment process the hook must be invoked
• a hook type - either a command or one of the two script types
• a target - which type(s) of servers will use this hook
• a hook field - the command or script being run (or invoked)

Any hook may (optionally) also have:

• a name which appears in your logs and user interface when the hook runs (to help you identify what is running)

Understanding hook points and ordering

Hook points are used to define the point in your deployment process at which a hook should be invoked. This is obviously critical when there are tight dependencies between the components of your application (i.e. one component relies on another component being installed first), but it is also important in terms of what actions and commands are possible. For example, running tasks against a database before the database server is installed will not work!

It's important to understand the order in which hook points will occur in the flow of deployment. The simplified deployment process below shows where each deploy hook is triggered. Hooks are marked in bold italic. Some hooks have several possible values (x, y, z). Click on the hook name to see a list of available options:

Deployment process (simplified)

1. Server is fired up
2. Operating system and standard system components installed → first_thing
3. before_agent → Cloud 66 Agent is installed → after_agent
4. before_x → Database is installed → after_x
5. before_y → Database replication is configured → after_y
6. before_data_mount → Data is mounted → after_data_mount (GlusterFS specific)
7. before_haproxy → HAProxy is installed → after_haproxy (runs on HAProxy servers only)
8. custom_server (runs on custom servers only)
9. before_nginx → NGINX is installed → after_nginx
10. before_docker → Docker is installed after_docker
11. last_thing

X, Y and Z

• x, y and z represent database & storage engine installation , replication configuration and application framework installation respectively.
• The last_thing hook runs only when ALL servers reach that point

For more details on hook points, please read our Reference guide.

Hook Targets

The target of a hook is the server or set of servers on which it must be executed.

You can use any to run the hook across your entire application, but you can also choose to run it on a specific type of server.

Calling environment variables in deploy hooks

You can pull environment variables from your application configuration into your hooks using the following format:

username: <%= ENV['DB_USER'] %>

These environment variables need to be available in your app before the deploy hook will work correctly. Please follow our tutorial if you're unsure of how to do this.

Hook types

There are different types of deploy hooks, and the fields available (and required) vary by type:

• Commands: run standard Linux shell commands.
• Snippets: use “off the shelf” scripts to install common packages. These snippets are open source, and are created by Cloud 66 or other third parties.
• Inline Scripts: use your own inline shell scripts for more complex procedures

Using command hooks

The simplest type of hook is the command. This executes a command in Ubuntu as part of the deployment process.

You may recognise this example from our tutorial:

production: # Environment
    first_thing: # Hook point
      - command: apt-get install nmap -y # Hook type
        target: any # Hook fields
        execute: true

This hook will install the nmap package on our server during the deployment process. You can find all the avaible options for command hooks in our reference guide.

Using snippet hooks

Snippets are pre-defined hook scripts hosted in a Cloud 66 repository. These are commonly used scripts that we provide for ease of use. Like scripts (see below) these snippets are written in bash.

For example the Cloud 66 ImageMagick snippet installs ImageMagick, the popular image processing application. You can use the example below to test it in your demo app:

production: # Environment
    first_thing: # Hook point
      - snippet: cloud66/imagemagick # Hook type
        target: any # Hook fields
        execute: true

You can test this in the same way as you did with the command hook in our tutorial.

Using inline script hooks

Script hooks allow you to add your own logic to a deploy hook, either by calling a separate file using source or by using inline. Both of methods use bash scripting (as do snippets above).

The hook below will create an arbitrary log file in /tmp using a simple inline script:

first_thing: # Hook point
 - inline: |

     #!/usr/bin/env bash
     echo "script called!" >> /tmp/inline_script.log
   target: any
   execute: true
   apply_during: all
   owner: root:root

Using existing script hooks

Existing script hooks allow you to copy a script in your code repository to a chosen location on your server and (if required) to execute it.

The hook below will wait until the very end of the deployment process (last_thing), then copy the shell script named magic-shell.sh into the /usr/local/bin directory on your server and finally execute it.

last_thing:
 - source: scripts/magic-shell.sh
   destination: /usr/local/bin
   target: any
   execute: true
   apply_during: all

Advanced examples of Deploy Hooks

We've put together some examples of the most common use cases, but there an almost infinite number of possibilities. Essentially if you can install it on a Linux server or code it into a shell script, you can run it from a deploy hook.

Querying server metadata

You can query a server's metadata (including deployment details) using the following command:

curl -s -H "Accept: application/json" https://$CLOUD66_ACCOUNT_API_KEY:X@app.cloud66.com/api/tooling/metadata/$CLOUD66_APPLICATION_API_KEY

You can use this command in a deploy hook, for example:

first_thing:
- target: any
  execute: true
  apply_during: all 
  sudo: false
  inline: |
    #!/usr/bin/env bash
    current_deployer=$(curl -s -H "Accept: application/json" https://$CLOUD66_ACCOUNT_API_KEY:X@app.cloud66.com/api/tooling/metadata/$CLOUD66_APPLICATION_API_KEY/deployment/triggered_by | jq -r '.response')
    echo "Deployed by: $current_deployer"

For more details please read our full Querying server metadata doc.

Install packages on each server deployed

If your application relies on a package or software library that is not installed by default by Ubuntu, or by Cloud 66, you may ask, how do I install a package or library on each server I deploy?

You can use a deploy hook to fetch and install the package whenever you deploy a server.

For this example we're going to use glances - a popular FOSS process monitor - but you can install literally anything that has an Ubuntu repository using this method.

We need to set up our deploy hook to install the package from the Ubuntu. Every deploy hook needs:

• a hook point - in this case we are using the first_thing hook point to ensure this is installed as early as possible in case any other processes depend on it
• A hook type - we are using the command hook type
• A target - in this case we are using any because we want the package on all our servers
• A hook field - this is where we put the commands we're going to run.

Any hook may (optionally) also have:

• a name which appears in your logs and user interface when the hook runs (to help you identify what is running)

Our hook code will look like this:

first_thing:
 - command: sudo apt-get install glances
   name: installing glances
   target: any
   execute: true

Now that we have our hook code ready, we need to add it to our application. We do this by creating a file in the /.cloud66 directory of our repo called deploy_hooks.yml and dropping in the YAML code we just wrote above. If you're not well versed in YAML, it's often helpful to use an online linter to test that your code is valid and conformant.

Once you have added the deploy_hooks.yml file to your repo you can deploy (or redeploy) your application and glances will be installed during the process. To test that it is installed correctly you can SSH into your server and run glances - it should bring up the monitoring interface.

Installing non-standard versions of packages

If your application depends on a specific version of a Linux package or software library, the best way to ensure the correct version is installed is by using a deploy hook.

To illustrate how this is done we're going to use ImageMagick - the beloved and ubiquitous image processing software installed on millions of servers around the world. Suppose your application relies on an older version of ImageMagick - in this case Version 6.9.9-51.

We need to set up our deploy hook to download the source code for this version and then build and compile it. Every deploy hook needs:

• a hook point - in this case we are using the first_thing hook point to ensure this is installed as early as possible in case any other processes depend on it
• A hook type - we are using the inline hook type - we'll go into the details of this later
• A target - in this case we are using rails but any would also work
• A hook field - this is where we put the commands we're going to run. We're going to use these instructions as the basis for our commands.

Any hook may (optionally) also have:

• a name which appears in your logs and user interface when the hook runs (to help you identify what is running)

Hooks are written in YAML and ours will look like this:

first_thing:
- target: rails
  name: ImageMagick v6.9.9-51
  execute: true
  apply_during: build_only 
  sudo: true
  inline: |
    #!/usr/bin/env bash
    set -e
    version="6.9.9-51"
    echo "Installing ImageMagick v$version"

    echo "1. Fetching Tarball"
    curl -sL https://codeload.github.com/ImageMagick/ImageMagick6/tar.gz/"$version" --output /tmp/image-magick-"$version".tar.gz 

    echo "2. Extracting Tarball"
    cd /tmp
    mkdir -p /opt/image-magick/v"$version"
    tar -xvzf image-magick-"$version".tar.gz -C /opt/image-magick/v"$version" --strip 1

    echo "3. Building ImageMagick"
    cd /opt/image-magick/v"$version"
    ./configure
    make
    make install 
    ldconfig /usr/local/lib

    echo "4. Tidying up"
    rm -f /tmp/image-magick-"$version".tar.gz

    echo "ImageMagick install complete!"

Two things to take note of here:

1. In order to execute properly, the script needs a place on the disk to run - we are using the /tmp/ directory
2. We have added another option - apply_during - this allows us to set our hook to only run when a server is first built, and not on every deploy. You can find out more about these options in our reference doc.

Now that we have our hook code ready, we need to add it to our application. We do this by creating a file in the /.cloud66 directory of our repo called deploy_hooks.yml and dropping in the YAML code we just wrote above. If you're not well versed in YAML, it's often helpful to use an online linter to test that your code is valid and conformant.

Once you have added the deploy_hooks.yml file to your repo you can deploy (or redeploy) your application and version 6.9.9-51 of ImageMagick will be installed during the process. To test that it is installed correctly you can SSH into your server and run identify -version.

Debugging deploy hooks

Automating deploy hooks can sometimes be tricky. To avoid issues, it's good practice to run each of your commands manually to see that they complete successfully.

If a specific command doesn't show any output, you can use the echo $? command after issuing your command to see its exit code. If it returns a zero, your command was successful, whereas a one means it has failed.

Hook points

The deployment process is divided into a number of steps, and hook points allow you to intervene at various points during this process.

The table below is arranged in the order in which each hook point occurs in the deployment process (from earliest to latest):

Targets

Every deploy hook must have a target - a server or set of servers on which it must be executed. You can simply choose any to run the hook across your entire application, but you can also choose to run it on a specific type of server. The options are any, rails, mysql, postgresql, mongodb, redis, sinatra, padrino, custom.

If you set the target of a hook to anything other than any then you will need to pay attention to the run_on field. This determines whether the hook will be applied to a single server in that group (the default) or to all the servers in that group.

Hook types

There are different types of deploy hooks, and the fields available (and required) vary by type:

• Snippets: use "off the shelf" scripts to install common packages. These snippets are open source, and are created by Cloud 66 or other third parties.
• Commands: run your own commands.
• Inline Scripts: use your own inline scripts for more comprehensive procedures

Hook fields reference guides

Each hook type has a different set of hook fields available, with some shared fields. Please be careful to note that:

• Some fields are mutually exclusive (e.g. you cannot specify both sudo and run_as fields)
• Some fields have different defaults for different hook types. For example the execute field defaults to false for inline scripts and snippets, but to true for commands.

Hook fields: snippets

Hook fields: commands

Hook fields: inline scripts

Hook fields: existing scripts

Existing scripts are a Rails-only feature.