About Chef Automate Pipelines

[edit on GitHub]

Each project contains a configuration file in its source repository, located at .delivery/config.json, that specifies the build cookbook to use for the project, and in turn, the build cookbook contains recipes that control what happens in the pipeline phases. The config file also allows customization of the behavior of Chef Automate and the build cookbook. You can create a config file (as well as a build cookbook) using the Chef Automate CLI tool and the init subcommand: delivery init or delivery init --local.

When Chef Automate executes a phase, it selects a build node or runner to run the job. On the build node/runner, the project’s source is fetched and synchronized to the revision matching the head of the feature branch for the change. The build node/runner reads the project’s config.json file and uses this information to fetch the appropriate build cookbook. Finally, the build node/runner runs a local chef-zero run to execute the appropriate phase.

If you are using Chef Automate to manage changes in Chef cookbooks, you can wrap, or use directly, delivery-truck, a build cookbook for building and testing cookbooks. The delivery-truck and delivery-sugar cookbooks contain helpers that can be used for non-cookbook workflows as well. You can wrap or modify the delivery-truck cookbook to suit your own needs.

Here is an example of a build cookbook recipe that runs JUnit tests with Maven. For example:

log 'Running unit'

repo = node['delivery_builder']['repo']

execute 'run my JUnit tests' do
  command 'mvn test'
  cwd repo
end

This code logs that the unit tests are running and runs JUnit tests against the current repo.

The following example shows how to configure Chef Automate to ignore and/or run certain Foodcritic rules, and to exclude running tests that are located in the specified cookbook directories:

{
  "version": "2",
  "build_cookbook": {
    "name": "delivery-truck",
    "git": "https://github.com/chef-cookbooks/delivery-truck.git"
  },
  "delivery-truck": {
    "lint": {
      "foodcritic": {
        "ignore_rules": ["FC009", "FC057", "FC058"],
        "only_rules": ["FC002"],
        "excludes": ["spec", "test"],
        "fail_tags": ["any"]
      }
    }
  }
}

where:

• ignore_rules is set to ignore Foodcritic rules FC009, FC057, FC058
• only_rules is set to run only Foodcritic rule FC002; omit this setting to specify all rules not specified by ignore_rules
• excludes prevents Foodcritic rules from running if they are present in a cookbook’s /spec and/or /test directories
• fail_tags states which rules should cause the run to fail; omit this setting to specify correctness

Because build cookbooks read the configuration file, use the configuration file to customize the build cookbook to suit the needs of a particular project. In this way, you can share some “standard” version of a build cookbook with others and then use extra data in the config file to tailor the cookbook as needed.

Configuration Settings

The behavior of pipeline phases can be customized using the project’s config.json file. Each config.json file has a set of required settings as well as optional settings that can be set, such as which build nodes to use for specific phases, whether to skip certain phases, and so on. The basic settings are defined below; however, additional settings, such as which linting rules to follow, can also be added. See the delivery-truck readme for examples on how to add custom settings.

version

Required

The version setting specifies the version of the configuration that the Chef Automate server must user. The current default value is 2,

build-cookbook

Required

The build_cookbook setting specifies the location of the build-cookbook used by this project. A build-cookbook may be fetched from five locations:

• A local directory within the project
• A git repository
• A Chef Supermarket instance (public or private)
• A Chef server
• A Chef Automate server

build_nodes

Optional

The build_nodes setting specifies which build nodes to use for specific phases in the Chef Automate pipeline. The build node may be defined as well as queried via wildcard search.

Note

This setting should only be used with build nodes that use the previous push job-based dispatch system. Use the job_dispatch setting when using the new ssh-based job dispatch system.

skip_phases

Optional

The skip_phases setting specifies which phases are skipped by Chef Automate during the execution of a change through the pipeline. If a phase is defined as skipped, this applies to all stages in the pipeline.

Currently, the functional.rb, quality.rb, security.rb, and smoke.rb recipes are blank by default and should be set to skipped in the config.json file:

"skip_phases": [
  "functional",
  "quality",
  "security",
  "smoke"
]

dependencies

Optional

The dependencies setting specifies run-time dependencies on which the current project depends. These dependency associations affect how projects are promoted through the Union, Rehearsal, and Delivered stages. Dependencies may be defined in the following ways:

• "project_name"
• "project_name:pipeline_name"
• "org_name/project_name"
• "org_name/project_name:pipeline_name"

If only a project name is provided, the master pipeline for that project is the dependency.

build-cookbook Locations

The following examples show how to specify the location of the build-cookbook.

A local directory

"build_cookbook": {
  "name": "build-cookbook",
  "path": ".delivery/build-cookbook"
}

A git source

"build_cookbook": {
   "name"  : "delivery-truck",
   "git"   : "https://github.com/chef-cookbooks/delivery-truck.git",
   "branch": "master"
}

A public Supermarket (https://supermarket.chef.io)

"build_cookbook": {
   "name": "delivery-truck",
   "supermarket": "true"
}

A private Supermarket

"build_cookbook": {
   "name": "delivery-truck",
   "supermarket": "true",
   "site": "https://private-supermarket.example.com"
}

A Chef server

"build_cookbook": {
   "name": "delivery-truck",
   "server": "true"
}

A Chef Automate server

"build_cookbook": {
   "name": "delivery-truck",
   "enterprise": "chef",
   "organization": "chef-cookbooks"
}

Build Nodes and Phases

The following example shows how to specify build nodes to be used for specific phases.

"build_nodes": {
  "provision": ["name:builder-*-2.delivery.chef.co AND platform_version:14.04"],
  "deploy": ["name:builder-*-2.delivery.chef.co AND platform_version:14.04"],
  "functional": ["name:builder* AND platform_version:14.04 NOT name:builder-*-2.delivery.chef.co"]
}

Run-time Dependencies

The following example shows a run-time dependency against the master branch of a project named BackendAPI:

{
  "version": "2",
  "build_cookbook": {
    "name": "build-cookbook",
    "path": ".delivery/build-cookbook"
  },
  "skip_phases": [],
  "dependencies": ["BackendAPI"]
}

Multiple Pipelines

To set up a second pipeline, there is an assumption that the branch that will become the second pipeline exists in your local project.

1. Push that branch to the Chef Automate Server git push delivery $BRANCH_NAME
2. Navigate to the project’s page (e/$ENT_NAME/#/organizations/$ORG_NAME/projects/$PROJECT_NAME) in the Chef Automate web UI and click on the Pipelines tab.
3. Click on Add A New Pipeline on the top of the page.
4. Give the new pipeline a descriptive name and input the base branch.

To make a pipeline other than master the default for a single project, create a cli.toml file in the /.delivery directory in the root of the project that includes pipeline = "$BRANCH_NAME". Now, all delivery-cli commands that target a pipeline will target $BRANCH_NAME.

If you wish to target a pipeline that is NOT the defined default, add the --pipeline=$BRANCH_NAME flag to the delivery-cli command.

Example:

delivery review --pipeline=$BRANCH_NAME

The commands that take this flag are:

• delivery init
• delivery review
• delivery diff
• delivery job
• delivery setup