With the emergence of trends such as continuous deployment and delivery, the continuous integration server is not limited to integrating your products, but has become a central piece of infrastructure. Many of these jobs depend on the other jobs to be successfully completed. So you would want to proceed with a certain action only if the previous action is completed or do a different action. For example, consider below series of actionable tasks related to a project:

1. Stop the server.
2. Remove Website.
3. Drop SQL Database.
4. Clean up the file system.
5. Create SQL Database.
6. Create Website.
7. Start the server.

So you can either create either a massive job or create small jobs which are linked together. If you go with the creating many small jobs, you can re-execute that job(read as redo that action), and not do the previous related jobs to it. In other words, you can start from a failing step rather than from very initial stage. So its always advisable to create multiple small jobs and link them together to create a pipeline.

In this blog post, we’ll learn how to create linking between different jobs or projects in Jenkins. For our learning purposes, we have created two small jobs named ‘Job 01’ and ‘Job 02’ in Jenkins server:

These two jobs contain nothing but a simple echo statement echoing the name of the job. Here are few of the steps required to create a very basic job.

One to One Relationship between Jobs

The first approach is pretty simple: one job triggers another one after successful completion. To achieve it, you can either configure a post-build action starting the next job (downstream) or configure the trigger of the current build (upstream). Both ways are totally equivalent. In order to this, let’s go to job 02 and select configure from dropdown:

Now go to ‘Build triggers’ and select ‘Build after other projects are built’. Under ‘Projects to Watch’ to select ‘job 01’:

Let’s select ‘Trigger only if build is stable’ so that ‘Job 02’ is executed only when ‘Job 01’ is successfully completed. After this, select save and apply. Now when you build ‘Job 01’ and see the log output, it should be something like this:

If you look at last 2nd line, you can see that the it started a new build of ‘Job 02’. Since it contains a link, instead of going to dashboard, you can directly click on the link to go to job. It will contains a new section named ‘Upstream Projects’ and would have listed ‘job 01’ as the build. Again from build history pane, you can see the recently build of this ‘Job 02’.

Using this one-to-one dependency approach is probably the simplest way to orchestrate jobs on Jenkins.

One to Many Relationship between Jobs

You can also create a master job and associate various child jobs to it. So let’s say we have ‘Job 01’ as master job and ‘Job 02’, ‘Job 03’, ‘Job 04’ as child jobs of it. Again, we’ll use the same method to create links between jobs as above and initiate build.

A Special Note

Please keep below points in mind, before going for one-to-many relationship:

1. If one of the child job fails, other child jobs will still be triggered. They may or may not have some inter-dependency later in the flow. Off course, you can use the success of multiple jobs as the trigger as well. So you need to plan it carefully.
2. There is no order defined for calling the child jobs. This is because Jenkins threads are asynchronous in nature.

Configuring custom relationships between Jobs

Let’s suppose your flow is something like this:

              /-> Job 02 -\
  Job 01 -> *                * ->  Job 04 -> Job 05
              \-> Job 03 -/

We can use the ‘Join Plugin’ for same. This plugin allows a job to be run after all the immediate downstream jobs have completed.   In this way,  the execution can branch out and perform many steps in parallel, and then run a final aggregation step just once after all the parallel work is finished. You can go to Manage Jenkins -> Manage Plugins -> Available tab and search for this plugin:

Select and install it.

Now Let’s go to ‘Job 01’ and go to post-build section. In build other projects, add ‘Job 02’ and ‘Job 03’. Then add join trigger from post build steps and enter ‘Job 04’. This will ensure that ‘Job 04’ is executed only when both ‘Job 02’ and ‘Job 03’ are completed. Again, you can use this plugin to create multiple diamond shape dependencies like above. So this helps in allowing creating complex pipelines.