Add linux slave node in the Jenkins

As per best practices, the master node should be only used for storing configuration and backup purposes. Only slaves should be used for build. In this blog post, we’ll discover steps required for adding slave node in the Jenkins farm. Most of these steps will cover how to prepare linux slave server for Jenkins usage. The below commands are for CentOS 7 server but these can be easily translated to other linux distros.

Install Java on the Slave server

Run below command on the server:

sudo yum update
sudo yum install openjdk-8-jre
sudo yum install openjdk-8-jdk

You can check if jvm is installed properly using java -version.

In order to help Java-based applications locate the Java virtual machine properly, you need to set two environment variables: “JAVA_HOME” and “JRE_HOME”:

export JAVA_HOME=’/usr/lib/jvm/jre-1.8.0-openjdk’
export JRE_HOME=’/usr/lib/jvm/java-8-openjdk-amd64/jre’

Edit profile script and add these two export commands to it so that these variables are always available whenever the system restarts.

Add administrative service user to the Slave server

This is important from administrative and auditing point of view. In our case, let’s say that the service account name is Jenkins. We’ll also create a user group named jenkins. For this, run below command

sudo useradd jenkins -U -s /bin/bash

Verify that user and group are created by checking /etc/passwd and /etc/group files. Now change the password associated with this account using:

sudo passwd jenkins

and enter new password when asked. Now, configure sudo privileges for this user by modifying /etc/sudoers:

Modifying sudoers to assign sudo privileges.JPG
Modifying sudoers to assign sudo privileges

Configure SSH Key authentication for Jenkins

First we need to create the key pair on the master machine:

ssh-keygen -t rsa

Once you have entered the Gen Key command, you will get a few more questions about file location to save keypair and passphrase. It’s up to you whether you want to use a passphrase. Entering a passphrase does have its benefits: the security of a key, no matter how encrypted, still depends on the fact that it is not visible to anyone else. The only downside, of course, to having a passphrase, is then having to type it in each time you use the Key Pair. for our purposes, we’ll leave the passphrase as empty.

Below is a sample run from my lab machine:

Generate ssh keypair on the slave jenkins server.JPG
Generate ssh keypair on the master jenkins server

The public key is now located in /home/jenkins/.ssh/id_rsa.pub. The private key (identification) is now located in /home/jenkins/.ssh/id_rsa.

Once the key pair is generated, it’s time to place the public key into the slave machine’s authorized_keys file with the ssh-copy-id command:

ssh-copy-id jenkins@10.20.3.132

You need to replace the username and password in the above command as per your environment. Also note that if you are doing this on a cloud virtual machine, do the same for the internal as well as public ip of the machine.

You should see something like below output:

[jenkins@centos2 ~]$ ssh-copy-id jenkins@10.20.3.132
The authenticity of host '10.20.3.132 (10.20.3.132)' can't be established.
ECDSA key fingerprint is 53:c2:32:63:12:a2:8f:29:25:40:fa:0a:b1:d4:8c:f4.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
jenkins@10.20.3.132's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'jenkins@10.20.3.132'"
and check to make sure that only the key(s) you wanted were added.

Now you can go ahead and login into the machine 10.20.3.132 with your username and you will not be prompted for password.

Setup relationship between slave and master

Login into Jenkins master machine with administrative credentials.  First go to Manage Jenkins -> Manage plugins and install ‘SSH Slaves Plugin’. Now Go to Manage Jenkins -> Manage node:

Manage node section.JPG
Manage node section

Select new node from left pane. Then enter slave machine’s IP address and select ‘Permanent Agent’ and click okay. This will ask for further details.

In the ‘# of executors’, select maximum number of concurrent builds that Jenkins may perform on this agent. Generally, this is set as per no of processor cores available on the remote machine. For our purposes, we’ll set it to 10.

In the ‘remote root directory’, add the path for a directory dedicated to be used by agent which should be /home/jenkins. In the launch method, select ‘launch slave agents via ssh’ and add the slave machine’s ip address and credentials.

These are going to be details for our case:

Add slave machine's details.JPG
Add slave machine’s details

In the last, click save and then okay. It’ll take few minutes to connect and bring the slave node online. For checking logs, click on the slave machine name and then click logs:

Click on the log tab associated to dropdown for Slave machine.png
Click on the log tab associated to dropdown for Slave machine.png

Once you click logs, you should be able to see output like below:

[02/14/17 07:39:01] [SSH] Opening SSH connection to 10.20.3.132:22.
[02/14/17 07:39:02] [SSH] Authentication successful.
[02/14/17 07:39:03] [SSH] The remote users environment is:
BASH=/usr/bin/bash
BASHOPTS=cmdhist:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_EXECUTION_STRING=set
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="4" [1]="2" [2]="46" [3]="1" [4]="release" [5]="x86_64-redhat-linux-gnu")
BASH_VERSION='4.2.46(1)-release'
DIRSTACK=()
EUID=1001
GROUPS=()
HOME=/home/jenkins
HOSTNAME=centos2.local
HOSTTYPE=x86_64
ID=1001
IFS=$' \t\n'
LANG=en_US.UTF-8
LESSOPEN='||/usr/bin/lesspipe.sh %s'
LOGNAME=jenkins
MACHTYPE=x86_64-redhat-linux-gnu
MAIL=/var/mail/jenkins
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/local/bin:/usr/bin
PIPESTATUS=([0]="0")
PPID=16448
PS4='+ '
PWD=/home/jenkins
SELINUX_LEVEL_REQUESTED=
SELINUX_ROLE_REQUESTED=
SELINUX_USE_CURRENT_RANGE=
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=1
SSH_CLIENT='10.20.2.244 44392 22'
SSH_CONNECTION='10.20.2.244 44392 10.20.3.132 22'
TERM=dumb
UID=1001
USER=jenkins
XDG_RUNTIME_DIR=/run/user/1001
XDG_SESSION_ID=35
_=/etc/bashrc
command_not_found_handle () 
{ 
    local runcnf=1;
    local retval=127;
    [[ $- =~ i ]] || runcnf=0;
    [ ! -S /var/run/dbus/system_bus_socket ] && runcnf=0;
    [ ! -x /usr/libexec/packagekitd ] && runcnf=0;
    [ ${COMP_CWORD-} ] && runcnf=0;
    if [ $runcnf -eq 1 ]; then
        /usr/libexec/pk-command-not-found "$@";
        retval=$?;
    else
        local shell=`basename "$SHELL"`;
        echo "$shell: $1: command not found";
    fi;
    return $retval
}
[02/14/17 07:39:03] [SSH] Checking java version of java
[02/14/17 07:39:03] [SSH] java -version returned 1.8.0_121.
[02/14/17 07:39:03] [SSH] Starting sftp client.
[02/14/17 07:39:03] [SSH] Copying latest slave.jar...
[02/14/17 07:39:04] [SSH] Copied 715,860 bytes.
Expanded the channel window size to 4MB
[02/14/17 07:39:04] [SSH] Starting slave process: cd "/home/jenkins" && java  -jar slave.jar
channel started
Slave.jar version: 3.2
This is a Unix agent
Evacuated stdout
Agent successfully connected and online

If everything mentioned above is configured correctly, it should be able to successfully connect to the slave machine.

24 thoughts on “Add linux slave node in the Jenkins

  1. I have done this a couple of times already, but this is the first time I come across some notes using the ssh-copy-id command. Thanks so much!

    Like

  2. Thanks, Mohit Goyal

    It worked for me but in the logs, it gave the below error. I have SFTP installed in my slave Ubuntu 16.04.3 LTS

    could you please let me know how to fix this issue?

    [01/11/18 18:35:08] [SSH] Starting sftp client.
    ERROR: [01/11/18 18:35:08] [SSH] SFTP failed. Copying via SCP.
    java.io.IOException: The subsystem request failed.
    at com.trilead.ssh2.channel.ChannelManager.requestSubSystem(ChannelManager.java:739)
    at com.trilead.ssh2.Session.startSubSystem(Session.java:355)
    at com.trilead.ssh2.SFTPv3Client.(SFTPv3Client.java:100)
    at com.trilead.ssh2.SFTPv3Client.(SFTPv3Client.java:119)
    at hudson.plugins.sshslaves.SFTPClient.(SFTPClient.java:43)
    at hudson.plugins.sshslaves.SSHLauncher.copySlaveJar(SSHLauncher.java:1138)
    at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:148)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:833)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:810)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    Caused by: java.io.IOException: The server denied the request.
    at com.trilead.ssh2.channel.ChannelManager.requestSubSystem(ChannelManager.java:735)
    … 12 more
    [01/11/18 18:35:08] [SSH] Copying latest slave.jar…
    Expanded the channel window size to 4MB
    [01/11/18 18:35:08] [SSH] Starting slave process: cd “/home/tushar” && exec java -jar slave.jar
    channel started
    Remoting version: 3.14
    This is a Unix agent
    Evacuated stdout
    Agent successfully connected and online

    Like

  3. Hello Mohit
    Can you share how to configure Jenkins CI is hosted on Linux and we have different Windows VM to run our automation test suite.

    Like

  4. Hi, I did the same and It’s not working yet. I have the next error:

    ERROR: Server rejected the 1 private key(s) for git

    Can you help me please?

    Like

      1. Thanks Mohit.

        I was using the same ssh key that I’m using for connect with GitLab, that was my error. Now create a new ssh key for my slave but when try to connect and run, this is my new error

        Authentication failed.

        On Host Key Verification Strategy with Know host say:

        SSH host key matches key in Known Hosts file. Connection will be allowed.

        But is rejected and I don’t know why. The result is the same with Non verifying verification.

        Like

  5. Thanks Mohit,
    This got me thru the SSH Configuration part of the linux Node/Slave setup. I will add a couple of things I had problems with here just in case someone else encounters them:

    1. JAVA_HOME was not set to the correct value, not sure why this happens. The solution for me was to add and environment variable to the node Name: JAVA_HOME Value:
    2. X11 there was an X problem which causes drivers to fail to start, solution
    sudo apt-get install Xvfb on the slave and add another environment variable to the node configuration in Jenkins Name: DISPLAY , Value: 0: – this got me a bit further
    3. I was using driver.manage().window().maximize() to max the window size, apparently this does not work on a node with Chrome and Xvfb. Switch to options.addArguments(“–start-maximized”);

    Like

  6. How are you able to login to ‘jenkins’ on your master machine ? Per my experience you need to be root to install Jenkins properly on master and account is created as a service account without a shell, meaning there is no user home dir etc – https://stackoverflow.com/questions/18068358/cant-su-to-user-jenkins-after-installing-jenkins
    I attempted to first create CentOS account ‘jenkins’ (sudo user) and then instal Jenkins, this was not very succesfull

    Like

  7. After completing the setup you could run passwd -l jenkins on slave so that only authentication with password is no longer possible. You will still be able to login with the user ssh key or with sudo su – jenkins.

    Like

  8. pravee@ubuntu:~$ ssh-copy-id ubuntu@192.168.75.128
    /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed

    /usr/bin/ssh-copy-id: ERROR: ssh: connect to host 192.168.75.128 port 22: Connection refused. I am getting the error

    Like

  9. I have this problem
    just before slave dev gets launched …
    executing pre-launch scripts …
    Warning: no key algorithms provided; JENKINS-42959 disabled
    [04/25/18 16:54:31] [SSH] Opening SSH connection to 199.100.1.162:22.
    [04/25/18 16:54:31] [SSH] WARNING: No entry currently exists in the Known Hosts file for this host. Connections will be denied until this new host and its associated key is added to the Known Hosts file.
    Key exchange was not finished, connection is closed.
    java.io.IOException: There was a problem while connecting to 199.100.1.162:22
    at com.trilead.ssh2.Connection.connect(Connection.java:834)
    at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1324)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:831)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:820)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    Caused by: java.io.IOException: Key exchange was not finished, connection is closed.
    at com.trilead.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:95)
    at com.trilead.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:237)
    at com.trilead.ssh2.Connection.connect(Connection.java:786)
    … 7 more
    Caused by: java.io.IOException: The server hostkey was not accepted by the verifier callback
    at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:548)
    at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
    at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)
    … 1 more
    [04/25/18 16:54:31] Launch failed – cleaning up connection
    [04/25/18 16:54:31] [SSH] Connection closed.

    Like

  10. Hi Mohit,

    I have deployed the Jenkins WAR file (2.107.2) into WebLogic 12c (12.2.1.2) and login to application. I have established connection between master (where weblogic is running) and slave linux machine identified using ssh public-private key. I have trying the same steps above that you have listed to add a slave into jenkins, but it fails to start with below error

    Warning: no key algorithms provided; JENKINS-42959 disabled
    [04/20/18 08:39:05] [SSH] Opening SSH connection to XXX.abc.com:22.
    ERROR: Unexpected error in launching a slave. This is probably a bug in Jenkins.
    java.lang.NoSuchMethodError: com.trilead.ssh2.Connection.connect(Lcom/trilead/ssh2/ServerHostKeyVerifier;III)Lcom/trilead/ssh2/ConnectionInfo;
    at hudson.plugins.sshslaves.SSHLauncher.openConnection(SSHLauncher.java:1324)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:831)
    at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:820)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    [04/20/18 08:39:05] Launch failed – cleaning up connection
    [04/20/18 08:39:05] [SSH] Connection closed.

    As a result, I am unable to add a node

    Let me know if you have any ideas to try

    Thanks

    Srinivasan

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s