Unable to run a python script inside docker from jenkins pipeline

Question

The reason why I'm asking this question here is due to ros buildfarm. This could be completely out of context for the community. I'm just looking for any sort of help possible.

I created a python package (for automating the build stages in jenkins) that can be installed using pip. Also I created a dockerfile that will clone the python package from the github, perform pip install and export the path where the executables (entry points, however you would like to call it) are installed in the container (example: ~/.local/bin). This is how my dockerfile looks like.

FROM ros:melodic-ros-core-stretch
RUN apt-get update && apt-get -y install python-pip
RUN git clone <private-repo-with-personal-access-token>
RUN pip install <package-name>
RUN export PATH=~/.local/bin:$PATH

So I built this image, ran the container and typed in one of the executables (entry points) that prints hello world. Works perfectly fine. Have no problem whatsoever on that. Coming to the big picture, I want to call this executable (entry points) in the jenkins pipeline. I initially had a problem with that and then I learnt that the jenkins pipeline when setup with a docker image, runs on top of the container but uses the workspace allocated for jenkins which will be /var/lib/jenkins/workspace. This is how my jenkins pipeline script looks like

pipeline {
    agent {
         docker {
              args '--network host -u root:root'
              image '<private-docker-hub-image>'
              registryCredentialsId 'docker-credentials'
              registryUrl 'https://registry.hub.docker.com'
         }
    }
    stages {
        stage('Test') {
            steps {
                sh 'test-build'
            }
        }
    }
}

This is the error I'm getting.

Started by user Automated Build Environment
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/First_item
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] withDockerRegistry
$ docker login -u <docker username> -p ******** https://registry.hub.docker.com
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /var/lib/jenkins/workspace/First_item@tmp/251189be-62eb-4134-84ca-d70190ab080f/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . <private-dockerhub-image>

Error: No such object: <private-dockerhub-image>
[Pipeline] sh
+ docker inspect -f . registry.hub.docker.com/<private-dockerhub-image>
.
[Pipeline] withDockerContainer
Jenkins does not seem to be running inside a container
$ docker run -t -d -u 125:130 --network host -u root:root -v /var/lib/jenkins:/var/lib/jenkins -w /var/lib/jenkins/workspace/First_item -v /var/lib/jenkins/workspace/First_item:/var/lib/jenkins/workspace/First_item:rw,z -v /var/lib/jenkins/workspace/First_item@tmp:/var/lib/jenkins/workspace/First_item@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** registry.hub.docker.com/<private-dockerhub-image> cat
$ docker top ab1f2ec9b503f9b916ee96943dc849acc90716f7543b4841ff3901b7a65aea54 -eo pid,comm
[Pipeline] {
[Pipeline] stage (hide)
[Pipeline] { (Test)
[Pipeline] sh
+ test-build
/var/lib/jenkins/workspace/First_item@tmp/durable-7cb108f7/script.sh: 1: /var/lib/jenkins/workspace/First_item@tmp/durable-7cb108f7/script.sh: test-build: not found
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
$ docker stop --time=1 ab1f2ec9b503f9b916ee96943dc849acc90716f7543b4841ff3901b7a65aea54
$ docker rm -f ab1f2ec9b503f9b916ee96943dc849acc90716f7543b4841ff3901b7a65aea54
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // withDockerRegistry
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 127
Finished: FAILURE

My question is how does rosdep, bloom-generate and several other scripts work in the pipeline but not the scripts that I installed through pip. I would really appreciate any help in pointing me on what I'm missing.

Thank you. Help much appreciated.

---

Originally posted by venkisagunner on ROS Answers with karma: 89 on 2019-02-22

Post score: 0

Answer 1

Sorry. The solution was really simple. The mistake I was making was I updated the dockerfile and created a new image, pushed it to dockerhub but forgot to prune and update the latest copy in the instance where Jenkins master was running. So basically my dockerhub had lets just say version. 101 and the local machine where I created the dockerfile and docker image had version. 101. But the Jenkins master was in v.52. So even though I made changes the updated image was not pulled by the Jenkins master and boom there you go ! This was a dumb mistake on my part.

---

Originally posted by venkisagunner with karma: 89 on 2019-02-22

This answer was ACCEPTED on the original site

Post score: 0