How to I use TRExFitter image in Gitlab CI?

RECAST
#1

When I try to use the trexfitter image in gitlab CI/CD, the CI job always fails with the error

sh: 3: set: Illegal option -o pipefail

The full output from the CI job is here:
https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18600626

I think this may have to do with this image using the dash shell instead of bash, but I don’t know how to fix it in a CI job

Thanks.
~Joe

#2

@jhaley The GitLab repository is private and so not viewable (I’m getting a 404). Can you share the contents of the .gitlab-ci.yml or at least the relevant sections of it?

#3

Ah, sorry about that. Here is the current version of.gitlab-ci

variables:
 GIT_SUBMODULE_STRATEGY: recursive # Now TRExFitter and its submodules will be cloned in the CI job

stages:
  - greeting
  - dockerimage
  - prepare_trex_input
  - run_trex
  
hello world:
  stage: greeting
  script:
    - echo "Hello World"
    - find . -path ./.git -prune -o -print

make docker image:
  stage: dockerimage
  image:
    name: gitlab-registry.cern.ch/ci-tools/docker-image-builder
    entrypoint: [""]
  script:
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
    - /kaniko/executor --context "${CI_PROJECT_DIR}"
                       --dockerfile "${CI_PROJECT_DIR}/Dockerfile"
                       --destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHORT_SHA}"
  
prepare input for trex:
  stage: prepare_trex_input
  image: atlas/analysisbase:21.2.186 #ATLAS analysisbase image
  before_script:
    - echo $SERVICE_PASS | kinit $CERN_USER
    - . /release_setup.sh
  script:
    - echo $PWD
    - python prepare_fit_input.py ci_test_input.txt 10 a 50
    - ls -l Fit

run trex:
  stage: run_trex
  image: gitlab-registry.cern.ch/trexstats/trexfitter:trexfitter-00-04-15
  allow_failure: true
  script:
    - echo $PWD
    - cd /
    - echo $PWD
    - trex-fitter h /workdir/analysis/Fit/TRexConfig.rex
    - trex-fitter d /workdir/analysis/Fit/TRexConfig.rex
    - trex-fitter w /workdir/analysis/Fit/TRexConfig.rex
    - ls -l BoostedAllhadronicVLQAnalysisRECASTFit

And here is the full ci output from the run_tex stage:

[1]Running with gitlab-runner 14.4.0 (4b9e985a)

[2](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L2) on default-runner-58cd994587-rzw5x SPD8yozD

[3](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L3) feature flags: FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR:true

[4](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L4)Resolving secrets00:00

[6](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L6)Preparing the "docker" executor00:20

[7](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L7)Using Docker executor with image gitlab-registry.cern.ch/trexstats/trexfitter:trexfitter-00-04-15 ...

[8](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L8)Authenticating with credentials from job payload (GitLab Registry)

[9](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L9)Pulling docker image gitlab-registry.cern.ch/trexstats/trexfitter:trexfitter-00-04-15 ...

[10](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L10)Using docker image sha256:524facfb322c0ca67385aa9d46958434326f3c546736c74cd7ba734684e25cd5 for gitlab-registry.cern.ch/trexstats/trexfitter:trexfitter-00-04-15 with digest gitlab-registry.cern.ch/trexstats/trexfitter@sha256:d4aa6dce40f3bcbc3955d0169db1f74347e663abb85d76cc10e09f03c35f9402 ...

[11](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L11)Not using umask - FF_DISABLE_UMASK_FOR_DOCKER_EXECUTOR is set!

[13](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L13)Preparing environment00:03

[14](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L14)Running on runner-spd8yozd-project-116797-concurrent-0 via default-runner-58cd994587-rzw5x...

[16](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L16)Getting source from Git repository00:04

[17](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L17)Fetching changes with git depth set to 50...

[18](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L18)Initialized empty Git repository in /builds/sasingh/all-hadronic-vlq-t_ht/.git/

[19](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L19)Created fresh repository.

[20](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L20)Checking out 719be236 as RECAST-docker...

[21](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L21)Updating/initializing submodules recursively with git depth set to 50...

[23](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L23)Executing "step_script" stage of the job script00:01

[24](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L24)Using docker image sha256:524facfb322c0ca67385aa9d46958434326f3c546736c74cd7ba734684e25cd5 for gitlab-registry.cern.ch/trexstats/trexfitter:trexfitter-00-04-15 with digest gitlab-registry.cern.ch/trexstats/trexfitter@sha256:d4aa6dce40f3bcbc3955d0169db1f74347e663abb85d76cc10e09f03c35f9402 ...

[25](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L25)sh: 3: set: Illegal option -o pipefail

[27](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L27)Cleaning up project directory and file based variables00:01

[29](https://gitlab.cern.ch/sasingh/all-hadronic-vlq-t_ht/-/jobs/18602589#L29)ERROR: Job failed: exit code 2
#4

This is a known GitLab bug, that they don’t treat as such. (c.f. Run your CI/CD jobs in Docker containers | GitLab).

The fix is just to override entrypoint everywhere you use a custom image with [""]. e.g.:

  image:
    name: gitlab-registry.cern.ch/trexstats/trexfitter:trexfitter-00-04-15
    entrypoint: [""]
#5

That did it. Thank you!

~Joe