Running messageWriter in REANA

RECAST
#1

Hi,

to familiarize myself with REANA I’m trying to run something very simple in it. Like the messageWriter job from lesson 7 of the RECAST tutorial. My setup (on lxplus) follows:

lxplus7122 messageWriter % tree
.
├── reana.yml
└── specs
    ├── steps.yml
    └── workflow.yml

1 directory, 3 files

lxplus7122 messageWriter % cat reana.yml
# reana.yaml
version: 0.6.0
inputs:
  parameters:
    msg: 'Technically correct is the best kind of correct'
  directories:
    - specs
workflow:
  type: yadage
  file: specs/workflow.yml
outputs:
  files:
    - outputfile.txt

lxplus7122 messageWriter % cat specs/steps.yml
messagewriter:
  environment:
    environment_type: docker-encapsulated
    image: yadage/tutorial-messagewriter
    imagetag: latest
  process:
    process_type: interpolated-script-cmd
    script: |
      /code/message_writer '{message}' {outputfile}
  publisher:
    publisher_type: interpolated-pub
    publish:
      msgfile: '{outputfile}'

lxplus7122 messageWriter % cat specs/workflow.yml
stages:
- name: writing_stage
  dependencies: [init]
  scheduler:
    scheduler_type: singlestep-stage
    parameters:
      message: {step: init, output: msg}
      outputfile: '{workdir}/outputfile.txt'
    step: {$ref: specs/steps.yml#/messagewriter}

When I validate all this, I get a warning among other things:

lxplus7122 messageWriter % ~/.local/bin/reana-client validate
==> Verifying REANA specification file... /afs/cern.ch/user/y/ysmirnov/public/data2015_2018_analysis/reana/messageWriter/reana.yml
  -> SUCCESS: Valid REANA specification file.
==> Verifying workflow parameters and commands...
  -> WARNING: Yadage parameter "msgfile" found on step "writing_stage" is not defined in input parameters.

I don’t really understand what it means. How bad is it?
Finally, after I submit it to REANA, the job fails after a few minutes. The logs do not tell me much as they appear to be empty. Or maybe I’m looking in the wrong place?

Any idea what’s wrong here? How do I go about debugging a job like this?

Thanks!

1 Like
#2

A quick update on this:
I figured that validation succeeds with no warnings or anything with the following specs:

lxplus705 messageWriter % cat reana.yml
# reana.yaml
version: 0.6.0
inputs:
  parameters:
    msg: 'Technically correct is the best kind of correct'
    msgfile: '{workdir}/outputfile.txt'
  directories:
    - specs
workflow:
  type: yadage
  file: specs/workflow.yml
outputs:
  files:
    - outputfile.txt

lxplus705 messageWriter % cat specs/workflow.yml
stages:
- name: writing_stage
  dependencies: [init]
  scheduler:
    scheduler_type: singlestep-stage
    parameters:
      message: {step: init, output: msg}
      outputfile: {step: init, output: msgfile}
    step: {$ref: specs/steps.yml#/messagewriter}

, no changes for the specs/steps.yml.

I however haven’t been able to check if it actually works as I’m having difficulties running anything in REANA as I explained here.