Problem accessing recast eos area in steps.yml

Hi! I’m now having some problem accessing the files in recast eos area. My recast repository is:
https://gitlab.cern.ch/recast-atlas/HDBS/ana-hdbs-2018-19
and my eos place is:
/eos/project-r/recast/atlas/ANA-HDBS-2018-19

When I try to copy some file from the eos area to the working area, I will have error:
2022-02-02 13:31:44,214 | pack.skimming_DAOD.r | INFO | b"cp: cannot stat ‘/eos/user/m/milu/samples/mc16_13TeV.307730.MGPy8EG_A14NNPDF23LO_vbfHVT_Agv1_VzWZ_lvll_m0250.deriv.DAOD_STDM5.e8374_s3126_r9364_p4097/nothing.list’: No such file or directory". But the file does exist there.

I have already setup the variables RECAST_USER, RECAST_PASS and RECAST_TOKEN and I’m able to pull the analysis images properly. I also have:
source /release_setup.sh
export RECAST=True
in my steps.yml (Cern Authentication)
I have checked the username and password, and they should be correct. Because if I deliberately change the password to a wrong one, I will get this error in the .run.log file as:
2022-02-02 10:24:50,785 | pack.skimming_DAOD.r | INFO | b’kinit: Password incorrect while getting initial credentials’
If I set the correct password I will not get this line.

One thing that’s even more strange is that when I did “ls /eos/project-r/recast/atlas/ANA-HDBS-2018-19/ > ls.list”, I got “DAODs” printed in ls.list. However in that path I have totally 5 directories and files: ANNKazuya config DAODs ExportToRF test.txt. But ls showed only 1 of them.

Do you have any idea why this is happening? What have I missed for accessing eos files? Thank you very much!

Hi @milu. The reason that lines like this

cp -r /eos/project-r/recast/atlas/ANA-HDBS-2018-19/DAODs .

don’t work is because cp is only for local files on the same file system. Your workflow running on CI is on an arbitrary runner machine that is not part of the EOS file system. To copy files from EOS to the runner you need to use xrdcp or stream them using xrootd.

This is what we mean in the docs by

All the images and data in the example workflow are public, but in general your analysis may need to pull CERN-internal images from the gitlab registry, and copy private data stored on eos via xrootd.

Here are some additional discussion topics on use of xrootd and xrdcp:

Hi @feickert ,

Thank you for the reply! I tested xrtcp, it can successfully copy the file in my personal eos space to the container with: xrdcp root://eosproject.cern.ch//eos/user/m/milu/samples/mc16_13TeV.307730.MGPy8EG_A14NNPDF23LO_vbfHVT_Agv1_VzWZ_lvll_m0250.deriv.DAOD_STDM5.e8374_s3126_r9364_p4097/nothing.list .

However I can’t copy the file from my RECAST eos area with:
xrdcp root://eosproject.cern.ch//eos/project-r/recast/atlas/ANA-HDBS-2018-19/test.txt .

which gave error: INFO | b’Run: [ERROR] Server responded with an error: [3011] Unable to open file /eos/project-r/recast/atlas/ANA-HDBS-2018-19/test.txt; No such file or directory\x00 (source)’

Do I need special authorization to access files in the RECAST eos space?

If the file exists and you’ve ensured that you have the necessary accounts setup then you have a permission error you need to fix on your side.

c.f. this comment for an example

but in general it is a good idea to use recast auth check-access-image and recast auth check-access-xrootd for the relevant files or directories to ensure that you have the permissions setup before trying to access them.

So for you a start would be to add something like the following to your .gitlab-ci.yml

  # Authentication
  # Authenticate to pull your analysis image(s)
  - eval "$(recast auth setup -a ${RECAST_USER} -a ${RECAST_PASS} -a ${RECAST_TOKEN} -a default)"
  # Authenticate to download inputs from eos via xrootd
  - eval "$(recast auth write --basedir authdir)"

  # Check access to images and files
  - recast auth check-access-xrootd root://eosproject.cern.ch//eos/project/r/recast/atlas/ANA-HDBS-2018-19/

hi @feickert

After doing - recast auth check-access-xrootd root://eosproject.cern.ch//eos/project/r/recast/atlas/ANA-HDBS-2018-19/
I got:
Kerberos: ok milu@CERN.CH
Access: not ok

So it seems that I indeed have an acess problem. How should I fix it from my side? Thanks!

How should I fix it from my side?

You don’t have proper permissions, so you’ll need to check what is causing your access to be deined.

Read through the instructions in the Prerequisites section of the Writing Workflows part of the docs pertaining to accounts and permissions. If you’ve done that and can verify that you have your repository has the proper CI/CD variables set then verify that you are actually on the atlas-ana-hdbs-2018-19-analysis-team egroup (which is how access to the EOS space gets controlled through LDAP).