Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

CMS: check /HIAllPhysics/HIRun2010-ZS-v2/RECO file content #3683

Open
tiborsimko opened this issue Sep 26, 2024 · 6 comments · May be fixed by #3704
Open

CMS: check /HIAllPhysics/HIRun2010-ZS-v2/RECO file content #3683

tiborsimko opened this issue Sep 26, 2024 · 6 comments · May be fixed by #3704
Labels
Experiment: CMS Topic: files

Comments

@tiborsimko
Copy link
Member

The record https://opendata.cern.ch/record/14011 says:

    "number_events": 75482193,
    "number_files": 28954,
    "size": 166030973333340

CMS DAS for /HIAllPhysics/HIRun2010-ZS-v2/RECO says:

        "nevents": 79329398,
        "nfiles": 32715,
        "size": 190908704707416

The record file index says:

  • 32715 files
  • 190908704707416 size

This number of files, the total size, and the listing of individual 32715 file paths from the record index file is fully consistent with what is being reported by CMS DAS.

However, the actual directory content on EOSPUBLIC under /eos/opendata/cms/hidata/HIRun2010/HIAllPhysics/RECO/ZS-v2/ shows:

  • 31600 files
  • 184065703469069 size

  1. We should check and update total number of events and files as appropriate.

  2. There appears to be 1115 missing files; the full list is at https://simko.web.cern.ch/tmp/recid-14011-missing-files.txt
@tiborsimko
Copy link
Member Author

@katilp @jmhogan @tpmccauley Can you please have a look at the list of missing files above? We might need to schedule data transfer to recover these files from the grid?

@jmhogan
Copy link
Contributor

jmhogan commented Oct 16, 2024

@tiborsimko thanks for the poke, I had missed this in my spam folder (I've got filtering set up now, hopefully that happens less).

It does look like these missing files are only reported as existing on T1_US_FNAL_TAPE, so perhaps their transfer didn't complete successfully (the DAS record seems to suggest that).

@katilp Do I have permission to set up Rucio rules as DPOA? I haven't done it before, so I'm not sure.

@katilp
Copy link
Member

katilp commented Oct 23, 2024

@jmhogan Sorry for the delay, it should all be in https://cms-opendata-releaseguide.docs.cern.ch/transfer/site_setup/
Let me check.

@katilp
Copy link
Member

katilp commented Oct 23, 2024

@jmhogan I've added you to t3_ch_cern_opendata_local

@jmhogan
Copy link
Contributor

jmhogan commented Oct 23, 2024

Missing blocks are:

None of them have a rule placing them at our site, so I think I just need to make 7 new rules for these particular blocks.

@jmhogan
Copy link
Contributor

jmhogan commented Oct 23, 2024

Rules:

bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#2c90d627-4424-4126-869a-840defda80f1 1 T3_CH_CERN_OpenData
5d3b53e10ca34e868923fad580f99dc9
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#bebc4ce9-7fad-411b-a789-d9b981e46f6f 1 T3_CH_CERN_OpenData
92373b0f64674b1c8ed6bd4b0cb4093a
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#52656ab9-000d-4b32-ae37-6e60f0dc48e9 1 T3_CH_CERN_OpenData
63d6f7ce6daa4248adbbd8f4d6bd7a73
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#51e3f795-4299-4bf9-b1d1-75064bddb15d 1 T3_CH_CERN_OpenData
48cbd8bcd7974cb0a0714e3d4a31cd49
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#50e679df-4d8c-4a6e-82dd-80cf5bf69d62 1 T3_CH_CERN_OpenData
70eaa77fd2ff41a79a5d588591fbd79e
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#3953b27d-5e00-444b-9621-f007c8f6c7f9 1 T3_CH_CERN_OpenData
70d16e65cc7c4d209ebd62236545a3b8
bash-4.4$ rucio add-rule cms:/HIAllPhysics/HIRun2010-ZS-v2/RECO#3852bf2b-3173-4df9-b267-f3a276a7ccdb 1 T3_CH_CERN_OpenData
905f6160f5c140d6bc7294aa116833e9

@jmhogan jmhogan linked a pull request Dec 11, 2024 that will close this issue
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
Experiment: CMS Topic: files
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

CMS: contents fix for /HIAllPhysics/HIRun2010-ZS-v2/RECO jmhogan/opendata.cern.ch
3 participants
@tiborsimko @katilp @jmhogan