Consider PU location for top level MC task #9659
Conversation
|
Jenkins results:
|
|
Jenkins results:
|
|
squashing |
_wmbsPreparation needs to receive a list of PNNs, not PSNs
|
Jenkins results:
|
| locations.add(siteInfo[0]['pnn']) | ||
| except Exception as ex: | ||
| self.logger.error('Error getting storage element for "%s": %s' % (site, str(ex))) | ||
| siteInfo = self.getLocationInfo.execute(conn=self.getDBConn(), |
There was a problem hiding this comment.
@amaltaro I have only one question. Why did you take the siteInfo query out of the 'try: - except:' block?
There was a problem hiding this comment.
It should be caught upstream, without breaking the whole cycle, but only this one element.
I also removed it, because as you can see in the old line 352, it wasn't covered by any try/except block as well.
|
Thank you for your comments, Todor. I still think input data and pileup data location should not be correlated in WMBS, only for building the DESIRED_Sites lists. Otherwise we have an impact in the ACDC documents, where data location would actually be their intersection with the pileup location. |
Fixes #9658
Status
ready
Description
When adding MCFakeFile files into
wmbs_file_detailstable, consider the common locations of the WQE instead of setting every single PNN as a candidate for that file.In other words, when fetching a WQE without any input dataset, but with pileup dataset, set the MCFakeFile location to the same location as of the pileup data, such that we ensure that jobs will only be submitted to sites hosting some of the pileup data (be it a top level task or the subsequent ones).
NOTE: we should probably make the same changes to top level tasks *with input data (or ACDC data). I'm not making those changes right now because I believe we would have negative side effects. E.g., misassignment generating an empty file location; e.g., ACDC collections with much less locations than it should... and anything else that still didn't come to my mind.
Is it backward compatible (if not, which system it affects?)
yes
Related PRs
none
External dependencies / deployment changes
none