Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Missing epo:hasLegalBasis for certain sdk versions #52

Closed
cristianvasquez opened this issue Sep 2, 2024 · 5 comments · Fixed by #55
Closed

Missing epo:hasLegalBasis for certain sdk versions #52

cristianvasquez opened this issue Sep 2, 2024 · 5 comments · Fixed by #55
Assignees
@schivmeister
Labels
bug Something isn't working

Comments

@cristianvasquez
Copy link
Contributor

cristianvasquez commented Sep 2, 2024 •
edited
Loading

Hello,

Notice 00152054-2024 constains a cbc:RegulatoryDomain referring to the Procedure Legal Basis, but no corresponding value for epo:hasLegalBasis was found

<cbc:RegulatoryDomain>32014L0023</cbc:RegulatoryDomain>

Source XML: https://ted.europa.eu/en/notice/152054-2024/xml

This issue has been reported in the following cases:

  • EF14 for SDK version 1.7, 1.8 and 1.10  (00152266-2024, 00152054-2024, 00191175-2024)
  • EF15 for SDK version 1.7, 1.8 and 1.10 (00140913-2024, 00179143-2024, 00144555-2024)
  • EF16 for SDK version 1.6 and 1.7 (00127096-2024, 00126821-2024)
  • EF19 for SDK version 1.6, 1.8 and 1.10  (00129735-2024, 00129801-2024, 00128154-2024)

Some notices contain epo:hasLegalBasis, for example EF12 for SDK version 1.10
@schivmeister schivmeister self-assigned this Sep 2, 2024
@schivmeister
Copy link
Collaborator

Thanks for reporting. The issue is that some notices, such as PriorInformation, do not have the toplevel cac:TenderingProcess. This throws off the corresponding mapping as it is under a wrong iterator (it should not be at the iterator of the TenderingProcess but the root).

@schivmeister schivmeister added the bug Something isn't working label Sep 2, 2024
@muricna
Copy link

muricna commented Sep 3, 2024

This is not clear to me as only EF14 is a PIN used as a call for competition, EF15 is a qualification system and both EF 16 and EF 19 are competition notices.

@schivmeister
Copy link
Collaborator

@muricna that is indeed correct. however, we are not able to reproduce the reported issue for most of the cited notices, i.e. those other than the PIN (EF14) and the qualification system (EF15).

@cristianvasquez it would be good if you can confirm how you came to report the competition notices. Perhaps, there was a misunderstanding/misinterpretation, or another underlying issue that is being exposed?

@OceaneAbrard
Copy link
Collaborator

Hello,
I verified again the notices that Cristian has reported using the mapping present in the Tags 1.0.0-rc.7 and the epo:hasLegalBasis is still missing from the following notices:
EF14 for SDK version 1.7, 1.8 and 1.10 (00152266-2024, 00152054-2024, 00191175-2024)
EF15 for SDK version 1.7, 1.8 and 1.10 (00140913-2024, 00179143-2024, 00144555-2024)
EF19 for SDK version 1.6, 1.8 and 1.10 (00129490-2024, 00129801-2024, 00128154-2024)

I verified the EF16 that were reported, and these ones are no longer missing the epo:hasLegalBasis.

@schivmeister
Copy link
Collaborator

@OceaneAbrard Thank you for confirming. It is expected that EF19 will still exhibit this issue, as they appear to be competition notices that do not have a tendering process defined at the root of the notice, and the mapping was not written to deal with that. This will be fixed. Only EF16 is not reproducible, in that case, and will be assumed to not be affected.

P.S: Note that it would be best if all testing in this context is carried out against the tag 1.0.0-rc.6, upon which this ticket is and the next CN release will be based, as rc.7 is a CAN release only.

schivmeister added a commit that referenced this issue Sep 20, 2024
@schivmeister
Fix gh-52 missing hasLegalBasis for certain non-EF16 CN notices
33f9605 
The predicate mapping was in the wrong TriplesMap (TMap) that has an
incompatible iterator (a toplevel `TenderingProcess`, which is not
guaranteed to exist), and an absolute XPath reference. It should've been
in the root TMap which has the right iterator (simply the root of the
XML document `/*`, which also allows a relative XPath in the mapping).

Affects EF14, EF15 and EF19 as reported. Tested with EF14 notice
152054-2024.xml.

The following cases have data (from Official Journal publications)
included for testing:

- EF14 for SDK version 1.8 (00152054-2024)
- EF15 for SDK version 1.7 (00140913-2024)
- EF19 for SDK version 1.10 (00128154-2024)

As downloaded from URLs of the form
https://ted.europa.eu/en/notice/152054-2024/xml (first example).

Closes gh-52.
@schivmeister schivmeister mentioned this issue Sep 20, 2024
Merged
schivmeister added a commit that referenced this issue Sep 20, 2024
@schivmeister
Fix gh-52 missing hasLegalBasis for certain non-EF16 CN notices
1d7253e 
The predicate mapping was in the wrong TriplesMap (TMap) that has an
incompatible iterator (a toplevel `TenderingProcess`, which is not
guaranteed to exist), and an absolute XPath reference. It should've been
in the root TMap which has the right iterator (simply the root of the
XML document `/*`, which also allows a relative XPath in the mapping).

Affects EF14, EF15 and EF19 as reported. Tested with EF14 notice
152054-2024.xml.

The following cases have data (from Official Journal publications)
included for testing:

- EF14 for SDK version 1.8 (00152054-2024)
- EF15 for SDK version 1.7 (00140913-2024)
- EF19 for SDK version 1.10 (00128154-2024)

As downloaded from URLs of the form
https://ted.europa.eu/en/notice/152054-2024/xml (first example).

Closes gh-52.
schivmeister added a commit that referenced this issue Sep 20, 2024
@schivmeister
Add output and validation assets for CN MS D2.2 second revision
a4b2fb8 
As part of the update to CN MS v1.3-1.8+1.10, namely implementing fix
for gh-52.

Assertion changes are merely renames as they are the same content with
different file names due to newly generated IDs, so can be ignored.

The changes to reports can also be ignored, as they do not always change
functionally across executions, with the exception of only those parts
that reflect the change(s), of which there is very little in this
iteration.

The true changes can be observed among the RDF outputs, namely in the
newly added test data files, showing the introduction of the
`epo:hasLegalBasis` predicate. One may diff against the latest changeset
requesting statistics only on the CN Turtle files, in the following
manner:

```sh
$ git diff HEAD~ --stat mappings/package_cn_v1.*/output/*/*/*.ttl
 .../package_cn_v1.10/output/op_test_cn_gh_issues/2024-OJS044-00128154/2024-OJS044-00128154.ttl     | 1527 ++++++++++++++++++++++++++++++++++++++++++++++++
 mappings/package_cn_v1.10/output/sdk_examples_cn/cn_23/cn_23.ttl                                   |    1 +
 .../package_cn_v1.10/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl    |    3 +-
 mappings/package_cn_v1.3/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.3/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
 mappings/package_cn_v1.4/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.4/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
 mappings/package_cn_v1.5/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.5/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
 mappings/package_cn_v1.6/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.6/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
 mappings/package_cn_v1.7/output/op_test_cn_gh_issues/2024-OJS048-00140913/2024-OJS048-00140913.ttl |  355 +++++++++++
 mappings/package_cn_v1.7/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.7/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
 mappings/package_cn_v1.8/output/op_test_cn_gh_issues/2024-OJS052-00152054/2024-OJS052-00152054.ttl |  296 ++++++++++
 mappings/package_cn_v1.8/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.8/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
 mappings/package_cn_v1.9/output/sdk_examples_cn/cn_23/cn_23.ttl                                    |    1 +
 .../package_cn_v1.9/output/sdk_examples_cn_invalid/INVALID_cn_24_empty/INVALID_cn_24_empty.ttl     |    3 +-
```

Notice that some existing example data are modified, in particular, with
said predicate `epo:hasLegalBasis`.
@schivmeister schivmeister mentioned this issue Sep 20, 2024
Merged
csnyulas added a commit that referenced this issue Sep 20, 2024
@csnyulas
Merge pull request #55 from OP-TED/feature/d22-fixes-r1
062abdf 
Fix gh-52 missing hasLegalBasis for certain non-EF16 CN notices
schivmeister added a commit that referenced this issue Sep 27, 2024
@schivmeister
Apply fix for gh-52 missing hasLegalBasis to CN rules in CAN
d37bc96 
Now bring CAN up to parity with CN :)
schivmeister added a commit that referenced this issue Oct 2, 2024
@schivmeister
Add output assets for CAN MS D4.1 revision
e2db2a6 
- refersToProcedure gh-59
- MonetaryValue instantiation gh-56
- hasLegalBasis gh-52 (port from CN D2.2 second revision)
- announcesRole PaymentExecutor / BudgetProvider (sample data added)
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees

@schivmeister schivmeister

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix gh-52 missing hasLegalBasis for certain non-EF16 CN notices OP-TED/ted-rdf-mapping-eforms
4 participants
@cristianvasquez @schivmeister @muricna @OceaneAbrard