Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

#1650 provide WoT TM validation #1936

Merged
merged 27 commits into from
Aug 26, 2024

Conversation

thjaeckle
Copy link
Member

@thjaeckle thjaeckle commented May 6, 2024

  • restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
  • adding "validator" concept and first sample implementation
  • using a json-schema-validation library to create validation exception details
  • applying validator to:
    • thing attribute updates
    • feature property updates
    • thing actions (inputs and outputs)
    • feature actions (inputs and outputs)
    • thing events
    • feature events
  • each aspect of validation is fully configurable
    • whether to enable/disable
    • also dynamically, e.g. excluding aspects for certain models from validation
  • adding traces in order to determine how much this validation costs "extra" in a productive setup

Open points:

  • Provide actual validation of TMs for all hierarchy levels of changes to Things
  • Optimize conversion from Ditto-Json to Jackson
  • Provide validation for WoT actions and their responses
  • Provide validation for WoT events
  • Add support for WoT oneOf for DataSchema - validation must only match "one of" the provided schemas
  • Provide advanced configuration options to be able to "skip" validation on certain conditions, e.g.:
    • when auth-subject is "xyz"
    • when model URL matches a configured regex
  • Add (enforcement) metrics in order to be able to inspect how much time is spent for validation
  • Provide tests
  • Provide Helm configuration
  • Provide documentation

Resolves: #1650

@thjaeckle thjaeckle added this to the 3.6.0 milestone May 6, 2024
@thjaeckle thjaeckle self-assigned this May 6, 2024
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch 8 times, most recently from 7230638 to 4a8e4b6 Compare May 7, 2024 14:10
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch 2 times, most recently from 45ae2b5 to 49b206e Compare June 24, 2024 07:50
@thjaeckle thjaeckle added the WoT Web of Things related enhancements label Jun 26, 2024
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch 3 times, most recently from c0ab94d to c05c867 Compare July 17, 2024 05:33
thjaeckle added 14 commits July 29, 2024 13:31
* restructuring of "ditto-wot" module to enable re-usability of non-pekko/non-Ditto specifics
* adding "validator" concept and first sample implementation (WIP)

Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
…hingModel

Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
…desired properties validation

* taking care of e.g. "ditto:category" defined
* first validation error will provide the error output (which is for now simpler and results in quicker obtained validation errors)

Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
* validating features and properties as part of "modify thing"
* handling "ditto:category" correctly
* ensuring completeness of defined features
* validation on creation/modification
  * of thing
  * of attributes
  * on attribute
  * on features
  * on feature

Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
…property

Signed-off-by: Thomas Jäckle <thomas.jaeckle@beyonnex.io>
* enforce modifying thing definition + feature definition
* enhance configuration options
…their responses

* also updated some dependencies
* simplified DefaultWotThingModelValidator a little
…iding "type" (valid for e.g. "oneOf") use

* update json-schema-validator library
…ation in smaller pieces

* strict split between config reading in DefaultWotThingModelValidation and static functionality in Internal* classes
thjaeckle added 11 commits July 29, 2024 13:32
* based on either DittoHeaders or on ThingDefinition/FeatureDefinition URLs
* fixed and harmonized other traces while doing that in order to get correct parent hierarchy of spans
…ased on the WoT model

* restructured some too long methods
* fixed remaining TODOs
* also for dynamic part
* changed config structure a bit in order to better fit Helm YAML config
…e corner cases

* e.g. deeply nested updates, removing definition via PATCH
* enabled WoT based validation by default
* first choosing potentially new definition and only as fall-back the old one
@thjaeckle thjaeckle force-pushed the wot-tm-validation branch from dbc03b1 to 608894f Compare July 29, 2024 11:46
@thjaeckle thjaeckle marked this pull request as ready for review July 29, 2024 11:46
@thjaeckle
Copy link
Member Author

FMPOV ready to be reviewed :)

It is of course a massively huge PR - let's discuss how to review it most efficiently @alstanchev @kalinkostashki

@thjaeckle thjaeckle requested a review from alstanchev July 31, 2024 07:15
@thjaeckle
Copy link
Member Author

@alstanchev did you manage to have a look? would want to merge this PR soon ..

@alstanchev
Copy link
Contributor

alstanchev commented Aug 26, 2024

yes, LGTM 👍

@thjaeckle thjaeckle merged commit e4ce12e into eclipse-ditto:master Aug 26, 2024
10 checks passed
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
WoT Web of Things related enhancements
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add option to enforce validation based on referenced WoT model
2 participants