Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

ECS schema validation in logger #58

Open
nachiket-lab opened this issue Sep 8, 2021 · 3 comments
Open

ECS schema validation in logger #58

nachiket-lab opened this issue Sep 8, 2021 · 3 comments

Comments

@nachiket-lab
Copy link

Hi,

This is more of a question than an issue. Is there a validation feature in ecs-logging-python library? I wish to avoid mapping conflict later on during the application on-boarding process, and I wish to make the app ECS compatible from it's very inception.

Currently adding extra fields is simple:

logger.debug(f"Creating directory. {x}", extra={"not_an_ecs_field.method": "get", "source.ip": 22})

I have added two fields, one of which is not really an ECS schema field while the other has an invalid data type. I was wondering if I could pass in the ECS schema as a JSON and have the logger do the validation. I read an option to pass in a validator here, but not sure what it expects.

@basepi
Copy link
Contributor

basepi commented Sep 13, 2021

We validate the spec in our tests using this fixture. We don't currently validate in code because that's a bit of overhead that we'd like to avoid, in general. It's possible we could make this usable inside of the logging classes, behind a configuration flag.

Note that the validate kwarg that you found is only a boolean, it just tells the formatter to throw an error if there is invalid or mismatched style and fmt kwargs. We would need to build this in ourselves.

@nachiket-lab
Copy link
Author

We could use Pydantic to do the validation, and ensure that the schema is maintained. I guess you could also do it using normal python dataclass definition, but not sure about the overhead it would have in either of the two use-cases.
I have the models ready, and they are generated using this file from ECS.
Do you think either of the approaches make sense?

@basepi
Copy link
Contributor

basepi commented Sep 14, 2021

Pydantic certainly seems like it would be the better path, as it's known for its speed. Our implementation was just for testing and so is not optimized.

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

2 participants