Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

release 0.4.0 #39

Merged
merged 10 commits into from
Jan 20, 2022
Merged

release 0.4.0 #39

merged 10 commits into from
Jan 20, 2022

Conversation

san-zrl
Copy link
Contributor

@san-zrl san-zrl commented Dec 22, 2021

This PR contains the latest changes for release 0.4.0 of sf-processor:

  • Simplified processor modes to enrich and alert
    • enrich adds policy info, priority, and tags records that match rules. Non-matching records are pushed as is (non-blocking). This is the default mode.
    • alert adds policy info, priority, and tags to matching records and raises them to alerts. Non-matching records are blocked.
  • drop rules can be used to pre-filter the record stream before applying rules. The old filter keyword is silently supported (not documented).
  • Policy engine processes records concurrently in a thread pool of workers. The pool size is configurable (default is 5). Record processing is asynchronous, there is no guarantee on the record order in the output when processing records concurrently.
  • Support for pluggable user-defined actions. Actions are specified using the action key in a rule. Value is a comma-separated list of action names. Each action can add context such as tags to the record. Actions attached to a particular rule are executed in the order they were specified.
  • The hash feature is not part of this PR. This version does not provide any built-in actions.
  • There is no alert action anymore. Policy files in the repo are updated.
  • Updated documentation. This also includes documentation in writing user-defined action plugins.

Signed-off-by: Andreas Schade <san@zurich.ibm.com>
Signed-off-by: Andreas Schade <san@zurich.ibm.com>
Signed-off-by: Andreas Schade <san@zurich.ibm.com>
Signed-off-by: Andreas Schade <san@zurich.ibm.com>
Signed-off-by: Andreas Schade <san@zurich.ibm.com>
Signed-off-by: Andreas Schade <san@zurich.ibm.com>
@araujof araujof self-requested a review January 10, 2022 21:24
@araujof araujof self-assigned this Jan 10, 2022
Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
…p lifecycle when used with policy monitor; refactor policy monitor and simplified usage in policy engine

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
…nterpreter constructor

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof
Copy link
Member

araujof commented Jan 20, 2022

Thanks for this PR. It fixes the ambiguity we had in the policy engine modes and add pluggable, user-defined actions to the processor. Nice job! I went through the code changes, and made some additional changes, fixes, and refactoring:

  • fix linting issues: comment style, error message style, naming style, etc.
  • refactor thread pool and fix start/stop lifecycle when used with policy monitor; if using the monitor, the interpreter pool wouldn't be closer/stopped prior to assigning a new interpreter.
  • refactor policy monitor and simplified usage in policy engine (unrelated to this PR, but since I was there :))

I tested the pipeline modes, in both enrich and alert mode, and it looks good.

Copy link
Member

@araujof araujof left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

…ith monitor parameters

Signed-off-by: Frederico Araujo <frederico.araujo@ibm.com>
@araujof araujof merged commit 5300e52 into sysflow-telemetry:dev Jan 20, 2022
araujof added a commit that referenced this pull request Jan 15, 2025
This PR fixes the mode ambiguity in the policy engine and adds pluggable action handling to the processor.
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants