Tenzir Node v4.20

Tenzir Node v4.20 is here, bringing a host of under-the-hood improvements that pave the way for exciting upcoming changes to the Tenzir Platform.

Discover Tenzir Node v4.20

📣 Dive into the details of Tenzir Node v4.20 from our announcement blog post.
🎯 Check out the new features on app.tenzir.com.
🚀 Follow the quickstart guide, get your fingers dirty and explore the demo node with just a few clicks and a guided tour. Security data has never been easier.
🎓 Learn the basics of pipelines and get familiar with the Tenzir Query Language (TQL). You'll master security data in no time.
💭 Understand why Tenzir was developed and how it measures up against the competition.
📝 Read all the details in our changelog.

Tenzir v4.19.6

This patch release contains the following bug fixes and improvements over Tenzir v4.19:

Tenzir v4.19.6

• Pipelines with the python operator now deploy more quickly, as their deployment no longer waits for the virtual environment to be set up successfully.
• We fixed a bug in the shell operator that could cause the process to crash when breaking its pipe. Now, the operator shuts down with an error diagnostic instead.
• The slice operator no longer crashes when used with a positive begin and negative end value when operating on less events than -end, e.g., when working on a single event and using slice 0:-1.
• The tenzir command-line utility gained a new option --strict, causing it to exit with a non-zero exit code for pipelines that emit at least one warning.

Tenzir v4.19.5

• The serve operator no longer uses an excessive amount of CPU.

Tenzir v4.19.4

• The packages plugin is now available in the static binary release artifacts.

Tenzir v4.19.3

• Pipelines from packages now correctly remember their last run number and last state when reinstalling the package.

Tenzir v4.19.2

• The subscribe operator now delivers metrics more consistently.
• The throttle operator allows for limiting the bandwidth of a pipeline.
• We've made some changes that optimize Tenzir's memory usage. Pipeline operators that emit very small batches of events or bytes at a high frequency now use less memory. The serve operator's internal buffer is now soft-capped at 1Ki instead of 64Ki events, aligning the buffer size with the default upper limit for the number of events that can be fetched at once from /serve. The export, metrics, and diagnostics operators now handle back pressure better and utilize less memory in situations where the node has many small partitions. For expert users, the new tenzir.demand configuration section allows for controlling how eagerly operators demand input from their upstream operators. Lowering the demand reduces the peak memory usage of pipelines at some performance cost.

Tenzir v4.19.1

• We fixed a problem where a stoppingpython operator could cause a crash.
• The python operator is now compatible with Python 3.9 again.
• The AMQP connector now handles connections with heartbeats correctly.
• We restored the performance of the partitions operator to pre --experimental-include-ranges times, when the flag is not used.

Tenzir v4.19.5

This patch release contains the following bug fixes and improvements over Tenzir v4.19:

Tenzir v4.19.5

• The serve operator no longer uses an excessive amount of CPU.

Tenzir v4.19.4

• The packages plugin is now available in the static binary release artifacts.

Tenzir v4.19.3

• Pipelines from packages now correctly remember their last run number and last state when reinstalling the package.

Tenzir v4.19.2

• The subscribe operator now delivers metrics more consistently.
• The throttle operator allows for limiting the bandwidth of a pipeline.
• We've made some changes that optimize Tenzir's memory usage. Pipeline operators that emit very small batches of events or bytes at a high frequency now use less memory. The serve operator's internal buffer is now soft-capped at 1Ki instead of 64Ki events, aligning the buffer size with the default upper limit for the number of events that can be fetched at once from /serve. The export, metrics, and diagnostics operators now handle back pressure better and utilize less memory in situations where the node has many small partitions. For expert users, the new tenzir.demand configuration section allows for controlling how eagerly operators demand input from their upstream operators. Lowering the demand reduces the peak memory usage of pipelines at some performance cost.

Tenzir v4.19.1

• We fixed a problem where a stoppingpython operator could cause a crash.
• The python operator is now compatible with Python 3.9 again.
• The AMQP connector now handles connections with heartbeats correctly.
• We restored the performance of the partitions operator to pre --experimental-include-ranges times, when the flag is not used.

Tenzir v4.19.4

This patch release contains the following bug fixes and improvements over Tenzir v4.19:

Tenzir v4.19.4

• The packages plugin is now available in the static binary release artifacts.

Tenzir v4.19.3

• Pipelines from packages now correctly remember their last run number and last state when reinstalling the package.

Tenzir v4.19.2

• The subscribe operator now delivers metrics more consistently.
• The throttle operator allows for limiting the bandwidth of a pipeline.
• We've made some changes that optimize Tenzir's memory usage. Pipeline operators that emit very small batches of events or bytes at a high frequency now use less memory. The serve operator's internal buffer is now soft-capped at 1Ki instead of 64Ki events, aligning the buffer size with the default upper limit for the number of events that can be fetched at once from /serve. The export, metrics, and diagnostics operators now handle back pressure better and utilize less memory in situations where the node has many small partitions. For expert users, the new tenzir.demand configuration section allows for controlling how eagerly operators demand input from their upstream operators. Lowering the demand reduces the peak memory usage of pipelines at some performance cost.

Tenzir v4.19.1

• We fixed a problem where a stoppingpython operator could cause a crash.
• The python operator is now compatible with Python 3.9 again.
• The AMQP connector now handles connections with heartbeats correctly.
• We restored the performance of the partitions operator to pre --experimental-include-ranges times, when the flag is not used.

Tenzir v4.19.3

This patch release contains the following bug fixes and improvements over Tenzir v4.19:

Tenzir v4.19.3

• Pipelines from packages now correctly remember their last run number and last state when reinstalling the package.

Tenzir v4.19.2

• The subscribe operator now delivers metrics more consistently.
• The throttle operator allows for limiting the bandwidth of a pipeline.
• We've made some changes that optimize Tenzir's memory usage. Pipeline operators that emit very small batches of events or bytes at a high frequency now use less memory. The serve operator's internal buffer is now soft-capped at 1Ki instead of 64Ki events, aligning the buffer size with the default upper limit for the number of events that can be fetched at once from /serve. The export, metrics, and diagnostics operators now handle back pressure better and utilize less memory in situations where the node has many small partitions. For expert users, the new tenzir.demand configuration section allows for controlling how eagerly operators demand input from their upstream operators. Lowering the demand reduces the peak memory usage of pipelines at some performance cost.

Tenzir v4.19.1

• We fixed a problem where a stoppingpython operator could cause a crash.
• The python operator is now compatible with Python 3.9 again.
• The AMQP connector now handles connections with heartbeats correctly.
• We restored the performance of the partitions operator to pre --experimental-include-ranges times, when the flag is not used.

Tenzir v4.19.2

This patch release contains the following bug fixes and improvements over Tenzir v4.19:

Tenzir v4.19.2

• The subscribe operator now delivers metrics more consistently.
• The throttle operator allows for limiting the bandwidth of a pipeline.
• We've made some changes that optimize Tenzir's memory usage. Pipeline operators that emit very small batches of events or bytes at a high frequency now use less memory. The serve operator's internal buffer is now soft-capped at 1Ki instead of 64Ki events, aligning the buffer size with the default upper limit for the number of events that can be fetched at once from /serve. The export, metrics, and diagnostics operators now handle back pressure better and utilize less memory in situations where the node has many small partitions. For expert users, the new tenzir.demand configuration section allows for controlling how eagerly operators demand input from their upstream operators. Lowering the demand reduces the peak memory usage of pipelines at some performance cost.

Tenzir v4.19.1

• We fixed a problem where a stoppingpython operator could cause a crash.
• The python operator is now compatible with Python 3.9 again.
• The AMQP connector now handles connections with heartbeats correctly.
• We restored the performance of the partitions operator to pre --experimental-include-ranges times, when the flag is not used.

Tenzir v4.19.1

This patch release contains the following bug fixes and improvements over Tenzir v4.19:

Tenzir v4.19.1

• We fixed a problem where a stoppingpython operator could cause a crash.
• The python operator is now compatible with Python 3.9 again.
• The AMQP connector now handles connections with heartbeats correctly. Thank you @satta!
• We restored the performance of the partitions operator to pre --experimental-include-ranges times, when the flag is not used.

Tenzir v4.19

With the new Tenzir 4.19 release you can now write and deploy packages, enjoy the benefits of the improved python operator, and use the buffer operator to control back pressure.

Discover Tenzir v4.19

📣 Dive into the details of Tenzir v4.19 from our announcement blog post.
🎯 Check out the new features on app.tenzir.com.
🎓 Learn more by following the getting started guide and user guides.
💭 Understand why Tenzir was developed and how it measures up against the competition.
📝 Read all the details in our changelog.

Tenzir v4.18.5

This patch release contains the following bug fixes and improvements over Tenzir v4.18:

Tenzir v4.18.5

• The unflatten operator now correctly preserves field order and overwrites in case of a name conflict.

Tenzir v4.18.4

• The subscribe operator no longer propagates back pressure to its corresponding publish operators when part of a pipeline that runs in the background, i.e., is not visible on the overview page on app.tenzir.com. An invisible subscriber should never be able to slow down a publisher.

Tenzir v4.18.3

• Fixed an issue where null records were sometimes transformed into non-null records with null fields.
• We fixed an issue that sometimes caused subscribe to fail when multiple publish operators pushed to the same topic at the exact same time.
• We fixed a bug that caused a potentially unbounded memory usage in export --live, metrics --live, and diagnostics --live.
• metrics export now includes an additional field that shows the number of queued events in the pipeline.

Tenzir v4.18.2

• We fixed a memory leak in export that was introduced with v4.18.1.

Tenzir v4.18.1

• We fixed a bug that caused deduplicate <fields...> --distance <distance> to sometimes produce incorrect results when followed by where <expr> with an expression that filters on the deduplicated fields.
• Pipelines that use the every modifier with the export operator no longer terminate after the first run.
• Setting the tenzir.endpoint option to false now causes the node not to listen for node-to-node connections. Previously, the port was always exposed for other nodes or tenzir processes to connect.

Tenzir v4.18.4

This patch release contains the following bug fixes and improvements over Tenzir v4.18:

Tenzir v4.18.4

• The subscribe operator no longer propagates back pressure to its corresponding publish operators when part of a pipeline that runs in the background, i.e., is not visible on the overview page on app.tenzir.com. An invisible subscriber should never be able to slow down a publisher.

Tenzir v4.18.3

• Fixed an issue where null records were sometimes transformed into non-null records with null fields.
• We fixed an issue that sometimes caused subscribe to fail when multiple publish operators pushed to the same topic at the exact same time.
• We fixed a bug that caused a potentially unbounded memory usage in export --live, metrics --live, and diagnostics --live.
• metrics export now includes an additional field that shows the number of queued events in the pipeline.

Tenzir v4.18.2

• We fixed a memory leak in export that was introduced with v4.18.1.

Tenzir v4.18.1

• We fixed a bug that caused deduplicate <fields...> --distance <distance> to sometimes produce incorrect results when followed by where <expr> with an expression that filters on the deduplicated fields.
• Pipelines that use the every modifier with the export operator no longer terminate after the first run.
• Setting the tenzir.endpoint option to false now causes the node not to listen for node-to-node connections. Previously, the port was always exposed for other nodes or tenzir processes to connect.