diff --git a/development/developer-guide.md b/development/developer-guide.md index cf001047f..cc459f9f6 100644 --- a/development/developer-guide.md +++ b/development/developer-guide.md @@ -1,42 +1,43 @@ # Developer guide for beginners on contributing to Fluent Bit -Assuming you have some basic knowledge of C, this guide should help you understand how to make code changes to Fluent Bit. +If you have some knowledge of C, this guide should help you understand how to make code changes to Fluent Bit. ## Libraries -Most external libraries are embedded in the project in the [/lib](https://github.com/fluent/fluent-bit/tree/master/lib) folder. To keep its footprint low and make cross-platform builds simple, Fluent Bit attempts keep its dependency graph small. +Most external libraries are embedded in the project in the [/lib](https://github.com/fluent/fluent-bit/tree/master/lib) folder. To keep its footprint low and maximize compatibility in cross-platform builds, Fluent Bit attempts to keep its dependency graph small. -The external library you are mostly likely to interact with is [msgpack](https://github.com/msgpack/msgpack-c). +The external library that you're mostly likely to interact with is [MessagePack](https://github.com/msgpack/msgpack-c). -For cryptographic support, Fluent Bit uses the system installed version of OpenSSL. -Please make sure to install openssl libraries and headers before building Fluent Bit. +For cryptographic support, Fluent Bit uses the system installed version of OpenSSL. You must install OpenSSL libraries and headers before Building Fluent Bit. -### Memory Management +### Memory management -When you write Fluent Bit code, you will use Fluent Bit's versions of the standard C functions for working with memory: +When you write Fluent Bit code, you'll use the Fluent Bit versions of the standard C functions for working with memory: -* [`flb_malloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `malloc`, allocates memory. -* [`flb_calloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `calloc`, allocates memory and initializes it to zero. -* [`flb_realloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `realloc`. -* [`flb_free()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `free`, releases allocated memory. +* [`flb_malloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `malloc`, allocates memory. +* [`flb_calloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `calloc`, allocates memory and initializes it to zero. +* [`flb_realloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `realloc`. +* [`flb_free()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `free`, releases allocated memory. -Note that many types have a specialized create and destroy function. For example, [`flb_sds_create()` and `flb_sds_destroy()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h) \(more about this in the next section\). +{% hint style="info" %} +Many types have specialized create and destroy functions, like [`flb_sds_create()` and `flb_sds_destroy()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h). +{% endhint %} ### Strings -Fluent Bit has a stripped down version of the popular [SDS](https://github.com/antirez/sds) string library. See [flb\_sds.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h) for the API. +Fluent Bit has a stripped down version of the popular [SDS](https://github.com/antirez/sds) string library. See [`flb_sds.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h) for the API. -In general, you should use SDS strings in any string processing code. SDS strings are fully compatible with any C function that accepts a null-terminated sequence of characters; to understand how they work, see the [explanation on Github](https://github.com/antirez/sds#how-sds-strings-work). +In general, you should use SDS strings in any string processing code. SDS strings are fully compatible with any C function that accepts a null-terminated sequence of characters. To understand how they work, see the [explanation on Github](https://github.com/antirez/sds#how-sds-strings-work). -### HTTP Client +### HTTP client Fluent Bit has its own network connection library. The key types and functions are defined in the following header files: -* [flb\_upstream.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_upstream.h) -* [flb\_http\_client.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_http_client.h) -* [flb\_io.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_io.h) +* [`flb_upstream.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_upstream.h) +* [`flb_http_client.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_http_client.h) +* [`flb_io.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_io.h) -The following code demonstrates making an HTTP request in Fluent Bit: +The following code demonstrates an HTTP request in Fluent Bit: ```c #include @@ -113,13 +114,13 @@ static flb_sds_t make_request(struct flb_config *config) } ``` -An `flb_upstream` structure represents a host/endpoint that you want to call. Normally, you'd store this structure somewhere so that it can be re-used. An `flb_upstream_conn` represents a connection to that host for a single HTTP request. The connection structure should not be used for more than one request. +The `flb_upstream` structure represents the host/endpoint that you want to call. Normally, you'd store this structure somewhere so that it can be reused. An `flb_upstream_conn` represents a connection to that host for a single HTTP request. This connection structure shouldn't be used for more than one request. -### Linked Lists +### Linked lists -Fluent Bit contains a library for constructing linked lists- [cfl\_list](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h). The type stores data as a circular linked list. +Fluent Bit contains a library for constructing linked lists: [`cfl_list`](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h). The type stores data as a circular linked list. -The [`cfl_list.h`](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h) header file contains several macros and functions for use with the lists. The example below shows how to create a list, iterate through it, and delete an element. +The [`cfl_list.h`](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h) header file contains several macros and functions for use with the lists. The following example shows how to create a list, iterate through it, and delete an element. ```c #include @@ -174,11 +175,11 @@ static int example() } ``` -### Message Pack +### MessagePack -Fluent Bit uses [msgpack](https://msgpack.org/index.html) to internally store data. If you write code for Fluent Bit, it is almost certain that you will interact with msgpack. +Fluent Bit uses [MessagePack](https://msgpack.org/index.html) to internally store data. If you write code for Fluent Bit, it's almost certain that you will interact with MessagePack. -Fluent Bit embeds the [msgpack-c](https://github.com/msgpack/msgpack-c) library. The example below shows manipulating message pack to add a new key-value pair to a record. In Fluent Bit, the [filter\_record\_modifier](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) plugin adds or deletes keys from records. See its code for more. +Fluent Bit embeds the [`msgpack-c`](https://github.com/msgpack/msgpack-c) library. The following example shows how to manipulate message pack to add a new key/value pair to a record. In Fluent Bit, the [`filter_record_modifier`](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) plugin adds or deletes keys from records. See its code for more. ```c #define A_NEW_KEY "key" @@ -262,19 +263,19 @@ static int cb_filter(const void *data, size_t bytes, return FLB_FILTER_MODIFIED; ``` -Please also check out the message pack examples on the [msgpack-c GitHub repo](https://github.com/msgpack/msgpack-c). +For more info, see the MessagePack examples in the [msgpack-c GitHub repository](https://github.com/msgpack/msgpack-c). ## Plugin API -Each plugin is a shared object which is [loaded into Fluent Bit](https://github.com/fluent/fluent-bit/blob/1.3/src/flb_plugin.c#L70) using dlopen and dlsym. +Each plugin is a shared object which is [loaded into Fluent Bit](https://github.com/fluent/fluent-bit/blob/1.3/src/flb_plugin.c#L70) using `dlopen`and `dlsym`. ### Input -The input plugin structure is defined in [flb\_input.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_input.h#L62). There are a number of functions which a plugin can implement, most only implement `cb_init`, `cb_collect`, and `cb_exit`. +The input plugin structure is defined in [`flb_input.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_input.h#L62). There are a number of functions which a plugin can implement, but most only implement `cb_init`, `cb_collect`, and `cb_exit`. -The [`"dummy"` input plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/in_dummy) is very simple and is an excellent example to review to understand more. +The [`"dummy"` input plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/in_dummy) is very basic and is an excellent example to review to understand more. -Note that input plugins can use threaded mode if the flag `FLB_INPUT_THREADED` is provided. +Input plugins can use threaded mode if the flag `FLB_INPUT_THREADED` is provided. To enable threading in your plugin, add the `FLB_INPUT_THREADED` to the set of `flags` when registering: ```c @@ -295,11 +296,11 @@ struct flb_input_plugin in_your_example_plugin = { ### Filter -The structure for filter plugins is defined in [flb\_filter.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_filter.h#L44). Each plugin must implement `cb_init`, `cb_filter`, and `cb_exit`. +The structure for filter plugins is defined in [`flb_filter.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_filter.h#L44). Each plugin must implement `cb_init`, `cb_filter`, and `cb_exit`. -The [filter\_record\_modifier](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) is a good example of a filter plugin. +The [`filter_record_modifier`](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) is a good example of a filter plugin. -Note that filter plugins can not asynchronously make HTTP requests. If your plugin needs to make a request, add the following code when you initialize your `flb_upstream`: +Filter plugins can not asynchronously make HTTP requests. If your plugin needs to make a request, add the following code when you initialize your `flb_upstream`: ```c /* Remove async flag from upstream */ @@ -308,17 +309,15 @@ upstream->flags &= ~(FLB_IO_ASYNC); ### Output -Output plugins are defined in [flb\_output.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_output.h#L57). Each plugin must implement `cb_init`, `cb_flush`, and `cb_exit`. +Output plugins are defined in [`flb_output.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_output.h#L57). Each plugin must implement `cb_init`, `cb_flush`, and `cb_exit`. -The [stdout plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/out_stdout) is very simple; review its code to understand how output plugins work. +The [stdout plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/out_stdout) is very basic; review its code to understand how output plugins work. -## Development Environment +## Development environment -Fluent Bit provides a standalone environment for development. -Developers who use different OS or distributions can develop on a simple, common stack. -The development environment provides the required libraries and tools for you. +Fluent Bit provides a standalone environment for development. Developers who use different operating systems or distributions can develop on a basic, common stack. The development environment provides the required libraries and tools for you. -Development environments provided for +Development environments are provided for: - [Devcontainer](https://github.com/fluent/fluent-bit/blob/master/DEVELOPER_GUIDE.md#devcontainer) - [Vagrant](https://github.com/fluent/fluent-bit/blob/master/DEVELOPER_GUIDE.md#vagrant). @@ -332,9 +331,11 @@ cmake -DFLB_DEV=On ../ make ``` -Note that Fluent Bit uses Cmake 3 and on some systems you may need to invoke it as `cmake3`. +{% hint style="info" %} +Fluent Bit uses CMake 3. On some systems you might need to invoke it as `cmake3`. +{% endhint %} -To enable the unit tests run: +To enable the unit tests, run the following command: ```text cmake -DFLB_DEV=On -DFLB_TESTS_RUNTIME=On -DFLB_TESTS_INTERNAL=On ../ @@ -343,7 +344,7 @@ make Internal tests are for the internal libraries of Fluent Bit. Runtime tests are for the plugins. -You can run the unit tests with `make test`, however, this is inconvenient in practice. Each test file will create an executable in the `build/bin` directory which you can run directly. For example, if you want to run the SDS tests, you can invoke them as follows: +You can run the unit tests with `make test`. However, this is inconvenient in practice. Each test file will create an executable in the `build/bin` directory, which you can run directly. For example, if you want to run the SDS tests, you can invoke them as follows: ```text $ ./bin/flb-it-sds @@ -351,8 +352,3 @@ Test sds_usage... [ OK ] Test sds_printf... [ OK ] SUCCESS: All unit tests have passed. ``` - -## Need more help? - -The best way to learn how Fluent Bit code works is to read it. If you need help understanding the code, reach out to the community, or open a PR with changes that are a work in progress. - diff --git a/vale-styles/FluentBit/Acronyms.yml b/vale-styles/FluentBit/Acronyms.yml index 14832fb66..a01805615 100644 --- a/vale-styles/FluentBit/Acronyms.yml +++ b/vale-styles/FluentBit/Acronyms.yml @@ -71,6 +71,7 @@ exceptions: - SCM - SCSS - SDK + - SDS - SIEM - SLA - SQL