CHANGE_NOTES

Copyright (c) 2013-2024, Nokia Solutions and Networks
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

  * Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
  * Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.
  * Neither the name of the copyright holder nor the names of its
    contributors may be used to endorse or promote products derived
    from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

================================================================================
CHANGE NOTES - Event Machine (EM) on ODP (em-odp)
================================================================================

The EM version numbering scheme reflects the used EM API version:
maj.min.impl(-fix)
    maj.min = EM API version supported
              maj = major API version number, reserved for big changes
             .min = minor API version, incremented when the API changes
      .impl = Implementation number supporting EM API version maj.min
              Updated for implementations with no API changes, starts at .0
       -fix = Fix number, added for bugfixes on a certain implementation
              version maj.min.impl (-0 is not added, first fix is -1)
Examples:
 1) EM version 2.7.0
    maj.min = 2.7: supports EM API v2.7
      .impl =  .0: first implementation supporting API 2.7 (starts at .0)
       -fix = n/a, no fix
 2) EM version 2.7.1-3
    maj.min = 2.7: supports EM API v2.7
      .impl =  .1: second implementation supporting API 2.7 (starts at .0)
       -fix =  -3: third bugfix for 2.7.1

- See em-odp/README for usage and compilation instructions.
- See em-odp/include/event_machine/README_API for API changes

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.8.0-1
--------------------------------------------------------------------------------
- Fixes:
 - event: fix em_event_clone_part()
   In em_event_clone_part(), when cloning an event allocated not from an
   EM-pool, but directly from an odp-pool, the event header and the EM user area
   was not copied correctly.
   When using odp_packet_copy_part() to clone the event, the ODP user area is
   not copied resulting in an invalid EM event header and EM user area.
   This is now fixed.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.8.0
--------------------------------------------------------------------------------
- Support for EM API v3.8 (em-odp/include/),
  see API additions and changes in em-odp/include/event_machine/README_API.
  Summary:
  * Timer: change the em_tmo_delete() API function and
           enhance the timeout cancel and delete logic.
     NOTE: Both an API and functional change, requires changes to
           user code logic!
  * Init: check that em_conf_init() has been called before em_init()

- Fixes:
 - timer: fix proper handling of the return values of odp_timer_cancel()
   The odp_timer_cancel() return values changed in ODP v1.41.1.
   The return value separates failure from expired timeout.

 - event, fix: update check for setting vector evtype
   In em_event_set_type(), when setting a new event type for a vector event:
   The new major event type is only used if EM_CHECK_LEVEL >= 1, thus move the
   variable declaration inside the if-statment.
   Avoids potential compilation warnings with EM_CHECK_LEVEL = 0.

- Programs:
 - New performance test program added.
  - programs/performance/loop_united.c
    Combines the existing performance test programs pairs, loop, loop_multircv,
    loop_refs and loop_vectors into a single program that can run all the
    individual tests via command line options.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.7.0
--------------------------------------------------------------------------------
- Support for EM API v3.7 (em-odp/include/),
  see API additions and changes in em-odp/include/event_machine/README_API.
  Summary:
  * Timer: em_timer_get_all() update
  * Helper APIs: em_version_print(), em_info_print()

- Startup printouts changed
  The EM startup printouts / logging has been changed to reduce startup clutter
  and give more control to the user about what is being printed and when.
  EM will print the version information at startup (wuth the same content as
  produced by the new helper API em_version_print()) but further printouts with
  info about the running instance has been moved into another new helper API
  em_info_print(), which the user can now call on demand
  (as is done by the EM example applications, see programs/common/cm_setup.c).

- configure, build: disable strict-aliasing (use -fno-strict-aliasing)

  Don't allow the compiler to assume the strictest aliasing rules.
  Unless turned off, the -fstrict-aliasing option is enabled at
  optimization levels -O2, -O3, -Os.

  EM contains some type casting that might violate C strict aliasing rules.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.6.0
--------------------------------------------------------------------------------
- Support for EM API v3.6 (em-odp/include/),
  see API additions and changes in em-odp/include/event_machine/README_API.
  Summary:
  * Packet event APIs: new API module for manipulation of packet events
  * Timer API: moved from add-ons to the base EM API
  * Event: em_event_pointer_and_size()
  * Deprecation macros/defines

- Add support for using ODP API 1.45
  Update EM to be able to run against ODP API v1.45.

- Deprecated EM API enable/disable configuration option
 - Add a new configure-script option that enables/disables decprecated EM APIs.
   Old EM APIs can be deprecated in new EM releases through the use of
   deprecation macros and defines found in event_machine_deprecated.h(.in)
   (additionally see include/event_machine/README_API for API v3.6 description
    of deprecation macros).
   The deprecation of an API will cause any use of it to fail the compilation.
   The user may, however, decide to enable the deprecated APIs (by using
   --enable-deprecated) during a transition period until the deprecated API
   usage has been replaced.

   configure.ac:
   -------------
   New configure option:
   --enable-deprecated  enable deprecated EM API definitions [default=disabled]

- Event State Verification (ESV) changes
 - Added ESV compile-time vs. runtime configuration checks.
   Verifies that ESV was activated also during
   compile time (with configure option: '--enable_esv')
   when enabling it during runtime (via the EM config file: esv.enable = true).
   Warn the user otherwise.
   The user might not notice that the run time config file option has no effect
   if ESV has been disabled during compilation.
   Example:
        build> ../configure --disable-esv …  (set the define EM_ESV_ENABLE to 0)
          vs.
        <em-odp>/config/em-odp.conf:
            esv: {
                    # Runtime ESV enable/disable option.
                    # Allows the user to disable ESV without recompilation when
                    # 'configure --enable-esv'.
                    enable = true
                    …
            }
        will show the following warning at startup:
        …
        EM ESV config: (EM_ESV_ENABLE=0)
          ESV disabled
          esv.enable: true(1)
          WARNING: ESV disabled (build-time) - config file option IGNORED!
        …
   Note that the warning is only printed, no other action is taken.

- EM Termination
 - Move functionality from em_term_core() to em_term().
   Now em_term_core() only flushes events stashed locally on the core
   and em_term() flushes the rest of the events at the end.
   Additionally, em_term() joins all available queue groups to be able to
   flush all events from the scheduler before terminating.

- Programs
 - New performance test programs added
   - programs/performance/queues_output.c
     Event Machine performance test for output queues.
   - programs/performance/loop_vectors.c
     Event Machine performance test for vector events

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.5.0
--------------------------------------------------------------------------------
- Support for EM API v3.5 (em-odp/include/),
  see API additions and changes in em-odp/include/event_machine/README_API.
  * Queue: replace 'EM_MAX_QUEUES' with config file option 'queue.max_num' and
           em_queue_get_max_num()
  * Queue conversions: EM-queue <-> {device-id, queue-id}
  * Event, pool: em_event_get_pool_subpool()
  * EM-handle to unsigned integer conversion functions.
  * EM device-ID: get the current device-id em_device_id()

- EM config file options - runtime config file config/em-odp.conf:
  New option to limit the maximum number of EM queues that can be created
  queue: {
        # The maximum number of EM queues that can be created.
        #
        # EM will allocate memory at startup to support 'max_num' queues.
        # The number must be large enough to cover the EM internal queues as
        # well as all the static and dynamic queues created by the application.
        # The smallest acceptable 'max_num' value depends on the values of
        # EM_MAX_CORES, EM_QUEUE_STATIC_MIN and EM_QUEUE_STATIC_MAX.
        # The smallest acceptable value allows creating one dynamic EM queue.
        # The largest 'max_num' depends on ODP capability.
        # A larger value will consume more memory.
        max_num = 1024
        ...
  }

  The 'queue.max_num' option replaces the removed EM API define 'EM_MAX_QUEUES'

  Config file version number bumped up to "0.0.20"

- Add support for using ODP API 1.43
  Update EM to be able to run against ODP API v1.43.
  Maintain compatibility with v1.42.

- Fixes:
 - Event: fix em_free_multi() when freeing many timer events
   In remove_active_timers(): if the loop removes the j:th event
   from the array, and the j+1:th event also happens to be an ODP_EVENT_TIMEOUT,
   then the function missed removing that following timeout.
   This is now fixed and the function is refactored.
   Both em_free() and em_free_multi() are improved to check for these tmos only
   when periodic ring timers have actually been created - if no ring timers ever
   created then no checks need to be done.

 - Dispatch: error logging should not access NULL q_elem
   The error logging in local-queue dispatch should not access the
   q_elem if it is NULL.

 - configure: prevent '-Wstringop-overflow' warnings by gcc when LTO is enabled
   GCC can generate false errors about object sizes, especially when LTO is used.
   As a workaround, make -Wstringop-overflow not generate errors as part of
   -Werror when GCC version 10 or later is used.
   Add -Wno-error=stringop-overflow both to CFLAGS and CXXFLAGS.

- Programs:
 - Programs, common: add cmd line options to dispatch with options

  Add new test program command line options that enables the use
  of em_dispatch_duration() instead of em_dispatch(), see b) below.

  EM dispatch options (optional):
    a) using em_dispatch() - default: uses em_dispatch() unless options
                                      in b) are given.
      -r, --dispatch-rounds <arg> Number of dispatch rounds (default: 0=forever)
    b) using em_dispatch_duration() - combinations of these options possible.
      --duration-rounds <arg> Dispatch for the given number of rounds.
      --duration-ns <arg> Dispatch for the given time in nanoseconds.
      --duration-events <arg> Dispatch until the given number of events
                                        have been handled.
      --duration-noevents-rounds <arg> Dispatch until no events have been
                                       received for the given number of rounds.
      --duration-noevents-ns <arg> Dispatch until no events have been received
                                   for the given time in nanoseconds.
      --burst-size <arg> The max number of events the dispatcher will request
                         in one burst from the scheduler.
                         (default: EM_SCHED_MULTI_MAX_BURST=%d)
      --wait-ns <arg> Scheduler wait-for-events timeout in nanoseconds.
                      The scheduler will wait for events, if no immediately
                      available, for 'wait_ns' per dispatch round.
                      (default: 0, do not wait)
      --sched-pause Pause the scheduler on the calling core when exiting the
                    EM dispatch function. If enabled, will also resume the
                    scheduling when entering dispatch (default: don't pause).

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.4.0
--------------------------------------------------------------------------------
- Support for EM API v3.4 (em-odp/include/),
  see API additions and changes in em-odp/include/event_machine/README_API.
  * Dispatch APIs: em_dispatch_duration()/_ns()/_rounds()/_events() and
                   em_dispatch_opt_init()
    Dispatch APIs with duration and option arguments for more control of the
    dispatch loop.
  * Pool statistics: new option em_pool_stats_opt_t::core_cache_available

- Dispatch configuration options
  em-odp.conf (runtime config file):
  ----------------------------------
  Add a new EM runtime config file option:
    'dispatch.sched_pause = true / false'
  that controls whether the em_dispatch() API-function pauses the EM core
  from scheduling when it returns.

  Note: the option only affects the em_dispatch() API,
        not em_dispatch_duration()/_ns()/_rounds()/_events().

  dispatch: {
        ...
        # Pause scheduling for an EM-core returning from em_dispatch().
        # The core is paused from participation in global scheduling and EM
        # empties all locally pre-scheduled events, as well as dispatches them,
        # before returning from the em_dispatch() function.
        # The scheduler should be paused on a core that is leaving the
        # dispatch / scheduling loop for a longer time to avoid locking atomic
        # or ordered contexts (thus blocking other cores from handling events
        # from those same contexts / queues).
        # Only affects the em_dispatch() function.
        sched_pause = false
  }

  Bump the em-odp.conf file version to 0.0.19

- Info: detect additional ARM v8 and v9 isa's.
  Update the SW ISA detection for EM with further ARM v8 and v9 ISAs:
  ARMv8.8-A, ARMv8.9-A, and ARMv9.3-A.
  Based on ODP's sysinfo.
  Detection of these ISAs require ODP API v1.42.1 or later.

- Fixes:
 - Timer: fix last ring timer delete bug
   Events for ring timeout indication is provided by timer from a common pool.
   This pool was deleted when last ring timer was deleted to save memory,
   but application may still have event(s) from that pool, create new ring timer
   and then re-use those events. That lead to corrupted event. Fixed by not
   deleting the shared pool until application termination.

 - Scheduler: em_ordered_processing_end() corrected.
   Allow ending the ordered context only if the currect scheduling context type
   is 'ordered', not if the current queue type is 'ordered'.

 - Dispatch, ESV: set core's (internal) current queue info before event init.
   Set the core's current queue before event-init in dispatch, since the init
   might notice early ESV errors and the error printout is then able to print
   the queue and EO information correctly.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.3.0
--------------------------------------------------------------------------------
- Support for EM API v3.3 (em-odp/include), see API additions and changes in
  em-odp/include/event_machine/README_API.
  * Timer ring event type changed to EM_EVENT_TYPE_TIMER_IND
    (from EM_EVENT_TYPE_TIMER) to ensure that applications that prior to
    v3.2.0 were using EM_EVENT_TYPE_TIMER for their "normal" timer events can
    continue to do so.
  * Timer APIs: em_tmo_get_timer() - Get timer from timeout
  * Event APIs: em_event_clone_part() - Partially clone an event
  * Pool APIs: Selected Pool Statistics - em_pool_stats_selected(),
                                          em_pool_subpool_stats_selected() etc.

- Event State Verification (ESV) changes
  - Add basic ESV state checks for timer-ring events. Since the same event might
    be processed on multiple cores, only a limited set of ESV checks can be done.
    For now only verify timer-ring event alloc and free.

- Dispatch configuration options
  - Add the ability to configure the wait-for-events timeout passed
    to the ODP scheduler during EM dispatch.
  - Config file (em-odp.conf) version number bumped up to "0.0.18"

    configure.ac:
    -------------
    Enable the EM dispatcher/scheduler to start waiting for events if none are
    immediately available. Waiting for events, rather than busy-waiting, might
    save power.
    New configure option:
    --enable-sched-wait   EM calls odp_schedule_multi(..., sched_wait_ns, ...),
                          where 'sched_wait_ns' is taken from
                          'dispatch.sched_wait_ns = value' in the
                          EM config file (config/em-odp.conf).
                          Set 'EM_SCHED_WAIT_ENABLE=1'
    --disable--sched-wait EM calls odp_schedule_multi_no_wait().
                          Set 'EM_SCHED_WAIT_ENABLE=0'
     no option given Use '#define EM_SCHED_WAIT_ENABLE 0|1' from source code

     em-odp.conf (runtime config file):
     ----------------------------------
     Runtime schedule-wait option in nanoseconds.
     Allows the user to configure the time (ns) the scheduler should wait
     for events before returning (when no events are immediately available).
     Only usable if the define EM_SCHED_WAIT_ENABLE has been set to '1'
     either via
     1) 'configure --enable-sched-wait' or
     2) #define EM_SCHED_WAIT_ENABLE 0 ==> 1
        (include/event_machine/platform/event_machine_config.h)
     Note that using option 1), i.e. via 'configure', overrides the value
     in event_machine_config.h.
     See configure.ac, config/em-odp.conf and event_machine_config.h
     sched_wait_ns = 0: no waiting for events, scheduler returns
                         immediately even if no events available
                         (converted to 'ODP_SCHED_NO_WAIT')
     sched_wait_ns = -1: waits indefinitly for events
                         (converted to 'ODP_SCHED_WAIT')
     sched_wait_ns > 0: waits 'sched_wait_ns' for events
                         (converted to 'odp_schedule_wait_time(sched_wait_ns)')
     Note that the 'sched_wait_ns' may be rounded up as specified by the
     ODP spec, configuration and implementation.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.2.0
--------------------------------------------------------------------------------
- Support for EM API v3.2 (em-odp/include), see API additions and changes in
  em-odp/include/event_machine/README_API.
  * Timer ring
  * Timeout type
  * Error codes

- EM config file options - config/em-odp.conf:
  - Config file version number bumped up to "0.0.17",
  - New 'timer:{}' options in the config file.

- Event State Verification (ESV) changes
  - Combine ESV operations for events/packets from e.g. pktio for potential
    performance gains.
    Combine 'mark allocated', 'mark sent' and 'mark em2usr for dispatch' into a
    single ESV operation (with a single atomic counter update).

- Performance optimizations
  - Reduce the sizes of commonly used (in fast path) data structures for
    potential caching gains (queue, pool, atomic group).
  - Adjust fast-path API error checking according to the used EM_CHECK_LEVEL.
  - Dispatch: check output queue buffering only with ordered schedule context.

- Misc.
  - daemon-eo: remove the unnecessary daemon-eo
  - event: replace odp_packet_user_ptr/set() with odp_packet_user_flag/set() to
    detect whether the event header has been initialized or not.

- Performance test applications added
  - em-odp/programs/performance/bench_event.c
     - Measure event related API-function performance (cycles), single-core
  - em-odp/programs/performance/pool_perf.c
    - Performance test tool for memory pool performance.
      Measures time consumed during calls to allocate and free buffers or
      packets. Command line arguments can be used to try different aspects.
  - em-odp/programs/performance/timer_test_ring.c
    - Timer ring test for alternative periodic timeouts.
      The purpose of this tool is to manually test periodic ring timer
      functionality. At least two EM timers are created. One standard timer for
      a heartbeat driving test states. The second timer (or many) is periodic
      ring for testing.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.1.0
--------------------------------------------------------------------------------
- Support for EM API v3.1 (em-odp/include), see API additions and changes in
  em-odp/include/event_machine/README_API.
  * Pool Statistics
  * Timer updates
  * Event Vectors

- EM config file options - config/em-odp.conf:
  - Config file version number bumped up to "0.0.16",
  - New 'pool.statistics:{}' options in the config file.

- Event State Verification (ESV) changes
  - Fix: evgen wrap handling in esv_usr2em/_multi() during a 'revert'
  - Use larger EVGEN_INIT value to protect from over/underflows
      Evgen values:  0x1000 - 0xEFFF
      Event handles: 0x1000 xxxx xxxx xxxx - 0xefff xxxx xxxx xxxx,
                     where x…x = ptr value)
  - Fix: Reset ESV-counters on the next alloc after using a reference.
    Event references do not use the ESV evgen counter but it is still updated by
    ESV operations. Thus it needs to be reset for the next alloc of the same
    event or it will contain invalid values.

- Event changes
  Highlighting EM internal changes related to the event header since it may
  affect users extending or accessing EM internals.
  -  Placement of the internal event header has been changed for odp_buffers.
     Now the event header always resides in the ODP buf/pkt/vector user area
     followed by the EM event user area (if any).
      Prev:  event_hdr_t *ev_hdr = odp_buffer_addr(odp_buf);
      Curr:  event_hdr_t *ev_hdr = odp_buffer_user_area(odp_buf);
      If changes have been made to EM internals in your project then this needs
      to be updated!
    - The internal event header size is reduced to fit within one cache line for
      potential caching and performance gains.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v3.0.0
--------------------------------------------------------------------------------
- Support for EM API v3.0 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.

- Event References
  See em-odp/include/event_machine/README_API

- Event Vectors
  See em-odp/include/event_machine/README_API

- EM debug timestamps
  See em-odp/include/event_machine/README_API
  - Enabled with new configure-script option '--enable-debug-timestamps':
    (see configure.ac)
    Override EM-define value 'EM_DEBUG_TIMESTAMP_ENABLE' to 0...2
    --enable-debug-timestamps=0...2   Set 'EM_DEBUG_TIMESTAMP_ENABLE' to the
                                      given value: 1=low overhead, 2=strict time
    --enable-debug-timestamps<=yes(=1) Set 'EM_DEBUG_TIMESTAMP_ENABLE' to 1
    --enable-debug-timestamps          Set 'EM_DEBUG_TIMESTAMP_ENABLE' to 1
    --disable-debug-timestamps or
      no option given                  Use 'EM_DEBUG_TIMESTAMP_ENABLE' value
                                       from source code (0...2, default 0)

- EM config file options - config/em-odp.conf:
 - Config file version number bumped up to "0.0.15", see new options below.

-  EM config file option: event_chaining.order_keep = false/true
   Option description:
     Events sent with em_send...() to an event chaining queue will be
      passed to the user provided 'event_send_device/_multi()' functions.
      EM can ensure that calls to these functions are:
      1) made in order (order_keep = true)
         or
      2) make no extra effort to keep the order (order_keep = false),
         e.g. if the user (or packet output) handles ordering or does not
         care about it.
   Note: Only meaningful when used together with (parallel-)ordered queues.
         Order is implicitly guaranteed when sent during an atomic context and
         never guaranteed when sent during a parallel context.
   Only set to 'true' if queues of type EM_QUEUE_TYPE_PARALLEL_ORDERED
   are used with event chaining, no benefit otherwise. Using 'false' lets EM
   save on some resources.

   Note! Currently only event_chaining.order_keep = false is supported.
         Trying 'true' gives the following printout:
         "Config option event_chaining.order_keep: true(1) currently not supported"

- Env additions
  - Return difference of cpu cycles (cycles2 - cycles1):
    uint64_t env_cycles_diff(uint64_t cycles2, uint64_t cycles1)

- Event State Verification (ESV) changes
  The introduction of Event References has caused ESV to change a bit.
  The ESV counters are always modified together atomically as one uint64_t,
  as before. One of the ESV-counters, 'free_cnt', has been replaced by 'ref_cnt'
  that stands for 'reference count', i.e. how many references exist for a
  certain event. A newly allocated event with no further references has
  'ref_cnt=1'. When one additional reference is created, the reference count
  increases to 'ref_cnt=2' etc. Freeing an event, or a reference, decreases the
  reference count. A freed event (back into the event pool) should have
  'ref_cnt=0'.
  The 'send_cnt' works as before, sending the event (or e.g. giving it to the
  timer) increases the 'send_cnt' while dispatching or dequeueing decreases the
  'send_cnt'. Also the 'evgen' count works as before but has been restricted
  to "normal" events only, i.e. 'evgen' will not be taken into account for
  references.

  1) "Normal" ESV Error format:    (double-free error)
  -----------------------------
  EM ERROR:0x80000010  ESCOPE:0xFF000603
  core:00 ecount:0(0) em_event_state.c:350 esv_error()
  ESV: Event:0x27fffd9c7f000 state error -- counts:	send:0 ref:-1 evgen:2(3)
  Help: OK: 'send <= ref, both >=0' AND 'hdl evgen == evgen'. Err otherwise
    prev-state:em_free() core:00: EO:0xA-"myEO1" Q:0xB-"myQ1" u32[0]:(n/a)
  => err-state:em_free() core:01: EO:0xC-"myEO2" Q:0xD-"myQ2" u32[0]:0x00000000
  event:0x00027fffd9c7f000: ptr:0x7fffd9c7f000

  2) Reference ESV Error format:    ("too many frees" error, here two frees)
  ------------------------------
  EM ERROR:0x80000010  ESCOPE:0xFF000610
  core:00 ecount:0(0) em_event_state.c:342 esv_error()
  ESV: RefEvent:0x27fffd9472000 state error -- counts:	send:0 ref:-1 (evgen:4 ignored for refs)
  Help: OK: 'send <= ref, both >=0'. Err otherwise
    prev-state:n/a (not valid for event references)
  => err-state:em_free() core:00: EO:0xC-"myEO2" Q:0xD-"myQ2" u32[0]:0x00000000
  event:0x00027fffd9472000: ptr:0x7fffd9472000

  3) Unmark ESV Error format:
  ---------------------------
  EM ERROR:0x80000010  ESCOPE:0xFF000611
  core:00 ecount:0(0) em_event_state.c:363 esv_error()
  ESV: Event:0x27fffd9c7f100 state error - Invalid 'unmark'-API use
    prev-state:em_free()              core:00: EO:0xA-"myEO1" Q:0xB-"myQ1" u32[0]:(n/a)
  => err-state:em_event_unmark_free() core:00: EO:0xC-"myEO2" Q:0xD-"myQ2" u32[0]:0x00000000

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v2.10.0
--------------------------------------------------------------------------------
- Support for EM API v2.10 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * EM Idle Hooks
  * Startup pools (changes mostly related to the EM config file)

- EM Idle Hooks
   (see include/event_machine/platform/event_machine_hooks.h for details)
   The user can provide idle hooks to EM that will be run on each EM core when
   there are no events to process ('to_idle' and 'while_idle' hooks).
   Further, the 'to_active' hook will be run, if provided, when the core again
   starts receiving events.
   User idle hook functions can be provided via em_conf_t::idle_hooks{...}
   when calling em_init() or via hook register functions. EM will call the
   given hooks in the dispatcher depending on whether there are events to be
   processed by the core.
   Note: the 'EM_IDLE_HOOKS_ENABLE' value can be overridden by a command-line
         option to the 'configure' script, e.g.:
         $build> ../configure ... --enable-idle-hooks
         The overridden value will be made available to the application via a
         pkgconfig set define.
   Idle hooks available:
   - em_idle_hook_to_idle_t:
     The 'to_idle' hook will be called by the EM dispatcher when a core is
     entering the IDLE state i.e. when the core doesn't get any new events
     to process.
   - em_idle_hook_to_active_t:
     The 'to_active' hook will be called by the EM dispatcher when a core is
     entering the ACTIVE state i.e. when the core gets events after being idle.
   - em_idle_hook_while_idle_t:
     The 'while_idle' hook will be called by the EM dispatcher when a core is
     already in the IDLE state and stays in it i.e. the core doesn't get any
     events.
   Register-functions:
     em_status_t em_hooks_register_to_idle(em_idle_hook_to_idle_t func);
     em_status_t em_hooks_unregister_to_idle(em_idle_hook_to_idle_t func);
     em_status_t em_hooks_register_to_active(em_idle_hook_to_active_t func);
     em_status_t em_hooks_unregister_to_active(em_idle_hook_to_active_t func);
     em_status_t em_hooks_register_while_idle(em_idle_hook_while_idle_t func);
     em_status_t em_hooks_unregister_while_idle(em_idle_hook_while_idle_t func);
   Alternatively provide the idle-hook functions as arguments to em_init() at
   start-up: em_init(em_conf_t::idle_hooks.to_idle_hook
                                          .to_active_hook
                                          .while_idle_hook);

- Startup pools
  (see config/em-odp.conf for more documentation and details)
  - EM config file option: 'startup_pools'
  Configure EM event-pools via the EM config file. These pools are created
  during EM startup and destroyed when EM terminates (if still available).
  When startup pools are used, EM will create them according to the
  configuration given in the EM config file during em_init().
  Startup pools are configured via the EM config file option: 'startup_pools'
  The user should override the EM default config file (config/em-odp.conf) with
  a separate runtime configuration file containing the desired application pool
  configurations and provide it to EM via the 'EM_CONFIG_FILE' env variable:
        $> EM_CONFIG_FILE=<path-to>/my-em.conf ./my-em-app [args]
  If no startup pools are given, only the default pool needs to be created. In
  this case, the default pool configuration is specified in the parameters of
  em_init(), more specifically, in the struct em_conf_t::default_pool_cfg

  Note that if the default pool is configured via the config file, it overrides
  the default pool configuration given to em_init().
  The priority regarding the default pool configuration is as follows:
  +--------------+      +-------------+      +-----------------------------+
  | Runtime.conf |  >   | em-odp.conf |   >  | em_conf_t::default_pool_cfg |
  +--------------+      +-------------+      +-----------------------------+
  The default pool configuration specified in runtime configuration file
  overrides the one given in the default configuration file(em-odp.conf), which
  overrides the one passed as a parameter to em_init().

  Note that the 'startup_pools'-options are pool specific configurations given
  directly to em_pool_create(name, pool, pool_cfg), thus overriding the global
  config file settings such as pool.align_offset, pool.user_area_size and
  pool.pkt_headroom that might be set in the config file.

- Core: support odp-thread counts larger than 256

- ESV: reduce the number of ESV state transitions for output from EM
  Event Chaining output and output via EM output queues are both based on
  calling a user provided function triggered from em_send() - no need to
  toggle ESV state (between usr2em <-> em2usr) many times. Keep the ownership of
  the event with the user until the user provided callback.

- Queue: check that the EM queue context is valid
  EM will verify that the used ODP queue context actually points to an
  EM queue-elem and not to something else.
  Only enabled with EM_CHECK_LEVEL=3 (configure option: --enable-check-level=3)

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v2.9.0
--------------------------------------------------------------------------------
- Support for EM API v2.9 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * Minor API change: documentation clean up and a define value increase.

- EM config file options - config/em-odp.conf:
 - Config file version number bumped up to "0.0.14", see new options below.

- EM pool statistics update
  - EM Pool statistics to rely on ODP pool statistics.
    EM does not have to maintain pool statistics since ODP provides the
    same (or extended) information. The EM API em_pool_info() remains the same,
    it now uses ODP pool statistics under the hood instead.

- Dispatch, init: enforce ctrl queue polling on first dispatch.
  - The first call to em_dispatch() on each core will trigger a poll of the
    internal unscheduled ctrl queues.
  - See "Changed EM internal communication" in the release notes for v2.8.0
    below for more details on EM's unscheduled ctrl queues.

- EM Command Line Interface (EM CLI) update
  - Further EM CLI commands added, use the command "help" to see all available
    commands:
        (<em-odp.conf>: cli.enable = true)
        $> telnet localhost 55555
        EM-ODP> help
        Commands available:
          help                 Show available commands
          quit                 Disconnect
          logout               Disconnect
          exit                 Exit from current mode
          history              Show a list of previously run commands
          enable               Turn on privileged commands
          ...
          <odp-cmds>
          ...
          em_agrp_print        [a|i <ag id>|n <ag name>|h]
          em_eo_print          [a|i <eo id>|n <eo name>|h]
          em_egrp_print
          em_info_print        [a|p|c|h]
          em_pool_print        [a|i <pool id>|n <pool name>|h]
          em_queue_print       [a|c|h]
          em_qgrp_print        [a|i <qgrp id>|n <qgrp name>|h]
          em_core_print
  - See "EM Command Line Interface (EM CLI)" in the release notes for v2.8.0
    below for more details on usage.

- Rate limit EM's input-poll and output-drain callback functions.
  - Config file options to control how often the user provided callback
    functions are called (see config/em-odp.conf for full documentation):
    - Core local interval for calling input poll and output drain functions
      (in dispatch rounds):
      poll_drain_interval = 1
    - Core local interval for calling input poll and output drain functions
      (in nanoseconds):
      poll_drain_interval_ns = 1000000L
    - Default values currently selected so that functionality is unchanged
      compared to earlier releases.

- EM timer updated to use new ODP APIs
  - EM timer updated to internally use odp_timer_start() (instead of the now
    deprecated odp_timer_set_abs())

-  Test files and scripts moved into the em-odp repo (robot-tests/, scripts/)

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v2.8.0
--------------------------------------------------------------------------------
- Support for EM API v2.8 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * Small API change, a new define is added.

- EM config file options - config/em-odp.conf:
 - Config file version number bumped up to "0.0.13", see new options below.

- EM Queue Group start-up improvements:
  - The EM default queue group (EM_QUEUE_GROUP_DEFAULT) is now available at
    start-up after init, before all cores have run local init.
    This allows an application to create queues using the default queue group
    before all other EM cores have initialized.
  - If enabled in the EM config file, the single-core EM queue groups are also
    similarly available at startup.

- Concurrent sync-APIs
  - EM allows concurrent sync-APIs. Multiple concurrent calls to e.g.
    em_eo_start_sync(), em_queue_group_modify_sync() etc. are allowed.
    Previously only one sync-operation at a time was permitted, now there is
    no limit.
    One restriction remains: concerned cores need to be dispatching
    (or processing other sync-API calls) for the operations to complete.
    A core that has left the dispatch loop or is sleeping etc. will not allow
    the sync-API operation to complete.

- Changed EM internal communication:
  The EM internal event communication previously used scheduled queues and
  single-core queue groups. This has now been replaced by unscheduled queues
  that each core polls periodically for internal control-events. The internal
  control operations and events are all triggered by EM APIs, e.g. sync-APIs.
  This change allows EM to support concurrent sync-APIs.
  Adjust the EM control queue polling rate to a suitable level based on the
  needs of the application, see options in the EM config file
  config/em-odp.conf:
  - dispatch.poll_ctrl_interval = 100 # Rate limit EM control queue polling
  - dispatch.poll_ctrl_interval_ns = 1000000L # poll max every 1ms
  See the EM config file for additional documentation and details.
  These options affect the timing of the EM control operations and the
  application may see changes in control operation duration and latency.
  Tune the "dispatch.poll_ctrl_..." options to a preferred level of
  performance vs. latency (performance impact of polling is quite small).

- EM single-core queue groups disabled by default:
  The EM single-core queue groups have been disabled by default since EM does
  not need them for internal communication anymore (see above).
  - EM single core queue group naming (if enabled and available):
    EM_QUEUE_GROUP_CORE_BASE_NAME + "%d": "core0", "core1", etc.
  Disabling saves on available queue groups especially when running with many
  cores (since EM would otherwise at startup create a queue group per core).
  Applications must override the default config if these single-core
  queue groups are needed, see option in the EM config file
  config/em-odp.conf:
  - queue_group.create_core_queue_groups = false(default)|true
  See the EM config file for additional documentation and details.

- EM Command Line Interface (EM CLI)
  The first version of the EM CLI has been integrated but is by disabled by
  default. A change to the EM config file is needed in order to use it.
  The EM CLI allows a developer to connect via telnet to a running EM instance
  in order to extract runtime information about the EM application.
  - Dependency: libcli needs to be availabe to use the EM CLI.
    If libcli is unavailable at compile time then the CLI feature is left out.
    - An autoconf configure script option "--with-libcli-path=[install-dir]" can
      be used to set the path to libcli if not installed in the standard path:
      em-odp/build $> ../configure ... --with-libcli-path=[install-dir] ...
    - EM CLI options (see the EM config file for additional documentation and
      details), config/em-odp.conf:
      - cli.enable = false(default)|true
      - cli.ip_addr = "127.0.0.1" # localhost
      - cli.port = 55555
  Currently, only very basic commands are available - new ones will be added and
  existing ones extended & improved.
  Note: existing commands might still be renamed and reworked.

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v2.7.0
--------------------------------------------------------------------------------
- Support for EM API v2.7 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * New APIs for Version, Event User Area, Event User Area ID *

- EM config file options - config/em-odp.conf:
 - Config file version number bumped up to "0.0.10"
 - Change the default Queue priority mode:
   queue.priority.map_mode = 1
     1: map according to ODP runtime number of priorities
        (linear fit to full range of ODP priorities available)
 - The Event User Area default size can be set via the 'user_area_size' option:
   config/em-odp.conf:
     pool: {
        ...
        # Default event user area size in bytes for all events from all pools.
        #
        # This is a global setting that can be overridden per pool using
        # 'em_pool_cfg_t::user_area{}' with em_pool_create().
        #
        user_area_size = 0
        ...
     }

- Tested against
  odp-linux - https://github.com/OpenDataPlane/odp
    Commit: 795d576fb46db723d0b5e11a20111bab574a5d0a [795d576]
    Date: Tuesday, 21 December 2021 11:42.06
    Commit Date: Monday, 27 December 2021 14:47.34
    linux-gen: sched: update random data table
  odp-dpdk - https://github.com/OpenDataPlane/odp-dpdk
    Commit: c3789c8e6ec34faee0b59080c1ab187cff1dc356 [c3789c8]
    Date: Friday, 17 December 2021 14:00.35
    Merge ODP v1.33.0.0

--------------------------------------------------------------------------------
Event Machine (EM) on ODP v2.6.0
--------------------------------------------------------------------------------
- Support for EM API v2.6 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * New APIs for Timer, Event, Queue, Extension-APIs *

- New configure script option (see em-odp/configure.ac):
  '--enable-debug-print' that enables EM debug printouts via the EM_DBG() macro:
        em-odp/build $> ../configure --prefix=… --enable-debug-print
  Currently if enabled, will print event header layout information at startup
  as well as EM timer runtime debug info.

- EM config file options - config/em-odp.conf:
 - Config file version bumped to "0.0.9" (user config files need to be updated!)
 - Options for setting EM Queue priority-levels:
   a) Option 'queue.priority.map_mode':
     Select the queue priority mapping mode, i.e how to map EM queue priorities
     to ODP scheduled queue priorities.
     0: legacy simple mode, map EM prios to ODP min/default/max prio-levels only
        (0 is the default mode)
     1: map according to the ODP runtime number of priorities
        (linear fit to full range of ODP prio-levels)
     2: custom mapping
        (use 'custom_map' below)
   b) Option 'queue.priority.custom_map':
     Custom priority map (required when 'queue.priority.map_mode = 2')
     This array needs to have EM_QUEUE_PRIO_NUM entries (typically 8).
     The first entry is for the lowest priority (0). The value is ADDED to
     odp_schedule_min_prio() and then passed to ODP, i.e. values are offsets
     from odp_schedule_min_prio(). Values given here must be valid for ODP
     runtime configuration, i.e. value plus odp_schedule_min_prio() must
     not exceed odp_schedule_max_prio().
     Example of a custom priority mapping:
        queue: {
                priority: {
                        map_mode = 2
                        custom_map = [0, 0, 1, 3 ,4 ,6 ,7, 7]
                }
        }

- EM SW ISA printout added at start-up:
    ...
    ===========================================================
    EM Info on target: em-odp
    ===========================================================
    EM API version:         v2.6, 64 bit (EM_CHECK_LEVEL:3, EM_ESV_ENABLE:1)
    EM build info:          v2.6.0 2021-10-15 16:46
    ODP API version:        1.32.0
    ODP impl name:          odp-linux
    ODP impl details:       odp-linux 1.32.0-0 (v1.32.0) 1.32.0.0
    CPU model:              Intel(R) Xeon(R) CPU E5-2697 v3
    CPU arch:               x86     // ARM etc.
    CPU ISA version:        Unknown // ARMv8.2-A  # HW detected, reported by ODP
     SW ISA version (ODP):  x86_64  // ARMv8.1-A  # ODP SW config (compile time)
     SW ISA version (EM):   x86_64  // ARMv8.1-A  #  EM SW config (compile time)

  Arch specific CFLAGS should be passed to EM and ODP via configure.
  Example for a made up ARMv8 SoC called "soc-x":
  odp-soc-x/build> ../configure … --with-platform=soc-x [default=linux-generic]
     em-odp/build> ../configure … CFLAGS=”-O3 -march=armv8.2-a+… -mcpu=soc-x”
  The whole chain of used SW libs/dependencies should preferably be compiled
  with the same -march/-mcpu options for best results:
    other-libs/build> ../configure … CFLAGS=”-O3 -march=armv8.2-a+… -mcpu=soc-x”

- Program termination: flush the scheduler in em_term_core() on each EM-core.
  Move the flushing of scheduled events from em_term() to em_term_core() so that
  all participating EM-cores do it.
  See example usage of in em-odp/programs/common/cm_setup.c:
  ('thread' in the description below refers to odph_thread_t that can be either
   linux processes or threads)
  em_term() has been moved from each EM-core's run-function to the common thread
  that created the EM-core threads. The common thread waits for the created
  EM-core threads to return/join.
  After each EM-core has paused and flushed the scheduler from core-local events
  (by calling em_term_local()) they terminate by "joining" back into the common
  thread, which calls em_term() once to terminate EM (and ODP) before exiting
  the application.

- Fixes:
  - queue group: fix: em_queue_group_modify_sync() incorrectly removed the
                      calling core
  - ESV: fix: usr2em-revert transition to handle evgen wrap correctly
    Event State Verification (ESV) incorrectly handled the revert of a usr-to-EM
    transition when the counter wrapped.

- Example programs (em-odp/programs)
  - Common Setup (cm_setup.c&h):
    - Move the flushing of scheduled events from em_term() to em_term_core().
      Descibed above in "Program termination".
  - Packet-IO setup (cm_pktio.c&h)
    - Add command-line option to select whether to use an
      1) ODP-packet-pool: -o, --pktpool-odp
         or an
      2) EM event-pool: -e, --pktpool-em (default if no option given)
      as the main event pool to be used by packet-io for received packets.
      Packet-IO is implemented via ODP so an EM event-pool needs to be converted
      to the corresponding ODP packet-pools with the new extension API
      'em_odp_pool2odp()'. Using an EM pool, instead of directly using an
      ODP pool, allows for more thorough ESV-checking also for packet-IO when
      em-odp.conf: esv.prealloc_pools = true.
  - Timer
  - Misc. performance and bug fixes in the example applications

- Tested against
  odp-linux - https://github.com/OpenDataPlane/odp
    Commit: 69f27643ec2287875048314d7975f0de4bcbbfa4 [69f2764]
    Date: Friday, 1 October 2021 18:12.09
    Commit Date: Tuesday, 12 October 2021 17:06.16
    configure.ac: move project to C standard revision 11
  odp-dpdk - https://github.com/OpenDataPlane/odp-dpdk
    Commit: 50c7b605b97474d26fcf600b4061968e3543b45b [50c7b60]
    Date: Monday, 11 October 2021 16:37.51
    Merge ODP v1.32.0.0

--------------------------------------------------------------------------------
Event Machine on ODP v2.5.0
--------------------------------------------------------------------------------
- Support for EM API v2.5 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * Changes for ESV, Timer, Event, Pool, Extension-APIs, C++ *

- Error: don't allow error handlers to return fatal EM-errors.
  Fatal EM reported errors must be dealt with in the error handler, since EM
  cannot continue running after a fatal error.
  EM will now abort if an error handler returns a fatal error.

- Event State Verification (ESV)
  ESV is an EM-internal feature that aims to catch invalid EM-event usage.
  The idea is to catch invalid/illegal EM event usage in the application and
  report it as a FATAL-error to immediately abort the application​.
  The event-state is tracked and updated every time the event is transferred
  between EM and the application, e.g. in dispatch, em_send(), em_alloc(),
  em_free() etc.​
  Example scenarios that are detected:​ double-free, double-send, use-after-send,
  use-after-free etc.
  Additionally, an event-generation-count (evgen) has been added into the
  EM event handle (em_event_t) to catch usage of events that are no longer
  “owned” by the application (EO).
  The event handle will also change throughout the lifetime of the event.
  Using an out-of-date/obsolete event handle results in ESV error​.
  An event handle becomes invalid after it has been given to EM, e.g. reusing a
  stored event-handle after it has been sent will result in error, even if the
  same event would have been received again, because the event-handle has
  changed​.
  ESV is (currently) disabled by default and can be taken into use via a
  configure-option​:
        em-odp/build $> ../configure --prefix=… --enable-esv​
        em-odp/build $> make clean && make –j16 install​
  Using --enable-esv defines EM_ESV_ENABLE as ‘1’.
  ​
  Example Scenario - ESV Error:
    a) EO-receives an event​
    b) em_send(event, …)​ - EO sends the event
    c) em_free(event)    - EO calls free for an already sent event
       ==> Fatal Error:
   "
   EM ERROR:0x80000010  ESCOPE:0xFF000603  EO:0x5-"eo-a"​
   core:00 ecount:0(0) em_event_state.c:335 evstate_error()​
   Event:0x27f4a84c5d9c0 state error -- \
      counts current(vs.expected): evgen:2(3) free:1(1) send:1(0)​
      prev-state:em_send() core:00: EO:0x5-"eo-a" Q:0x1d-"q-a" u32[0]:0x00000000​
   => new-state:em_free()  core:00: EO:0x5-"eo-a" Q:0x1d-"q-a" u32[0]:0x00000000​
      event:0x00027f4a84c5d9c0: ptr:0x7f4a84c5d9c0
   "
  Note: ESV-errors are always reported with the same error-code:​
        EM_FATAL(EM_ERR_EVENT_STATE) = 0x80000010​
        'prev-state' shows the previous good state of the event. i.e. the
        previous place where EM observed the event in a valid state.
        'new-state' shows the current detected illegal state that
        caused the error.

  Using ESV has an impact on performance.
  Each event-state transition updates an atomic counter and stores further
  state data (EO, queue, core etc)​.
  The impact, of course, depends on the “number of state-transitions / second”​

- Event State Verification (ESV) options - config/em-odp.conf:
  Note: Options only considered if ESV is enabled at compile-time
          via the configure option '--enable-esv', see configure.ac
  - Option 'esv.enable'
    Runtime ESV enable/disable option. Allows the user to disable ESV
    without recompilation when 'configure --enable-esv'.
  - Option 'esv.store_payload_first_u32'
    Store the first 32bits of the event payload during each valid
    event-state transition. This allows for a comparison of the payload
    content between the previous valid state and the invalid state.
    Note: The first 32bits of the payload will always be printed in the
          error log for the invalid state regardless of this setting.
    Enabling will impact performace somewhat.
  - Option 'esv.prealloc_pools'
    Preallocate all events in a pool during pool creation to set
    an initial ESV-state for each event that can be tracked over
    multiple allocs and frees.

- EM config file options - config/em-odp.conf:
  - Option 'pool.pkt_headroom':
    Default minimum packet headroom in bytes for events allocated from
    EM-pools of type: EM_EVENT_TYPE_PACKET. Ignored for other pool types.

- Fixes:
  - timer: fix timer capability max_tmo values
  - event: fix: incorrect loop index when allocating multiple events
  - queue: fix em_queue_get_index() return on error
  - code style fixes (reported by code checkers)

--------------------------------------------------------------------------------
Event Machine on ODP v2.4.0
--------------------------------------------------------------------------------
- Support for EM API v2.4 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  * Changes for the Timer, Pool, EO and Queue APIs *

- Parameter/attribute initialization function usage is now mandatory before
  calling "create"-functions for EO-multircv, Timer and Pool:
    - call em_eo_multircv_param_init() before em_eo_create_multircv()
    - call em_pool_cfg_init() before em_pool_create()
    - call em_timer_attr_init() before em_timer_create()

- Build:
  - configure: option --with-odp-lib=[name of used odp library] added.
    Add the 'configure' script option --with-odp-lib=odp-library-name
    that allows the user to specify the name of the used ODP library.
    The library name defaults to "libodp-linux" if the option is omitted.
    E.g. configure em-odp to use odp-dpdk instead of odp-linux:
      > cd em-odp/build
      > ../configure ... [--with-odp-path=...] --with-odp-lib=libodp-dpdk
      > make
  - pkg-config: update em-odp dependencies in the package config file.
    Add explicit dependency on ODP: "Requires: @libodp_name@" (where the ODP
    library name is taken from the configure option --with-odp-lib=... if given)
    (em-odp/pkgconfig/libemodp.pc.in => <install-dir>/lib/pkgconfig/libemodp.pc)
  - Support for 'make dist' and 'make distcheck'
    - make dist: create em-odp-2.4.0.tar.gz distribution
    - make distcheck: validate the distribution,
      i.e. use the .tar.gz: compile, install, clean etc.

- Fixes:
  - GCC 10 compilation fixed
  - pool: incorrect return value used when EM-pool creation failed
  - em_eo_get_name() return value used incorrectly
  - Refactoring EM example program startup

--------------------------------------------------------------------------------
Event Machine on ODP v2.3.0
--------------------------------------------------------------------------------
- Support for EM API v2.3 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API:
    - Event: multi-event alloc and free functions
    - Hooks: use the same hook-type for all allocs and the same for all frees
    - EO: multi-event EO-receive function
    - Dispatch: one common enter-callback for multi- and single-receive funcs.
    - Init: core masks for input_poll_fn and output_drain_fn added to em_conf_t
    - Init: use em_conf_init(em_conf_t *conf) to init the EM config params.
    - Event: APIs to get the event type(s) of multiple events
- Fix libemodp symbol visibility, only expose API symbols
- New example program:     timer_test_periodic
  New packet-io programs:  loopback_local_multircv, loopback_multircv
  New performance program: loop_multircv

--------------------------------------------------------------------------------
Event Machine on ODP v2.2.0
--------------------------------------------------------------------------------
- Support for EM API v2.2 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
- EM config file options - config/em-odp.conf
    - Option 'pool.align_offset':
      (Replaced 'pool.alloc_align' with 'pool.align_offset')

      Alignment offset in bytes for the event payload start address.
      Set the event payload alignment offset for events allocated from
      any pool. This is a global setting concerning all pools.
      A similar, but pool-secific option, is 'em_pool_cfg_t:align_offset{}'
      that overrides this global setting for a specific pool when given to
      em_pool_create(..., pool_cfg).

      Use this option to globally adjust the payload layout so that a
      specific part of it can be placed at a needed alignment for e.g.
      HW access.

      The default EM event payload start address alignment is a power-of-two
      that is at minimum 32 bytes (i.e. 32 B, 64 B, 128 B etc. depending on
      e.g. target cache-line size).
      The 'align_offset' option can be used to fine-tune the start-address
      by a small offset to e.g. make room for a small SW header before the
      rest of the payload that might need a specific alignment for direct
      HW-access.
      Example: setting 'align_offset = 8' makes sure that the payload
      _after_ 8 bytes will be aligned at minimum (2^x) 32 bytes for all
      pools that do not override this value.

      start: base - align_offset
           |
           v
           +------------------------------------------+
           | <----- |  Event Payload                  |
           +------------------------------------------+
                    ^
                    |
                   base (min 32B aligned, power-of-two)
- New autoconf configure-script options (configure.ac):
    --without-programs
      Build the EM library without example programs
    --with-config-file=FILE
      Override the default EM configuration file (default:config/em-odp.conf)
      that is used during the configure & build phases to generate the built-in
      default values for the EM runtime-config (note that runtime config options
      can further be changed during startup by setting the EM_CONFIG_FILE=file
      variable).
      The overriding file set with --with-config-file=FILE must include all
      EM configuration options (unlike the runtime file that only need to
      include the mandatory and changed options).
    --enable-check-level=VAL(0…3)
      Set the compile-time EM_CHECK_LEVEL,
      see include/event_machine/platform/event_machine_config.h

--------------------------------------------------------------------------------
Event Machine on ODP v2.1.2
--------------------------------------------------------------------------------
- EM config file options - config/em-odp.conf
  EM runtime configuration options added. The user can optionally override
  default config options by setting the environmental variable 'EM_CONFIG_FILE'
  to point to an alternative config file with desired values, e.g.:
        $ sudo EM_CONFIG_FILE=./my-overrides.conf ./em-application -c 0xfe
  The configuration file values in config/em-odp.conf are the default values
  set for em-odp during the configure/build phase and the values defined there
  are used unless overrided via the optional EM_CONFIG_FILE. A user provided
  config file does not have to include all available options, only the mandatory
  options as well as the options that the user wants to change can be included
  (while the missing options will be replaced with the default build-time
   values).
    - Option 'pool.statistics_enable':
      Enable EM pool usage statistics collection during runtime (true/false).
      Pool usage statistics is updated e.g. during em_alloc() and em_free(),
      thus, if enabled, inducing some overhead. Pool information can be
      obtained by calling em_pool_info() and, if pool usage statistics is
      enabled, will return also pool usage statistics.
    - Option 'pool.alloc_align': (changed to 'pool.align_offset' in v2.2.0)
      Alloc alignment in bytes, default = 0 (no extra alignment requirement)
      Set the exact alignment of the event payload start address (in bytes).
      The event payload address returned by em_event_pointer(event) will
      have the this alignment. Use power of two values only.
    - Option 'queue.min_events_default':
      Default minimum number of events that a queue can hold.
      This value will be used in queue creation (em_queue_create*()) if no
      other size information has been provided via the
      em_queue_conf_t::min_events parameter.
      Setting 'min_events_default = 0' will use the odp-implementation's
      default values (might vary from one odp-implementation to another).
    - Option 'event_chaining.num_order_queues'
      Number of queues used for ordering event-chaining events.
      Maintain the order of events sent to another device from within
      an EM ordered context (i.e. send from within the EO receive function
      when handling an event from a parallel-ordered queue).
      An event-chaining queue has no 'real' queue associated with it, the
      queue-id simply indicates that events sent to the queue should be
      sent out of EM via a user specified function (and perhaps out towards
      another device). Sending events out of EM requires some intervention,
      especially sending during an ordered context needs to maintain the
      event order as determined by the context. To maintain event order for
      event-chaining queues, em-odp uses a set of 'real' queues for order
      tracking - the number of these queues is set by this option.
      Note, 0 = no ordering.
- Event alloc size must be larger than zero, i.e. em_alloc(sz > 0, ...)
- Optimizations and performance improvements

--------------------------------------------------------------------------------
Event Machine on ODP v2.1.1
--------------------------------------------------------------------------------
- Example programs support graceful Ctrl-C termination, see em-odp/programs.
  Pressing Ctrl-C (once) initiates graceful example program termination instead
  of abruptly killing it. A second press of Ctrl-C will always kill the program
  immediately.
- Added the programs/performance/loop example that simply loops received events
  back into the queue they originated from and measures the per core
  performance (cycles/core and millions-of-events/core).
- Optimizations and performance improvements

--------------------------------------------------------------------------------
Event Machine on ODP v2.1.0
--------------------------------------------------------------------------------
- Support for EM API v2.1 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  Backwards compatible with EM 2.0 API as long as all fields of em_conf_t given
  to em_init() are intialized properly (inludes the new em_conf_t::api_hooks
  field).
- the EM_DEBUG #define has been removed and the behaviour made default.
- EM Timer: check the timer attributes against odp-timer capabilities during
  timer creation.
- Detect packets from outside of EM and initialize them as events before further
  EM processing.

--------------------------------------------------------------------------------
Event Machine on ODP v2.0.0
--------------------------------------------------------------------------------
- Support for EM API v2.0 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API
  Note: This is a larger enhancement release with new or modified arguments
        requiring changes to legacy code. All main features remain.
- Example/test application updated to use API v2.0.
- New examples for new API v2.0 features:
    - Local Queues:
      programs/example:     queue/queue_types_local.c
      programs/packet_io:   loopback_local.c
                            multistage_local.c
      programs/performance: queues_local.c
    Output Queues:
      All packet I/O examples have been updated to use output queues.
    Sync APIs:
      Example start-up has been modified to use _sync() versions of APIs.
    etc.

--------------------------------------------------------------------------------
Event Machine on ODP v1.4.0
--------------------------------------------------------------------------------
- Support for EM API v1.4 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API.
  Add a queue configuration argument to em_queue_create/_static(..., *conf).
  The 'conf' argument is optional (can be NULL) and can be used to pass extra
  attributes to the system specific implementation.
  See 'em_queue_conf_t' in include/event_machine/api/event_machine_types.h and
  the supported conf->flags in
  include/event_machine/platform/event_machine_hw_types.h:
  - conf->min_events can be used to request the 'minimum number of events' that
    a queue should be able to contain.
  - conf->flags can be used to request certain properties for the queue.
    At the time of writing odp supports these flags only for unscheduled queues.
    An unscheduled queue can be created as 'single-producer & single-consumer'
    by setting the following flags:
      unsch_conf->flags  = EM_QUEUE_FLAG_ENQ_NOT_MTSAFE;
      unsch_conf->flags |= EM_QUEUE_FLAG_DEQ_NOT_MTSAFE;
    An unscheduled queue can be created as 'non-blocking & lock-free' by setting
    the following flag:
      unsch_conf->flags = EM_QUEUE_FLAG_NONBLOCKING_LF;

  Event Pool name added - event pools can be given a name during creation and
  that name can be used to find the event pool handle later.
  The pool name given to em_pool_create(name, ...) can be used later on any
  core to obtain the event pool handle. em_pool_find() and em_pool_get_name()
  are new functions related to the added 'name'.

--------------------------------------------------------------------------------
Event Machine on ODP v1.3.0
--------------------------------------------------------------------------------
- Support for EM API v1.3 (em-odp/include), see API changes in
  em-odp/include/event_machine/README_API
- Support for EM add-ons APIs, read about add-on APIs in
  em-odp/include/event_machine/add-ons/README.
- EM event timer API v1.0.
  See new event timer examples/tests in em-odp/programs/example/timer/

--------------------------------------------------------------------------------
Event Machine on ODP v1.2.3
--------------------------------------------------------------------------------
- eo: em_eo_start(): if EO local-start functions are used: enable all the EO's
  queues only after the local-start funcs have been successfully run on all
  cores.

- eo: em_eo_stop(): the EO's global-stop function should be run only after all
  local-stop functions have completed on all cores. The EO's global-stop
  function can only be called after each core has seen and handled the
  stop-request, even if no local-stop functions are used, to ensure that no
  core is still processing EO-related code.

- eo: em_eo_delete() should remove all the remaining queues from the EO and
  delete them before deleting the actual EO.

- bugfix: queue group: incorrect check for zero mask during queue group delete.
  Calling em_queue_group_delete() could lead to an incorrectly reported error
  if the core mask was not modified to zero before deleting the queue group.
  Corrected.

- event group: add the EM_EVENT_GROUP_SAFE_MODE option (enabled by default).
  Guards event groups in undefined and error situations.
  Excess and aborted group events don't belong to a valid group when received.
  Most event group APIs check if the core local event group has expired during
  the receive function. Impacts performance when event groups are used.
  Set EM_EVENT_GROUP_SAFE_MODE to '0' to disable.

- misc: code clean-ups.

--------------------------------------------------------------------------------
Event Machine on ODP v1.2.2
--------------------------------------------------------------------------------
- ARM compilation and event alignment issues fixed

- EM packet pool size and packet pool buffer size defined in configuration
  parameters (event_machine_config.h)

- Build system changes
  - Installed header files added in library makefile
  - Shared object version information added
  - Unused package version number removed
  - Minor gitignore fix

--------------------------------------------------------------------------------
Event Machine on ODP v1.2.1
--------------------------------------------------------------------------------
- em_queue_group_get_name() implemented

- Strict style checking taken in use
  - Source files modified accordingly

- ODP specific API extensions implemented for packet I/O
  - EM internal includes removed from packet I/O setup

- Requires ODP v1.10.0.0 / v1.10.1.0

--------------------------------------------------------------------------------
Event Machine on ODP v1.2.0
--------------------------------------------------------------------------------
- First EM implementation on top of ODP, providing the same example
  applications than OpenEM.

- Support for EM API v1.2
  - em_queue_group_get_name() not yet implemented, waiting for ODP support

- Requires ODP v1.9.0.0