Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Refactor req-rep operators to use only ZMQ types #7

Merged
merged 17 commits into from
Jan 2, 2023

Conversation

glopesdev
Copy link
Member

@glopesdev glopesdev commented Dec 21, 2022

This PR refactors the req-rep operators to use only built-in types in ZeroMQ (byte[], string and NetMQMessage). This will allow removing the dependency from the OSC package and keep ZMQ support more agnostic to different serialization protocols.

Other feature highlights:

  • Request blocks waiting for the response from the server but can now be asynchronously cancelled.
  • Response can now operate in one of two modes:
    • As a source (i.e. with no inputs), it emits a ResponseContext which can be evaluated asynchronously using the new SendResponse operator. This provides the basis for flexibly evaluating responses in full asynchronous fashion while still keeping to the strict sequential nature of the req-rep pattern.
    • As a combinator (i.e. with one message input) it will pair incoming responses with requests using a blocking collection. This allows a cache of responses to be pre-computed and distributed back to clients on-demand.
  • Dealer now operates as fully asynchronously in distributing messages and routing back responses. The empty frame delimiter is prefixed by default to emulate the req-rep pattern.
  • Router can now also operate in source or combinator mode like the Response operator:
    • As a source (i.e. with no inputs) it works like an asynchronous version of the Response operator where multiple requests may be evaluated simultaneously and responses are correctly paired to clients by using message identities.
    • As a combinator (i.e. with one message input) incoming responses are immediately routed back to peers using each message identity.

Utility operators:

  • ToMessage constructs a multiple part message from a sequence of frames or data objects.
  • ConvertToFrame constructs a single frame for each data object in the sequence.
  • ConvertToString decodes a message frame into a string.
  • GetIdentity extracts all identity frames from a multiple part message, up to and including the empty frame delimiter.

@glopesdev glopesdev added the feature New planned feature label Dec 21, 2022
@glopesdev glopesdev added this to the 0.1.0 milestone Dec 21, 2022
@glopesdev glopesdev requested a review from RoboDoig December 21, 2022 01:09
@glopesdev
Copy link
Member Author

The last few commits refactor the basic req-rep operators to use a NetMQPoller instead of spinning up a Task object. This is to allow proper asynchronous cancellation. It is not really possible to guarantee that asynchronous disposal of a socket will always break out of the current blocking operation without a poller.

@glopesdev glopesdev merged commit 8461a29 into bonsai-rx:main Jan 2, 2023
@glopesdev glopesdev deleted the req-rep branch January 2, 2023 12:02
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
feature New planned feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant