Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

fix drain leak #1401

Merged
merged 1 commit into from
Jan 18, 2022
Merged

fix drain leak #1401

merged 1 commit into from
Jan 18, 2022

Conversation

BertKleewein
Copy link
Contributor

Fix a regression introduced by #1301. The cb = cb || nop line in _sendPacket was adding a callback where none existed before. This changed the behavior of sendPacket (without the underscore) by adding a listener to the drain event, and you can quickly overflow the max # of listeners if you're trying to publish faster than the stream will allow.

@@ -175,7 +175,7 @@ function sendPacket (client, packet, cb) {
debug('sendPacket :: writing to stream')
const result = mqttPacket.writeToStream(packet, client.stream, client.options)
debug('sendPacket :: writeToStream result %s', result)
if (!result && cb) {
if (!result && cb && cb !== nop) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code (the way sendPacket handles writeToStream return values) is built on a number of false assumptions:

  1. It assumes that writeToStream will only return false if the packet was queued and is waiting to send. This is wrong. writeToStream can also fail if packet validation fails (in mqtt-packet/writeToStream.js) If this happens, sendPacket will wait for the stream to drain, then call the callback with no error.
  2. If the stream is full, and writeToStream returns false, it waits for all packets to send before calling any of their callbacks. Once all packets are sent, the drain event fires, and all callbacks get called at once. If the sending pattern prevents the stream from draining completely, then the callbacks will never be called (or they might be called long after the packet was actually sent).
    3, This assumes that there can only be 1000 packets waiting to send. After that, the maxListeners on the drain event overflows. I'm not sure how well the library handles this error (Does it raise an exception? If so, who catches it?)

A better fix would be to pass the callback into the stream's write method so it gets called when the packet is actually sent. At least this is how TLSSocket.write behaves. I don't know if this applies to all of the stream implementations that mqtt.js supports. This would require an update to the mqtt-packet library to support an additional callback parameter to the writeToStream function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plus, packetsend is emitted before the packet is actually sent (if the packet is ever actually sent). Is that what we want?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opened an issue on this in #1403

@vishnureddy17 vishnureddy17 requested a review from YoDaMa January 14, 2022 21:35
@vishnureddy17
Copy link
Member

@redboltz I'm not able to request a review from you for some reason, but pinging you in case you have any feedback here.

Copy link
Contributor

@YoDaMa YoDaMa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@YoDaMa YoDaMa merged commit 7ec4b8f into mqttjs:main Jan 18, 2022
BertKleewein added a commit to BertKleewein/MQTT.js that referenced this pull request Mar 10, 2022
BertKleewein added a commit that referenced this pull request Mar 14, 2022
…tthub workflow (#1443)

* fix: fix regression from #1401 and allow CI test failures to break github workflow

* temporary rollback to verify break works

* put change back in

* add comment to fix

* fix misspelling
robertsLando pushed a commit that referenced this pull request Jun 30, 2023
robertsLando pushed a commit that referenced this pull request Jun 30, 2023
…tthub workflow (#1443)

* fix: fix regression from #1401 and allow CI test failures to break github workflow

* temporary rollback to verify break works

* put change back in

* add comment to fix

* fix misspelling
@BertKleewein BertKleewein deleted the bad-nop-no-doughnut branch July 6, 2023 15:41
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants