Skip to content

stream option should take any callable not just a proc #262

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Merged
merged 1 commit into from
May 15, 2023

Conversation

obie
Copy link
Contributor

@obie obie commented May 14, 2023

Due to the way that Ruby closures work for procs, it's quite limiting to limit the stream interface to just Proc objects. This PR makes a tiny change so that callable objects work too. (Anything that implements a call method.)

Here's a simple example from my project MagmaChat:

  class StreamProcessor
    def initialize(bot:, buffer:, message:)
      Rails.logger.info "💦💦💦 initializing stream processor for message: #{message} 💦💦💦"
      @buffer = buffer
      @message = message
    end

    def call(chunk, _bytesize=nil)
      Rails.logger.info "💦💦💦 streaming chunk: #{chunk} (#{_bytesize} bytes) 💦💦💦"
      if new_content = chunk.dig("choices", 0, "delta", "content")
        @buffer << new_content
        ActiveRecord::Base.logger.silence do
          @message.update(content: @message.content + new_content)
        end
      end
    end
  end

@alexrudall
Copy link
Owner

Nice. thanks @obie

@alexrudall alexrudall merged commit 1ded4b7 into alexrudall:main May 15, 2023
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants