Feature Request: Add Support for Rich Attachments to acts_as_chat ask method

[schappim]

Overview

Currently, the acts_as_chat implementation of the ask method lacks support for rich attachments (e.g., images, audio, PDFs) through the with: parameter. To bring parity with the core Chat class, I propose enhancing the ask method within acts_as_chat to handle attachments seamlessly.

Desired Functionality

Enhance the ask method so users can attach rich content directly:

# Text-only request
chat.ask "What can you tell me about this code?"

# Request with a single image attachment
chat.ask "What's in this image?", with: { image: "path/to/image.jpg" }

# Request with multiple attachments
chat.ask "Analyse this document and image together", with: {
  image: ["path/to/image1.jpg", "path/to/image2.png"],
  pdf: "path/to/document.pdf"
}

Implementation Steps

Step 1: Update the ask method in ChatMethods

Modify the ask method in the ChatMethods module to accept a with: parameter:

def ask(message = nil, with: {}, &block)
  message_attrs = { role: :user }

  if with.empty?
    message_attrs[:content] = message
  else
    message_attrs[:content] = RubyLLM::Content.new(message, with)
  end

  messages.create!(**message_attrs)
  to_llm.complete(&block)
end

alias say ask

Step 2: Enhance MessageMethods to handle Content objects

Update the extract_content method in the MessageMethods module:

def extract_content
  content.is_a?(RubyLLM::Content) ? content : content
end

Step 3: Update the persist_message_completion method

Modify the persist_message_completion method to ensure proper storage of Content objects:

def persist_message_completion(message)
  return unless message

  if message.tool_call_id
    tool_call_id = self.class.tool_call_class.constantize.find_by(tool_call_id: message.tool_call_id).id
  end

  transaction do
    @message.update!(
      role: message.role,
      content: message.content.is_a?(Array) ? message.content.to_json : message.content,
      model_id: message.model_id,
      tool_call_id: tool_call_id,
      input_tokens: message.input_tokens,
      output_tokens: message.output_tokens
    )

    persist_tool_calls(message.tool_calls) if message.tool_calls.present?
  end
end

Benefits

• Enables consistent, easy-to-use interfaces for attaching rich media.
• Enhances integration between ActiveRecord persistence and RubyLLM's content handling.
• Provides feature parity with the core Chat implementation, simplifying developer workflow.

[qelphybox]

@schappim, thanks for the proposal, I vote for that. Would you like to add a pull request with those updates? i'd use that even from the branch. Or i could make it if you mind :) 🙏

[phedinkus]

By chance, are you also using ActiveStorage for images/pdfs? The struggle I'm having, beside the with param missing from the acts_as_chat ask method, is accessing the file for Base64 encoding. I'd be fine doing the encoding myself but it would be nice if there was a more friendly way to load the content from the rails friendly document storage.