feat: Add Raw Response Metadata Handling - GPT Token Usage Example

[DZunke]

Currently it is not really possible to get into details of the interaction between user and model because the HTTPResponse is hidden behind the response converter and is then forgotten forever. So the only possibility currently to get details from the interaction would be a custom response and a custom response converter.

As a draft and to collect your ideas and other input i want to throw this stone into the lake with having the HTTPResponse also transported into the direction of the output processors. Could this give a bit more flexibility? What do you think?

Please have in mind this is only a draft to discuss. I thought about having this also possible if a request fails with an error status code in the future so the output processors could handle in a separation of convern way if there should be an exception thrown. Or do we maybe need an additional layer on top of the response converters itself?

For sure this is the simplest use case with the text response in mind. Maybe for the streamed response it would need a more "recording" approach with an output processor. But would surely be possible.

[OskarStark]

Looks great, I would only vote vor having a Metadata object instead of an array

[DZunke]

Looks great, I would only vote vor having a Metadata object instead of an array

Thanks ... 👍🏼 ... i thought about extending the collection, if it is generally fine, with a key value (as object) storage and search by name, etc. so a bit of comfort in the metadata then. Just was unsure if this kind of "transport" is in the interest of the library or if someone has other restructurings in mind that would make it "easier" or more complicated.

[chr-hertel]

Yeees, we need that - thanks for kicking it off! 😍
I also like the two levels here with a more structured metadata object and the raw response.

And I would even prefer to have them lower. Basically in the ResponseInterface directly:

namespace PhpLlm\LlmChain\Model\Response;

use PhpLlm\LlmChain\Model\Response\Metadata;
use Symfony\Contracts\HttpClient\ResponseInterface as SymfonyHttpResponse;

interface ResponseInterface
{
    /**
     * @return string|iterable<mixed>|object|null
     */
    public function getContent(): string|iterable|object|null;

    /**
     * Returns a key-value set based on provider specific response meta data.
     */
    public function getMetadata(): Metadata;

    /**
     * Return the original API response.
     */
    public function getRawResponse(): SymfonyHttpResponse;
}

I know that this would shift the responsibility from the OutputProcessor to every ResponseConverter, but maybe we can shift that into more clever abstractions/helper/trait/younameit if there are redundancies.

@4lxndr @tryvin that would also help with your use-cases, right?

[tryvin]

@chr-hertel Yes, it does! Thanks for letting me know!

I've been so busy dealing with external providers that I didn't have the time to do what I promised in the issue.

But I will keep an eye on this PR

[chr-hertel]

While working on #280 and with that fake PipelineResponse I was wondering if we should rather use terminology like Input/Output on this layer already/as well - instead of Request/Response - what do you think?

[DZunke]

Would this not very confusing? Then there would be two different "Output" named layers within the data flow.

[chr-hertel]

Yeah true, having that terminology twice could be annoying 🤔

[OskarStark]

ModelResponse? ModelOutput? To be more specific regarding the layer? Just thinking out loud

[chr-hertel]

Nevermind the pipeline if it's only the commit thingy - need to dig deeper why that is so flaky

[chr-hertel]

from an OOP point of view - why choose traits over an abstract base implementation of the response? isn't it that all response classes have the two traits?

with having the interface people could still decide to do their own - or we could do both - having a base class that uses both traits. not sure if i miss the downside here

[DZunke]

from an OOP point of view - why choose traits over an abstract base implementation of the response? isn't it that all response classes have the two traits?

with having the interface people could still decide to do their own - or we could do both - having a base class that uses both traits. not sure if i miss the downside here

Mhm. Yeah. First i wanted to have a most flexible approach but after revisiting it i have implemented an abstract base response to extend from if needed. But i also kept the traits as a handy helpers, if this is fine? See AsyncResponse where i have not extended from the BaseResponse but utilized the MetadataAwareTrait as it has it's own raw symfony response usage.

What do you think?

[chr-hertel]

Thanks a ton for taking care about this @DZunke - that feature was overdue!

[OskarStark]

Nice one 👏