[BUG] When used for long periods of time, responses become truncated

[Cheesy-Brik]

Description
When using a single ChatBot object instance for a while I notice it's reponses become strangely cutoff

Steps to Reproduce
I don't know what the exact issue is, but I running it through a discord bot for users to chat through, but after a while it's reponses become cutoff. I am using the "offical" version with the non-browser hosted version of the ai.

Expected behavior
it doesn't cutoff it's descriptions

[acheong08]

It's because of a 4000 token cutoff

[acheong08]

At that point, it only has an 800 token buffer

[acheong08]

You can ask it to continue

[acheong08]

I can add the ability to define the buffer space if necessary

[Cheesy-Brik]

I can add the ability to define the buffer space if necessary

Could you add a way to delete some of the previous tokens, or allow for it to delete the needed token space (I guess just deleting a token for every new one it writes). A defined buffer space would also be a helpful setting as rn I'm not using much ram at all.

[Cheesy-Brik]

Also thanks for replying so quickly, 4am for me didn't expect to get a response lmao

[acheong08]

On the latest commit, you can define the buffer space (in tokens) when initializing the Chatbot class. def __init__(self, api_key: str, buffer: int = None) -> None:

You can also remove history manually with
chatbot.prompt.chat_history.pop(0)

[Cheesy-Brik]

Thx

[Cheesy-Brik]

On the latest commit, you can define the buffer space (in tokens) when initializing the Chatbot class. def __init__(self, api_key: str, buffer: int = None) -> None:

You can also remove history manually with chatbot.prompt.chat_history.pop(0)

problem with loading convos, for some reason it comes up with

File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\revChatGPT\Official.py", line 339, in construct_prompt
    if len(self.enc.encode(prompt)) > (4000 - self.buffer or 3200):
TypeError: unsupported operand type(s) for -: 'int' and 'NoneType''

When trying to ask a question, problem with prompt loading maybe?

[Cheesy-Brik]

I loaded the conversation from a dumped json, so that's probably part of it

[acheong08]

It seems I have made some mistakes with saving/loading conversations. Haven't tested. Will do tomorrow

[acheong08]

Edit: Nevermind. I can't subtract if none. need more checking code. Fixing now...