Skip to content
This repository has been archived by the owner on Jan 5, 2025. It is now read-only.

wrong output of \n #162

Open
Z-ZHHH opened this issue Sep 21, 2023 · 2 comments
Open

wrong output of \n #162

Z-ZHHH opened this issue Sep 21, 2023 · 2 comments

Comments

@Z-ZHHH
Copy link

Z-ZHHH commented Sep 21, 2023

I run openchat on Webshop task. And the model output the wrong \n like \n, \\n, \\n. For example

'history': [{'observation': "WebShop [SEP] Instruction: [SEP] i'm looking for a topper for a birthday cake, and price lower than 30.00 dollars [SEP] Search", 'available_actions': {'has_search_bar': True, 'clickables': ['search']}, 'response': "{'id': 'chatcmpl-TxbAh6yUHXRtFoGYivTZQ9', 'object': 'chat.completion', 'created': 1695293501, 'model': 'openchat_v3.2_super', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': 'Thought:\\nI think I should use the search bar to look for the product I need.\\n\\nAction:\\nsearch[topper for birthday cake price < 30.00]\\n'}, 'finish_reason': 'stop'}], 'usage': {'prompt_tokens': 2279, 'total_tokens': 2325, 'completion_tokens': 46}}", 'action': '\\n', 'reward': 0, 'done': False}, 
{'observation': "WebShop [SEP] Instruction: [SEP] i'm looking for a topper for a birthday cake, and price lower than 30.00 dollars [SEP] Search", 'available_actions': {'has_search_bar': True, 'clickables': ['search']}, 'response': '{\'id\': \'chatcmpl-5tfxH6SqPyRDs8s9kexBpJ\', \'object\': \'chat.completion\', \'created\': 1695293524, \'model\': \'openchat_v3.2_super\', \'choices\': [{\'index\': 0, \'message\': {\'role\': \'assistant\', \'content\': "{\'id\': \'chatcmpl-TxbAh6yUHXRtFoGYivTZQ9\', \'object\': \'chat.completion\', \'created\': 1695293501,\'model\': \'openchat_v3.2_super\', \'choices\': [{\'index\': 0,\'message\': {\'role\': \'assistant\', \'content\': \'Thought:\\\\nI think I should use the search bar to look for the product I need.\\\\n\\\\nAction:\\\\nsearch[topper for birthday cake price < 30.00]\\\\n\'},

Beacuase of the problem, all tasks in Webshop failed, and the final result is zero. Could anyone help with that?

@codebanesr
Copy link
Contributor

I want to make sure I give you the right information. Could you explain a bit more about what you need help with? Some additional details would help me better understand the question

@Z-ZHHH
Copy link
Author

Z-ZHHH commented Sep 21, 2023

I want to reproduce the experiments of Openchat v3.2 super on Webshop (78.6% success rate). I installed the Webshop and Openchat environment and they work well. However, the success rate is zero on Webshop, I check the intermidiate response of Openchat and I found it ouput the wrong format of \n like Action: \\n search[ipad], but the correct format is Action: \n search[ipad].

# for free to subscribe to this conversation on GitHub. Already have an account? #.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants