Improve agenda review and add custom LLM support #21
Conversation
This change allows user to provide custom llm calling functions
- Modified the `tictoc::toc()` function call to store the elapsed time in a variable named `elapsed`. - Added a new line to calculate and display the generation speed in tokens per second.
This commit introduces a new function `use_custom_llm()` that allows users to send requests to custom (local or remote) language model endpoints compatible with the OpenAI API specification. The function accepts an API key as an optional argument for services that require it. This function replaces `use_local_llm()` which didn't allow to use API keys
- Add checks for empty agenda elements and missing required items - Convert some stop statements to warning statements - Improve error messages for invalid time formats - Update tests to reflect changes
Added a call to set_prompts() at the beginning of the entity_extractor function to ensure that the prompts are initialized before processing the text. This change ensures that any customizations to the prompts are applied correctly throughout the text processing workflow.
- Modified the `speech_to_summary_workflow` function to dynamically set the `summarization_output_length` based on whether a multipart summary is being generated. This change ensures that for multipart summaries, the output length per part is set to 1 page, while for single summaries it remains at 3 pages, optimizing the readability and relevance of generated summaries.
The validation logic for the 'agenda' argument in the speech_to_summary_workflow function has been updated to handle character type inputs correctly. Previously, the function stopped if the 'agenda' list had more than one element, which was not handling cases where 'agenda' was a character vector representing file paths. The updated condition now checks if 'agenda' is a character vector and has more than one element, providing a clearer error message that only one agenda file can be provided.
The speech_to_summary_workflow function now explicitly prompts the user to review the generated agenda by displaying it in the console. This enhancement aims to facilitate the review process by providing immediate visibility of the agenda content, reducing the need for users to manually locate and open the agenda file separately.
Improve error handling in LLM calls by checking for different error message formats and extracting the error message accordingly.
WalkthroughThe recent updates encompass refining error handling, enhancing customization options for language model endpoints, and improving the user interface in R functions. Changes include better validation mechanisms, dynamic parameter adjustments, and clearer user messages. These modifications aim to provide a more robust, user-friendly, and flexible toolset for handling language models and data processing tasks in R. Changes
Poem
Recent Review DetailsConfiguration used: CodeRabbit UI Files selected for processing (1)
Additional comments not posted (4)
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
| @@ -191,7 +191,7 @@ minutes if the "rolling" method is used. See \code{summarise_transcript} for | |||
| more details.} | |||
|
|
|||
| \item{summarization_output_length}{An indication to the LLM regarding the | |||
| length of the output. See \code{summarise_transcript} for more details.} | |||
| length of the output in pages. See \code{summarise_transcript} for more details.} | |||
There was a problem hiding this comment.
Tip
Codebase Verification
The expanded search in the Markdown files has successfully located a mention of the feature related to specifying the output length in terms of pages within the README.md file. This confirms that the feature is documented, albeit not in a formal "release notes" section, but rather as part of the general documentation in the README.
Given this information, it appears that the feature is indeed documented, though perhaps not in the specific location expected by the original review comment. This suggests that while the feature is mentioned, it might not be highlighted in a dedicated section for release updates, which could affect how users are informed about new features.
Result:
The feature regarding the output length specification in pages is documented in the README.md file. However, it is not explicitly mentioned in a section labeled as "release notes." It would be beneficial to ensure that such significant features are also highlighted in the release notes for better visibility and clarity for the users.
Analysis chain
The documentation now specifies the output length in pages, which aligns with the dynamic output length feature. This change should be clearly communicated in the release notes to inform users about the new output format.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if the release notes mention the new output length specification.
rg --type md 'output length in pages'
Length of output: 39
Script:
#!/bin/bash
# Broaden the search to include variations and check if the feature is mentioned in any Markdown files.
rg --type md -i 'output length|pages' -C 3
Length of output: 516
Enhancements
speech_to_summary_workflowfunction now displays the generated agenda in the console, facilitating review and reducing the need to locate the agenda file separately. (Commit: 3bed1cc).use_local_llm()withuse_custom_llm()to send requests to custom (local or remote) language model endpoints compatible with the OpenAI API specification, allowing to use also API Keys. (Commit: 0fdfe57).summarization_output_lengthin the workflow function based on whether a multipart summary is generated (shorter outputs) or not (longer outputs), optimizing the readability and relevance of summaries. (Commit: 2511287).Fixes
output_lengthwas not being considered in the summarization process. (Commit: 08e95d1).speech_to_summary_workflowfunction to handle character type inputs correctly and provide clearer error messages. (Commit: d200a55).Documentation
Summary
This pull request focuses on improving the user experience and adding support for custom language model endpoints. It enhances the agenda review process, ensures dynamic output length adjustment, fixes agenda validation, and adds documentation typo corrections.
Summary by CodeRabbit
New Features
interrogate_llmfunction to improve error handling and added generation speed calculation.use_custom_llmfunction allowing integration with custom language model endpoints, including optional API key support.Bug Fixes
use_openai_llmfunction description.Documentation
use_custom_llmfunction.Refactor
use_local_llmtouse_custom_llmand enhanced its functionality.