-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
feat: Audio Filter plugin #559
base: main
Are you sure you want to change the base?
Conversation
it seems like you haven't added any nanpa changeset files to this PR. if this pull request includes changes to code, make sure to add a changeset, by writing a file to
refer to the manpage for more information. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm but i wouldn't give my review too much weight, needs @theomonnom
livekit/src/plugin.rs
Outdated
let options = CString::new(options.as_ref()).unwrap_or(CString::new("").unwrap()); | ||
unsafe { | ||
let on_load = lib.get::<Symbol<OnLoadFn>>(b"on_load")?; | ||
on_load(options.as_ptr()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see the on_load function do a blocking request to our livekit server (on the krisp-plugin repo).
I think we should add 3 things:
- making sure we don't block the tokio loop when loaded.
- add retries to this request (or expose failures?)
- check how slow it is to load the weights into memory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Moved this initialization to
Room::connect
, now connect returns error if on_load fails. - Put plugin loading codes in spawn_blocking so that we can avoid tokio runtime are blocked.
livekit/src/plugin.rs
Outdated
let create_fn: CreateFn = unsafe { std::mem::transmute(self.create_fn_ptr) }; | ||
|
||
let options = CString::new(options.as_ref()).unwrap_or(CString::new("").unwrap()); | ||
let ptr = unsafe { create_fn(sampling_rate, options.as_ptr()) }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we check if the ptr is valid?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just added for null check for this in 1bdb932
This pull request introduces a feature that allows users to load their own audio filter library and use it as an inbound audio filter. It can be used for purposes such as noise cancellation, VAD, and more.
The plugins must be dynamic libraries (
cdylib
) and are required to export the following functions: