2 Way Audio Question

[KoljaV]

Hi,

thanks a lot for this great project and your support. Two way audio works great when using webRTC. Now, I am trying to integrate this in HomeKit. For that reason I either need an HTTP Input for the audio back channel or a specific rtsp stream (only used for audio back channel). Is there an either a HTTP or RTSP API where I can stream the audio to?

BR,
Kolja

[AlexxIT]

I don't know how you plan to use HTTP API. Because 2 way audio uses RTP packets with a specific audio codec. I just forward this packets to camera. WebRTC technology can generate this packets from user microphone out of box.

I can implement this packets on RTSP server (8554 port). But don't know software, that can use this technology.

[KoljaV]

This software generates an audio stream using ffmpeg an can stream it to rtsp: https://github.com/Sunoo/homebridge-camera-ffmpeg
But it is only an experimental feature.

Unfortunately, ffmpeg uses RTSP RECORD, which my camera rejects and there is no way of changing that. This is why I thought I could use a go2rtc stream for that.

[AlexxIT]

I had experiments on how to send sound from ffmpeg to the camera. This is not hard to do for a file on disk or for an audio stream. But I don't understand how to do this for audio from the microphone of any device. #57

I saw the homebridge project. I don't want to make a solution that a couple of geeks can setup. I want thousands of ordinary people to be able to use it. And to do that, it needs to be easy.

[KoljaV]

Got that point. Great work!
The homebridge project somehow extract the audio from HomeKit (most probably using some HomeKit API) and pipe it to ffmpeg (-i pipe:) as input. Ffmpeg then can do some audio codec conversion and send it back to a rtsp stream. Unfortunately it is not possible to manipulate the RTSP Control packets, which is why ffmpeg sends a RECORD.

[AlexxIT]

Now go2rtc support HomeKit to RTSP/WebRTC. I plan to add support for 2 way audio for Apple Home app. But haven't dealt with it yet.

In the future I plan to add support RTSP/WebRTC to HomeKit. Also with 2 way audio if it will be possible.

[KoljaV]

Sounds good. I will close my question. I believe that a preliminary step is to offer the audio backchannel as rtsp stream or as a http post interface instead of webRTC.

[gretel]

Now go2rtc support HomeKit to RTSP/WebRTC. I plan to add support for 2 way audio for Apple Home app. But haven't dealt with it yet.

now or no?

[AlexxIT]

At the moment.

[mofman]

Curious if two-way audio via homekit is still on the cards @AlexxIT?

[AlexxIT]

Oh. It's been almost two years.

This is a medium priority task at the moment. But there are a huge number of high-priority tasks.

[KoljaV]

Curious if two-way audio via homekit is still on the cards @AlexxIT?

I can recommend scrypted.app for that Feature.