5 second audio delay converting PCM-u to OPUS

[Simanias]

When I use go2rtc (version 1.9.8 and below) to convert my Hikvision audio from PCM-u to OPUS the recordings have a 5 to 10 second audio delay.

go2rtc:
  log:
    exec: trace
  streams:
    deurbel:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.20.47:554/Streaming/Channels/101
      - isapi://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.20.47:80
      - ffmpeg:deurbel#audio=opus
    deurbel_sub:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.20.47:554/Streaming/Channels/102

cameras:  
  deurbel:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/deurbel_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/deurbel?audio&video
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-aac

If I use the direct stream from the camera to record, the audio is okey:

cameras:
  deurbel:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/deurbel_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: 
            rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.20.47:554/Streaming/Channels/101
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-aac

Here is the probe of the camera:

{
  "producers": [
    {
      "id": 62,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "192.168.20.47:554",
      "url": "rtsp://*****:*****@192.168.20.47:554/Streaming/Channels/101",
      "sdp": "v=0\r\no=- 1737454098745453 1737454098745453 IN IP4 192.168.20.47\r\ns=Media Presentation\r\ne=NONE\r\nb=AS:5100\r\nt=0 0\r\na=control:rtsp://192.168.20.47:554/Streaming/Channels/101/\r\nm=video 0 RTP/AVP 96\r\nb=AS:5000\r\na=control:rtsp://192.168.20.47:554/Streaming/Channels/101/trackID=1\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 profile-level-id=420029; packetization-mode=1; sprop-parameter-sets=\r\nm=audio 0 RTP/AVP 0\r\nb=AS:50\r\na=control:rtsp://192.168.20.47:554/Streaming/Channels/101/trackID=2\r\na=rtpmap:0 PCMU/8000\r\na=Media_header:MEDIAINFO=494D4B48010100000400010010710110401F0000000;\r\na=appversion:1.0\r\n",
      "user_agent": "go2rtc/1.9.8",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, PCMU/8000"
      ],
      "receivers": [
        {
          "id": 63,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "childs": [
            64,
            66
          ],
          "bytes": 960,
          "packets": 3
        },
        {
          "id": 70,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "childs": [
            71
          ]
        }
      ],
      "bytes_recv": 996
    },
    {
      "id": 67,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "127.0.0.1:56594 forwarded 127.0.0.1:8554",
      "source": "exec:ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://127.0.0.1:8554/deurbel?audio\u0026source=ffmpeg:deurbel%23audio%3Dopus -c:a libopus -application:a lowdelay -min_comp 0 -vn -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/feaafb27d4e2013318cd865f9fec20a1",
      "url": "rtsp://127.0.0.1:8554/deurbel?audio\u0026source=ffmpeg:deurbel%23audio%3Dopus",
      "sdp": "v=0\r\no=- 0 0 IN IP4 127.0.0.1\r\ns=go2rtc/1.9.8\r\nc=IN IP4 127.0.0.1\r\nt=0 0\r\na=tool:libavformat LIBAVFORMAT_VERSION\r\nm=audio 0 RTP/AVP 96\r\nb=AS:64\r\na=rtpmap:96 opus/48000/2\r\na=control:streamid=0\r\n",
      "user_agent": "ffmpeg/go2rtc",
      "medias": [
        "audio, recvonly, OPUS/48000/2"
      ],
      "receivers": [
        {
          "id": 68,
          "codec": {
            "channels": 2,
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "childs": [
            69
          ]
        }
      ]
    }
  ],
  "consumers": [
    {
      "id": 65,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "127.0.0.1:56588",
      "source": "ffmpeg:deurbel#audio=opus",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.8\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 PCMU/8000\r\na=control:trackID=0\r\n",
      "user_agent": "go2rtc/ffmpeg",
      "medias": [
        "audio, sendonly, ANY"
      ],
      "senders": [
        {
          "id": 66,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "parent": 63,
          "bytes": 960,
          "packets": 3
        }
      ],
      "bytes_send": 1008
    },
    {
      "id": 61,
      "format_name": "probe",
      "protocol": "http",
      "remote_addr": "192.168.1.155:65413",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:134.0) Gecko/20100101 Firefox/134.0",
      "medias": [
        "audio, sendonly, ALL",
        "video, sendonly, ALL",
        "audio, recvonly, ANY"
      ],
      "senders": [
        {
          "id": 64,
          "codec": {
            "codec_name": "pcm_mulaw",
            "codec_type": "audio",
            "sample_rate": 8000
          },
          "parent": 63,
          "bytes": 960,
          "packets": 3
        },
        {
          "id": 69,
          "codec": {
            "channels": 2,
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "parent": 68,
          "bytes": 694,
          "packets": 4
        },
        {
          "id": 71,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 70
        }
      ]
    }
  ]
}

Any thoughts or solutions to tackle this issue, eg. ffmpeg options?

[Simanias]

Okey after mannually setting the input and output args, I'm now up to 0.2 seconds.

  deurbel:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/deurbel_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/deurbel?audio&video
          input_args:
            - -rtsp_transport
            - tcp
            - -fflags
            - nobuffer
            - -flags
            - low_delay
            - -timeout
            - "5000000"
          roles:
            - record
      output_args:
        record:
          - "-c:v"
          - "copy"
          - "-c:a"
          - "aac"
          - "-ar"
          - "8000"  # Correct place for setting the audio sample rate
          - "-f"
          - "segment"
          - "-segment_time"
          - "10"
          - "-segment_format"
          - "mp4"
          - "-reset_timestamps"
          - "1"
          - "-strftime"
          - "1"

[AlexxIT]

Looks like your issue not related to go2rtc. Because all of your changes not in go2rtc config.