[Config Support]: Database or disk is full

[tlundste]

Describe the problem you are having

Frigate won't fully start up. I do see a "database or disk is full" message in the logs during startup right after an indication that vacuum is starting.
Frigate was running (mostly) fine and had been running I think at least a week or more without issue. I started having a minor issue where the history was not loading, so I thought I should restart Frigate, and see if that cleaned anything up, but during restart, I noticed this message, and the restart loop began.
I'm running Frigate as a docker container in Ubuntu 24.04 LTS on hardware that also has home assistant and other containers. The 300 GB disk where all of the containers have their data was only 66% full when the issue began, and I've knocked that down to 64% (no change in startup behavior after that). The database is only about 45 MB, so I would expect there would be plenty of space (over 100 GB) for whatever this "vacuum" process does.
I've reviewed a similar article from Feb of this year, and I don't see a resolution on that one, and I believe I have my config set up appropriately (maybe not the most efficient and safe, but workable). I only have three cameras, and only keep 3 days of snapshots and recordings at this time. And, as indicated, with only a 45 megabyte database, 100 gigabytes of free space should be plenty.
Is there a parameter that limits the database to a specific size? I can see in my frigate/config folder, there is a backup.db file from back in Aug that is over 70 MB... so, I'm guessing that might have been a backup that I took during one of the major upgrades.

Version

0.14.0

Frigate config file

mqtt:
  host: a.b.c.133
detectors:
  coral:
    type: edgetpu
    device: usb
go2rtc:
  streams:
    frontdoor:
      - rtsp://admin-userID:password-string@a.b.c.14:554/cam/realmonitor?channel=1&subtype=0
    reo-main:
#      - rtsp://admin-userID:password-string@a.b.z.224:554/h264Preview_01_main
      - ffmpeg:https://a.b.z.224/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin-userID&password=password-string#video=copy#audio=copy#audio=opus
    reo-basement:
#      - rtsp://admin-userID:password-string@a.b.z.223:554/h264Preview_01_main
      - ffmpeg:https://a.b.z.223/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin-userID&password=password-string#video=copy#audio=copy#audio=opus
cameras:
  frontdoor:
    mqtt:
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
    ffmpeg:
      global_args: -hide_banner -loglevel warning
      hwaccel_args: []
      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport
        tcp -timeout 5000000 -use_wallclock_as_timestamps 1
      output_args:
        detect: -f rawvideo -pix_fmt yuv420p
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps
          1 -strftime 1 -c copy -codec:a aac
      inputs:
        - path: rtsp://127.0.0.1:8554/frontdoor
#        - path: rtsp://admin-userID:password-string@a.b.c.14:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - audio
            - detect
            - rtmp
            - record
    audio:
      enabled: true
      listen:
        - bark
      filters:
        bark:
          threshold: 0.8
    detect:
      width: 1920
      height: 1080
      fps: 5
    objects:
      track:
        - dog
        - person
        - car
      filters:
        car:
          mask:
            - 594,687,1260,695,1314,739,1625,755,1636,446,1729,386,1584,52,1737,53,1733,0,1625,0,1521,0,1297,169,1059,95,876,69,703,146,669,307,624,278,495,310,437,369
            - 191,1080,139,812,1140,1080
            - 1576,1080,1202,1080,1142,936,933,820,734,778,877,732,1137,748,1613,814
          threshold: 0.65
        person:
          mask:
            - 594,687,1260,695,1314,739,1625,755,1636,446,1729,386,1584,52,1737,53,1733,0,1625,0,1521,0,1297,169,1059,95,876,69,703,146,669,307,624,278,495,310,437,369
            - 358,965,537,887,522,861,348,869
            - 1072,0,1288,0,1290,1080,1081,1080
        dog:
          mask:
            - 594,687,1260,695,1314,739,1625,755,1636,446,1729,386,1584,52,1737,53,1733,0,1625,0,1521,0,1297,169,1059,95,876,69,703,146,669,307,624,278,495,310,437,369
    record:
      enabled: true
      retain:
        days: 2
        mode: all
      events:
        retain:
          default: 2
          mode: all
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2
    motion:
      mask: 
        0.321,0.617,0.615,0.619,0.849,0.609,0.854,0.416,0.901,0.357,0.825,0.048,0.875,0.064,0.908,0.063,0.903,0,0.846,0,0.792,0,0.676,0.156,0.543,0.051,0.397,0.078,0.34,0.156,0.343,0.257,0.325,0.257,0.26,0.256,0.217,0.325
    zones:
      fntdr_drvwy:
        coordinates: 0.458,0.68,0.324,0.747,0.319,0.66,0.375,0.656
        inertia: 3
        loitering_time: 0
        objects:
          - car
          - dog
          - person
      fntdr_prch:
        coordinates: 0.099,1,0.072,0.752,0.594,1
        inertia: 3
        loitering_time: 0
        objects:
          - dog
          - person
      fntdr_strt:
        coordinates: 
          0.575,0.688,0.604,0.692,0.608,0.608,0.532,0.606,0.393,0.629,0.317,0.622,0.322,0.652,0.379,0.662,0.468,0.675
        inertia: 3
        loitering_time: 0
        objects:
          - car
          - person
      fntdr_wlkwy:
        coordinates: 
          0.585,0.856,0.64,0.977,0.365,0.944,0.404,0.846,0.41,0.775,0.334,0.752,0.376,0.721,0.482,0.756
        inertia: 3
        loitering_time: 0
        objects:
          - dog
          - person
      fntdr_yard:
        coordinates: 
          0.821,1,0.626,1,0.595,0.867,0.486,0.759,0.382,0.72,0.457,0.678,0.592,0.693,0.84,0.754
        inertia: 3
        loitering_time: 0
        objects:
          - dog
          - person
          - car
  reo-main:
    mqtt:
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
    ffmpeg:
      global_args: -hide_banner -loglevel warning
      hwaccel_args: []
#      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport
#        tcp -timeout 5000000 -use_wallclock_as_timestamps 1
#      input_args: preset-rtsp-stream
      output_args:
#        detect: -f rawvideo -pix_fmt yuv420p
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps
          1 -strftime 1 -c copy -codec:a aac
      inputs:
#        - path: rtsp://admin-userID:password-string@a.b.z.224:554/h264Preview_01_main
        - path: rtsp://127.0.0.1:8554/reo-main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
#            - rtmp
    objects:
      track:
        - person
      mask:
        - 0.296,0.103,0.334,0.1,0.339,0.159,0.291,0.164
        - 0.38,0.11,0.378,0.179,0.424,0.175,0.424,0.108
        - 0.028,0.315,0.04,0.399,0.081,0.393,0.067,0.314
        - 0.566,0.337,0.817,0.363,0.822,0.207,0.869,0.223,0.872,0.151,0.868,0.055,0.828,0.059,0.813,0.134,0.769,0.115,0.759,0.017,0.626,0.019,0.631,0.123,0.571,0.162
        - 0.904,0.32,0.904,0.379,0.935,0.378,0.934,0.321
        - 0.328,0.234,0.365,0.231,0.372,0.305,0.331,0.313
        - 0.134,0.637,0.684,0.638,0.498,0.994,0.149,0.991
    record:
      enabled: true
      retain:
        days: 2
        mode: all
      events:
        retain:
          default: 2
          mode: all
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2
    motion:
      mask: 0.573,0.041,0.573,0,0.465,0,0.465,0.039
  reo-basement:
    mqtt:
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
    ffmpeg:
      global_args: -hide_banner -loglevel warning
#      hwaccel_args: []
#      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport
#        tcp -timeout 5000000 -use_wallclock_as_timestamps 1
      output_args:
        detect: -f rawvideo -pix_fmt yuv420p
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps
          1 -strftime 1 -c copy -codec:a aac
      inputs:
#        - path: rtsp://admin-userID:password-string@a.b.z.223:554/h264Preview_01_main
        - path: rtsp://127.0.0.1:8554/reo-basement
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
#            - rtmp
    objects:
      track:
        - person
      mask:
        - 0.085,0.222,0.126,0.226,0.128,0.082,0.171,0.097,0.175,0.223,0.178,0.263,0.21,0.256,0.207,0.366,0.269,0.339,0.268,0.396,0.208,0.437,0.217,0.811,0.122,0.835,0.119,0.481,0.049,0.49,0.043,0.424,0.101,0.39
        - 0,0.322,0.031,0.331,0.053,0.763,0,0.758
    record:
      enabled: true
      retain:
        days: 2
        mode: all
      events:
        retain:
          default: 2
          mode: all
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 2
    motion:
      mask: 0.581,0.047,0.58,0,0.52,0,0.518,0.048
    zones: {}
version: 0.14
camera_groups:
  Outside:
    order: 1
    icon: LuTreeDeciduous
    cameras: frontdoor
  Inside:
    order: 2
    icon: LuDoorOpen
    cameras:
      - reo-basement
      - reo-main

Relevant Frigate log output

2024-12-23 15:02:48.358242961  [2024-12-23 15:02:48] peewee_migrate.logs            INFO    : Starting migrations
2024-12-23 15:02:48.358325406  [2024-12-23 15:02:48] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-12-23 15:02:48.358631978  [2024-12-23 15:02:48] frigate.app                    INFO    : Running database vacuum
2024-12-23 15:02:48.392024534  database or disk is full

Relevant go2rtc log output

N/a

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

docker-compose file or Docker CLI command

frigate:
   container_name: frigate
   shm_size: 150m
   privileged: false # this may not be necessary for all setups
   restart: unless-stopped
   image: ghcr.io/blakeblackshear/frigate:0.14.0
   devices:
     - /dev/bus/usb:/dev/bus/usb
     - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
   volumes:
     - /etc/localtime:/etc/localtime:ro
     - /mnt/server/frigate/config:/config
     - /mnt/server/frigate/media:/media/frigate
     - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
       target: /tmp/cache
       tmpfs:
         size: 1000000000
   ports:
     - "5000:5000"
     - "1935:1935" # RTMP feeds
     - "8554:8554" # RTSP feeds
   environment:
     FRIGATE_RTSP_PASSWORD: "nunya"
     PLUS_API_KEY: longstringofstuff

Object Detector

Coral

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[tlundste]

I continued trying to troubleshoot things and clean other things up. I went through the process to remove unused docker images (older versions of running containers with new version images). After doing that, my mountpoint of the in-use space for the running containers (all the /mnt/server references in the docker config) had not changed... it still showed 100 GB free. I'm sure another mount point whose usage I was not monitoring reduced some, but no mount point was so full it couldn't take another several hundred MB of files, and as previously indicated, the DB file in /frigate/config was only around 45 MB. For kicks and grins, not expecting it to work, I started Frigate, and it came up. The current DB is now 25 MB instead of 45 MB, and the 300 GB mount point where I keep the running config files (and media) reduced from around 64% utilized down to 58% utilized. I'm assuming that is because things weren't working for long enough that my 3-day retention got hit, and a lot of stuff got deleted, after the system had been down for a few hours and possibly not functioning properly prior to that for who-knows-how-long.
So...
When the vacuum process runs... where does it need (I assume temporary) space in order to complete, cuz there was plenty of space in the mountpoint where the DB lives to do that. I'm guessing it needs extra space somewhere else.

[NickM-27]

vacuum typically writes to the same storage as the original DB in the .waf file