You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fix excessive stack usage when calling vorbis_analysis_wrote with lots of samples
`vorbis_analysis_wrote` increments `v->pcm_current` by `vals`, and this
incremented value can be used by `_preextrapolate_helper` right after to
allocate a float array in the stack `v->pcm_current` positions large.
Clearly, since `alloca` does not check that there is enough stack space
available to satisfy the allocation request, this can lead to a stack
overflow and memory corruption, which at best have no effect, more
likely cause segmentation faults, and at worst introduce security risks.
The documentation for `vorbis_analysis_buffer` and
`vorbis_analysis_wrote` does not specify a maximum value for `vals`. It
states that "1024 is a reasonable choice", but callers are free to use
larger or smaller counts as they wish. Therefore, `libvorbis` not
handling this case is undesirable behavior.
To better handle this case without throwing the performance benefits of
`alloca` out the window, let's check whether the allocation would exceed
256 KiB (an estimate for the minimum stack space available is 1 MiB,
which is [the default on Windows
platforms](https://learn.microsoft.com/en-us/windows/win32/procthread/thread-stack-size)),
and if so fall back to a heap allocated array. The heap array that may
be allocated for this purpose is freed when `vorbis_dsp_clear` is
called. `_preextrapolate_helper` takes neglible execution time
compared to the encoding process for usual sample block sizes, though.
Signed-off-by: Alejandro González <me@alegon.dev>
0 commit comments