Skip to content

Commit c9694a0

Browse files
committed
Fix excessive stack usage when calling vorbis_analysis_wrote with lots of samples
`vorbis_analysis_wrote` increments `v->pcm_current` by `vals`, and this incremented value can be used by `_preextrapolate_helper` right after to allocate a float array in the stack `v->pcm_current` positions large. Clearly, since `alloca` does not check that there is enough stack space available to satisfy the allocation request, this can lead to a stack overflow and memory corruption, which at best have no effect, more likely cause segmentation faults, and at worst introduce security risks. The documentation for `vorbis_analysis_buffer` and `vorbis_analysis_wrote` does not specify a maximum value for `vals`. It states that "1024 is a reasonable choice", but callers are free to use larger or smaller counts as they wish. Therefore, `libvorbis` not handling this case is undesirable behavior. To better handle this case without throwing the performance benefits of `alloca` out the window, let's check whether the allocation would exceed 256 KiB (an estimate for the minimum stack space available is 1 MiB, which is [the default on Windows platforms](https://learn.microsoft.com/en-us/windows/win32/procthread/thread-stack-size)), and if so fall back to a heap allocated array. The heap array that may be allocated for this purpose is freed when `vorbis_dsp_clear` is called. `_preextrapolate_helper` takes neglible execution time compared to the encoding process for usual sample block sizes, though. Signed-off-by: Alejandro González <me@alegon.dev>
1 parent 84c0236 commit c9694a0

File tree

2 files changed

+12
-2
lines changed

2 files changed

+12
-2
lines changed

include/vorbis/codec.h

+1
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ typedef struct vorbis_dsp_state{
6161

6262
float **pcm;
6363
float **pcmret;
64+
float *preextrapolate_work;
6465
int pcm_storage;
6566
int pcm_current;
6667
int pcm_returned;

lib/block.c

+11-2
Original file line numberDiff line numberDiff line change
@@ -372,6 +372,7 @@ void vorbis_dsp_clear(vorbis_dsp_state *v){
372372
if(v->pcm[i])_ogg_free(v->pcm[i]);
373373
_ogg_free(v->pcm);
374374
if(v->pcmret)_ogg_free(v->pcmret);
375+
if(v->preextrapolate_work)_ogg_free(v->preextrapolate_work);
375376
}
376377

377378
if(b){
@@ -417,11 +418,19 @@ static void _preextrapolate_helper(vorbis_dsp_state *v){
417418
int i;
418419
int order=16;
419420
float *lpc=alloca(order*sizeof(*lpc));
420-
float *work=alloca(v->pcm_current*sizeof(*work));
421+
float *work;
422+
int workbuf=v->pcm_current*sizeof(*work);
421423
long j;
422424
v->preextrapolate=1;
423425

424-
if(v->pcm_current-v->centerW>order*2){ /* safety */
426+
if(workbuf<256*1024)
427+
work=alloca(workbuf);
428+
else
429+
/* workbuf is too big to safely allocate on the stack */
430+
v->preextrapolate_work=_ogg_realloc(v->preextrapolate_work,workbuf);
431+
work=v->preextrapolate_work;
432+
433+
if(v->pcm_current-v->centerW>order*2 && work){ /* safety */
425434
for(i=0;i<v->vi->channels;i++){
426435
/* need to run the extrapolation in reverse! */
427436
for(j=0;j<v->pcm_current;j++)

0 commit comments

Comments
 (0)