exclusive class note cutting is too slow in most cases

[mrbumpy409]

FluidSynth version

2.4.2

Bug Description

Exclusive class is most often used in percussion banks to have one percussion sound muted by another. For example, the closed or pedal hi-hat will stop the open hi-hat from ringing, muted triangle will stop the open triangle, etc. With most SoundFont synths, the cutting of the previous voice is quite abrupt. FluidSynth softens this cutoff by reducing the volume and modulation envelope release times and killing the voice. This is actually quite a good idea. However, FluidSynth's cut time is still too long in most cases, leading to the supposedly cut voice being audible for far too long after it should have been silenced.

The code in question can be found in /src/synth/fluid_voice.c lines 1401–1408. Here, both volume and modulation envelope release times are reduced by 200, though I'm not sure how this translates into time or percentage. I have found that reducing the volume envelope release by 2000 and modulation envelope release by 1000 instead results in more natural-sounding note cuts, which can be clearly heard in the examples below.

Test Files and Audio

Here is a zip file containing the following files:

• exclusive class cutoff speed -200.ogg – the test MIDI played using stock FluidSynth 2.4.2, which has the volume and modulation envelope release times -200 on exclusive class note cutoff.
• exclusive class cutoff speed -2000.ogg – the test MIDI played using modified FluidSynth 2.4.2, which has the volume envelope release time -2000 and modulation envelope release time -1000 for exclusive class note cutoff.
• exclusive class cutoff speed.mid – the MIDI file used to generate the above audio files.
• exclusive class cutoff speed test.sf2 – the SoundFont file to be loaded into bank 0 when playing the MIDI file. Note that the test uses a melodic preset instead of percussive preset (channel 10) to work around this issue, which would otherwise interfere with the test.

The test plays the following GM drum kit exclusive class samples, each repeated a few times:

1. Open hi-hat muted by closed hi-hat.
2. Open hi-hat muted by hi-hat pedal.
3. Open triangle muted by muted triangle.
4. Two alternating record scratch samples.
5. Long whistle muted by short whistle
6. Long guiro muted by short guiro
7. Crash cymbal 1 muted by itself using velocity = 1.

Test Results

For each of the above percussion sounds, the exclusive class muting with "-200" is too slow. The open hat is still ringing a bit when it should be closed. The open triangle can still be heard when it should be muted. Perhaps the most obvious is the whistle, which has a huge overlap between the samples. In every case, the "-2000" setting is more natural, and never sounds too abrupt.

Now, since the cut time is determined by a reduction of the release envelopes, there will be some variance from sample to sample, depending on how it was programmed. However, I hope I have sufficiently demonstrated the release times expected in cases where (A) a cymbal is programmed to a natural decay length, and (B) where the release time is set to maximum for non-decaying samples such as the whistle or guiro.

The only downside I can think of to this change is that percussion sounds that are programmed with short release times will have a more abrupt note cutoff. However, the worst-case scenario is that these percussion sounds might sound the same as other SoundFont players when cut off—which is a perfectly acceptable outcome, if you ask me.

Additional context

The reason I've only proposed reducing the modulation envelope by 1000 instead of 2000 is due to my experience in SoundFont design. I find that the modulation envelope release should usually be set longer than the volume envelope release for the most natural decay.

[derselbst]

I'm all in favor to change that, esp. since the spec speaks about "rapidly terminated". But let me raise a few more questions.

The spec speaks about note "termination" and not about noteoff. But fluidsynth currently uses noteoff to "terminate" the voice. Is that correct or should we better use fluid_voice_off, which forces the voice into finished state? I'd suggest that I add your demo files to our test suite, then make an alternative proposal which uses fluid_voice_off, post the renderings for both proposals and we'll hear how they sounds.

Also, the spec says:

The scope of the exclusive class is the entire preset. In other words, any other instrument zone within the same preset holding a corresponding exclusive class will be terminated.

But as far as I see, fluidsynth doesn't account for that preset scope at the moment, i.e. potentially killing voices spawned by other presets.

Here, both volume and modulation envelope release times are reduced by 200, though I'm not sure how this translates into time or percentage

It's not reduced by - it's rather set absolutely. Both generators are in absolute timecents. The relationship between timecents and time is:
time = 2 ^ (cents / 1200)

yielding about 30ms and 56ms for your proposal.

[mrbumpy409]

The spec speaks about note "termination" and not about noteoff. But fluidsynth currently uses noteoff to "terminate" the voice. Is that correct or should we better use fluid_voice_off, which forces the voice into finished state?

My opinion is that FluidSynth's method sounds far better, especially with my proposed envelope values. Using voice termination (as is done in other SoundFont synths) often sounds unnaturally abrupt, e.g., the transition from open hat to closed hat isn't as seamless and doesn't sound acoustically accurate.

I'd suggest that I add your demo files to our test suite, then make an alternative proposal which uses fluid_voice_off, post the renderings for both proposals and we'll hear how they sounds.

I would be happy to render the test on my Audigy2 for comparison, so you can hear what voice termination sounds like. That would save you the effort of coding it into FluidSynth. I will make a recording when I get back from lunch.

Also, the spec says:

The scope of the exclusive class is the entire preset. In other words, any other instrument zone within the same preset holding a corresponding exclusive class will be terminated.

But as far as I see, fluidsynth doesn't account for that preset scope at the moment, i.e. potentially killing voices spawned by other presets.

So if you are saying that an exclusive class = 1 voice on channel 1 could terminate an exclusive class = 1 voice on channel 2, then yes, that isn't supposed to happen. However, since exclusive class isn't really used much outside of percussion banks, it's possible this situation has never been encountered.

It's not reduced by - it's rather set absolutely. Both generators are in absolute timecents. The relationship between timecents and time is: time = 2 ^ (cents / 1200)

yielding about 30ms and 56ms for your proposal.

Aah, then my theory about longer release times leading to longer cut times can be thrown out the window, and you can ignore my statement about the potential downside of this change. As such, this is even better!

[derselbst]

It was actually quite simple, here is the voiceoff rendering. The whistle has a tendency to stop with a crack. The rest also sounds quite unnatural I'd say. Feel free to post the Audigy rendering if you want to :)

exclusive class cutoff speed_voiceoff.zip

So if you are saying that an exclusive class = 1 voice on channel 1 could terminate an exclusive class = 1 voice on channel 2, then yes, that isn't supposed to happen

Right, we do check for a matching channel, and the preset is bound to that channel, obviously. Scratch that, it's all good.

[derselbst]

I have found that reducing the volume envelope release by 2000 and modulation envelope release by 1000 instead results in more natural-sounding note cuts

Btw, since it's set absolutely, is it still reasonable to have different durations for those generators?

[mrbumpy409]

Btw, since it's set absolutely, is it still reasonable to have different durations for those generators?

I think so, yes. An argument could be made for removing the modulation envelope release cutting altogether, since cutting the modulation envelope release time assumes that the modulation envelope is being used to filter a sound gradually as it decays. However, what if it is being used to apply a slow pitch bend instead? We probably don't want the pitch suddenly whipping down or up when the note is cut.

I'm guessing that removing the modulation envelope cut would be done by just removing the following lines:

    /* Speed up the modulation envelope */
    fluid_voice_gen_set(voice, GEN_MODENVRELEASE, -1000);
    fluid_voice_update_param(voice, GEN_MODENVRELEASE);

Removing this will have a small—probably unnoticeable—affect on the brightness of the sound when muting hi-hat and cymbal in the test files I shared, but will also avoid any unintended consequences that might be lurking in a SoundFont somewhere out there.

[derselbst]

Ok. I'd vote for removing it. That's correct, feel free to update your PR. I would merge it, after a final rehearsal.

[mrbumpy409]

I have updated the PR.