scaling a font's UPEM with fontMath

[anthrotype]

I would like to scale a UFO's unitsPerEm, while keeping the font the same by scaling everything else at the same time.
I thought I could simply use fontMath since it already implements __mul__ on info, kerning and glyph objects.
Here's what I have:

https://gist.github.com/anthrotype/62d0bfe1d38b8f11a199bd3b66574bcc

Now the problem is the components' xScale and yScale are also scaled by the given factor, which is not what I was expecting. Say they are set to 1.0 in the input font, if I multiply the glyphs by a factor or 2, I get the component scale set to 2.0. The simple contour glyphs have also been scaled by the same amount already, I don't want the components to be scaled again.

I was thinking we may add an option in MathGlyph constructor to only scale the component's offsets and exclude the other scale/shear attributes?