Camera Perspective does not change to 'user perspective' when rotated

[herr-edgy]

With default controls, when in orthographic view, and the user rotates the camera in some way, it changes to 'User Perspective'
However with the fly controls, it will not correctly update this.

When I front orthographic view, then moving via right click, it will change to Front Perspective while moving, then when letting go to Front Orthographic despite not being in Front View at all.

Imo it should just switch to User Perspective once the camera rotates a little and stay in User Perspective like with default controls for consistency and correctness.

Attached is a picture that is considered 'Front Orthographic' when it really isn't frontal at all. Imo it should also not change from the original Orthographic -> Perspective while moving -> Back to Orthographic again but that's more debatable.