Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Core (LV::AudioConvert): Rewrite audio sample conversion #381

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kaixiong
Copy link
Member

@kaixiong kaixiong commented Jan 18, 2025

Did extensive testing of the audio sample conversion and found a tonne of egregious numerical errors.

This PR fixes all of the known bugs. Where the conversion accuracy is sacrificed for performance, I have quantified the errors (with proofs!).

I don't know if they were introduced when I ported the original C macros to C++ generics, or during its refinements over time with modern C++-isms. If the bugs are present in the C original, I can backport the fixes over to 0.4.x.

More (gory) details to follow.

@kaixiong
Copy link
Member Author

Here is the test comparing all three implementations. The 0.5.x (master) and 0.4.x were adapted slightly to work with the test runner.

General observations

  • PR works in all cases
  • 0.5.x works incorrectly when converting between signed and unsigned integer samples involved narrowing/widening, and overflows when mapping 1.0 float values to integers.
  • 0.4.x works incorrectly in all cases except same-type conversions, largely down to careless mistakes: mixing up directionality converting to/from floats, wrong shifts (wrong units i.e. bit vs byte). It also has underflow/overflow issues with 32-bit integers.

All 49 possible conversions were tested. For each test, 8 input values spread across the input range are used. The output values are presented in the following order: PR, 0.5.x, 0.4.x.

int8_t -> uint8_t
[-128, 127] -> [0, 255]
       -128 ->           0    vs           0   vs           0
        -91 ->          37    vs          37   vs          37
        -55 ->          73    vs          73   vs          73
        -18 ->         110    vs         110   vs         110
         17 ->         145    vs         145   vs         145
         54 ->         182    vs         182   vs         182
         90 ->         218    vs         218   vs         218
        127 ->         255    vs         255   vs         255

int8_t -> int8_t
[-128, 127] -> [-128, 127]
       -128 ->        -128    vs        -128   vs        -128
        -91 ->         -91    vs         -91   vs         -91
        -55 ->         -55    vs         -55   vs         -55
        -18 ->         -18    vs         -18   vs         -18
         17 ->          17    vs          17   vs          17
         54 ->          54    vs          54   vs          54
         90 ->          90    vs          90   vs          90
        127 ->         127    vs         127   vs         127

int8_t -> uint16_t
[-128, 127] -> [0, 65535]
       -128 ->           0    vs           0   vs       33024
        -91 ->        9472    vs        9472   vs       33098
        -55 ->       18688    vs       18688   vs       33170
        -18 ->       28160    vs       28160   vs       33244
         17 ->       37120    vs       37120   vs       32802
         54 ->       46592    vs       46592   vs       32876
         90 ->       55808    vs       55808   vs       32948
        127 ->       65280    vs       65280   vs       33022

int8_t -> int16_t
[-128, 127] -> [-32768, 32767]
       -128 ->      -32768    vs      -32768   vs         256
        -91 ->      -23296    vs      -23296   vs         330
        -55 ->      -14080    vs      -14080   vs         402
        -18 ->       -4608    vs       -4608   vs         476
         17 ->        4352    vs        4352   vs          34
         54 ->       13824    vs       13824   vs         108
         90 ->       23040    vs       23040   vs         180
        127 ->       32512    vs       32512   vs         254

int8_t -> uint32_t
[-128, 127] -> [0, 4294967295]
       -128 ->           0    vs           0   vs        1024
        -91 ->   620756992    vs   620756992   vs        1320
        -55 ->  1224736768    vs  1224736768   vs        1608
        -18 ->  1845493760    vs  1845493760   vs        1904
         17 ->  2432696320    vs  2432696320   vs         136
         54 ->  3053453312    vs  3053453312   vs         432
         90 ->  3657433088    vs  3657433088   vs         720
        127 ->  4278190080    vs  4278190080   vs        1016

int8_t -> int32_t
[-128, 127] -> [-2147483648, 2147483647]
       -128 -> -2147483648    vs -2147483648   vs        1024
        -91 -> -1526726656    vs -1526726656   vs        1320
        -55 ->  -922746880    vs  -922746880   vs        1608
        -18 ->  -301989888    vs  -301989888   vs        1904
         17 ->   285212672    vs   285212672   vs         136
         54 ->   905969664    vs   905969664   vs         432
         90 ->  1509949440    vs  1509949440   vs         720
        127 ->  2130706432    vs  2130706432   vs        1016

int8_t -> float
[-128, 127] -> [-1, 1]
       -128 ->          -1    vs   -1.007874   vs           0
        -91 -> -0.70980394    vs -0.71653545   vs           0
        -55 -> -0.42745095    vs -0.43307087   vs           0
        -18 -> -0.13725483    vs -0.14173228   vs           0
         17 ->  0.13725495    vs  0.13385826   vs           0
         54 ->    0.427451    vs  0.42519686   vs           0
         90 ->  0.70980406    vs  0.70866144   vs 5.877472e-39
        127 ->           1    vs           1   vs           0

uint8_t -> uint8_t
[0, 255] -> [0, 255]
          0 ->           0    vs           0   vs           0
         36 ->          36    vs          36   vs          36
         72 ->          72    vs          72   vs          72
        109 ->         109    vs         109   vs         109
        145 ->         145    vs         145   vs         145
        182 ->         182    vs         182   vs         182
        218 ->         218    vs         218   vs         218
        255 ->         255    vs         255   vs         255

uint8_t -> int8_t
[0, 255] -> [-128, 127]
          0 ->        -128    vs        -128   vs        -128
         36 ->         -92    vs         -92   vs         -92
         72 ->         -56    vs         -56   vs         -56
        109 ->         -19    vs         -19   vs         -19
        145 ->          17    vs          17   vs          17
        182 ->          54    vs          54   vs          54
        218 ->          90    vs          90   vs          90
        255 ->         127    vs         127   vs         127

uint8_t -> uint16_t
[0, 255] -> [0, 65535]
          0 ->           0    vs           0   vs           0
         36 ->        9216    vs        9216   vs          72
         72 ->       18432    vs       18432   vs         144
        109 ->       27904    vs       27904   vs         218
        145 ->       37120    vs       37120   vs         290
        182 ->       46592    vs       46592   vs         364
        218 ->       55808    vs       55808   vs         436
        255 ->       65280    vs       65280   vs         510

uint8_t -> int16_t
[0, 255] -> [-32768, 32767]
          0 ->      -32768    vs           0   vs        -128
         36 ->      -23552    vs        9216   vs         -56
         72 ->      -14336    vs       18432   vs          16
        109 ->       -4864    vs       27904   vs          90
        145 ->        4352    vs      -28416   vs        -350
        182 ->       13824    vs      -18944   vs        -276
        218 ->       23040    vs       -9728   vs        -204
        255 ->       32512    vs        -256   vs        -130

uint8_t -> uint32_t
[0, 255] -> [0, 4294967295]
          0 ->           0    vs           0   vs           0
         36 ->   603979776    vs   603979776   vs         288
         72 ->  1207959552    vs  1207959552   vs         576
        109 ->  1828716544    vs  1828716544   vs         872
        145 ->  2432696320    vs  2432696320   vs        1160
        182 ->  3053453312    vs  3053453312   vs        1456
        218 ->  3657433088    vs  3657433088   vs        1744
        255 ->  4278190080    vs  4278190080   vs        2040

uint8_t -> int32_t
[0, 255] -> [-2147483648, 2147483647]
          0 -> -2147483648    vs           0   vs        -128
         36 -> -1543503872    vs   603979776   vs         160
         72 ->  -939524096    vs  1207959552   vs         448
        109 ->  -318767104    vs  1828716544   vs         744
        145 ->   285212672    vs -1862270976   vs       -1016
        182 ->   905969664    vs -1241513984   vs        -720
        218 ->  1509949440    vs  -637534208   vs        -432
        255 ->  2130706432    vs   -16777216   vs        -136

uint8_t -> float
[0, 255] -> [-1, 1]
          0 ->          -1    vs          -1   vs -1.1754944e-38
         36 ->  -0.7176471    vs    -0.71875   vs -1.180104e-38
         72 ->  -0.4352941    vs     -0.4375   vs -1.1754944e-38
        109 -> -0.14509797    vs  -0.1484375   vs -1.180104e-38
        145 ->  0.13725495    vs   0.1328125   vs 1.1663287e-38
        182 ->    0.427451    vs    0.421875   vs -1.180104e-38
        218 ->  0.70980406    vs    0.703125   vs 1.1755123e-38
        255 ->           1    vs   0.9921875   vs -1.180104e-38

int16_t -> int8_t
[-32768, 32767] -> [-128, 127]
     -32768 ->        -128    vs        -128   vs           0
     -23405 ->         -92    vs         -92   vs          73
     -14043 ->         -55    vs         -55   vs        -110
      -4681 ->         -19    vs         -19   vs         -37
       4680 ->          18    vs          18   vs          36
      14042 ->          54    vs          54   vs         109
      23404 ->          91    vs          91   vs         -74
      32767 ->         127    vs         127   vs          -1

int16_t -> uint8_t
[-32768, 32767] -> [0, 255]
     -32768 ->           0    vs           0   vs         128
     -23405 ->          36    vs          36   vs         201
     -14043 ->          73    vs          73   vs          18
      -4681 ->         109    vs         109   vs          91
       4680 ->         146    vs         146   vs         164
      14042 ->         182    vs         182   vs         237
      23404 ->         219    vs         219   vs          54
      32767 ->         255    vs         255   vs         127

int16_t -> uint16_t
[-32768, 32767] -> [0, 65535]
     -32768 ->           0    vs           0   vs           0
     -23405 ->        9363    vs        9363   vs        9363
     -14043 ->       18725    vs       18725   vs       18725
      -4681 ->       28087    vs       28087   vs       28087
       4680 ->       37448    vs       37448   vs       37448
      14042 ->       46810    vs       46810   vs       46810
      23404 ->       56172    vs       56172   vs       56172
      32767 ->       65535    vs       65535   vs       65535

int16_t -> int16_t
[-32768, 32767] -> [-32768, 32767]
     -32768 ->      -32768    vs      -32768   vs      -32768
     -23405 ->      -23405    vs      -23405   vs      -23405
     -14043 ->      -14043    vs      -14043   vs      -14043
      -4681 ->       -4681    vs       -4681   vs       -4681
       4680 ->        4680    vs        4680   vs        4680
      14042 ->       14042    vs       14042   vs       14042
      23404 ->       23404    vs       23404   vs       23404
      32767 ->       32767    vs       32767   vs       32767

int16_t -> uint32_t
[-32768, 32767] -> [0, 4294967295]
     -32768 ->           0    vs           0   vs      131072
     -23405 ->   613613568    vs   613613568   vs      168524
     -14043 ->  1227161600    vs  1227161600   vs      205972
      -4681 ->  1840709632    vs  1840709632   vs      243420
       4680 ->  2454192128    vs  2454192128   vs       18720
      14042 ->  3067740160    vs  3067740160   vs       56168
      23404 ->  3681288192    vs  3681288192   vs       93616
      32767 ->  4294901760    vs  4294901760   vs      131068

int16_t -> int32_t
[-32768, 32767] -> [-2147483648, 2147483647]
     -32768 -> -2147483648    vs -2147483648   vs      131072
     -23405 -> -1533870080    vs -1533870080   vs      168524
     -14043 ->  -920322048    vs  -920322048   vs      205972
      -4681 ->  -306774016    vs  -306774016   vs      243420
       4680 ->   306708480    vs   306708480   vs       18720
      14042 ->   920256512    vs   920256512   vs       56168
      23404 ->  1533804544    vs  1533804544   vs       93616
      32767 ->  2147418112    vs  2147418112   vs      131068

int16_t -> float
[-32768, 32767] -> [-1, 1]
     -32768 ->          -1    vs  -1.0000305   vs           0
     -23405 -> -0.71425956    vs -0.71428573   vs           0
     -14043 -> -0.42854965    vs -0.42857143   vs           0
      -4681 -> -0.14283973    vs -0.14285715   vs           0
       4680 ->  0.14283967    vs  0.14282662   vs           0
      14042 ->  0.42854965    vs  0.42854092   vs           0
      23404 ->   0.7142595    vs   0.7142552   vs           0
      32767 ->           1    vs           1   vs           0

uint16_t -> uint8_t
[0, 65535] -> [0, 255]
          0 ->           0    vs           0   vs           0
       9362 ->          36    vs          36   vs          73
      18724 ->          73    vs          73   vs         146
      28086 ->         109    vs         109   vs         219
      37448 ->         146    vs         146   vs          36
      46810 ->         182    vs         182   vs         109
      56172 ->         219    vs         219   vs         182
      65535 ->         255    vs         255   vs         255

uint16_t -> int8_t
[0, 65535] -> [-128, 127]
          0 ->        -128    vs           0   vs           0
       9362 ->         -92    vs          36   vs          73
      18724 ->         -55    vs          73   vs        -110
      28086 ->         -19    vs         109   vs         -37
      37448 ->          18    vs        -110   vs          36
      46810 ->          54    vs         -74   vs         109
      56172 ->          91    vs         -37   vs         -74
      65535 ->         127    vs          -1   vs          -1

uint16_t -> uint16_t
[0, 65535] -> [0, 65535]
          0 ->           0    vs           0   vs           0
       9362 ->        9362    vs        9362   vs        9362
      18724 ->       18724    vs       18724   vs       18724
      28086 ->       28086    vs       28086   vs       28086
      37448 ->       37448    vs       37448   vs       37448
      46810 ->       46810    vs       46810   vs       46810
      56172 ->       56172    vs       56172   vs       56172
      65535 ->       65535    vs       65535   vs       65535

uint16_t -> int16_t
[0, 65535] -> [-32768, 32767]
          0 ->      -32768    vs      -32768   vs      -32768
       9362 ->      -23406    vs      -23406   vs      -23406
      18724 ->      -14044    vs      -14044   vs      -14044
      28086 ->       -4682    vs       -4682   vs       -4682
      37448 ->        4680    vs        4680   vs        4680
      46810 ->       14042    vs       14042   vs       14042
      56172 ->       23404    vs       23404   vs       23404
      65535 ->       32767    vs       32767   vs       32767

uint16_t -> uint32_t
[0, 65535] -> [0, 4294967295]
          0 ->           0    vs           0   vs           0
       9362 ->   613548032    vs   613548032   vs       37448
      18724 ->  1227096064    vs  1227096064   vs       74896
      28086 ->  1840644096    vs  1840644096   vs      112344
      37448 ->  2454192128    vs  2454192128   vs      149792
      46810 ->  3067740160    vs  3067740160   vs      187240
      56172 ->  3681288192    vs  3681288192   vs      224688
      65535 ->  4294901760    vs  4294901760   vs      262140

uint16_t -> int32_t
[0, 65535] -> [-2147483648, 2147483647]
          0 -> -2147483648    vs           0   vs      -32768
       9362 -> -1533935616    vs   613548032   vs        4680
      18724 ->  -920387584    vs  1227096064   vs       42128
      28086 ->  -306839552    vs  1840644096   vs       79576
      37448 ->   306708480    vs -1840775168   vs     -145120
      46810 ->   920256512    vs -1227227136   vs     -107672
      56172 ->  1533804544    vs  -613679104   vs      -70224
      65535 ->  2147418112    vs      -65536   vs      -32772

uint16_t -> float
[0, 65535] -> [-1, 1]
          0 ->          -1    vs          -1   vs  4.5918e-41
       9362 ->  -0.7142901    vs -0.71429443   vs  4.5916e-41
      18724 -> -0.42858016    vs -0.42858887   vs -4.5918e-41
      28086 -> -0.14287025    vs  -0.1428833   vs -4.5918e-41
      37448 ->  0.14283967    vs  0.14282227   vs  4.5918e-41
      46810 ->  0.42854965    vs  0.42852783   vs  4.5918e-41
      56172 ->   0.7142595    vs   0.7142334   vs -4.5918e-41
      65535 ->           1    vs   0.9999695   vs -4.5918e-41

int32_t -> uint8_t
[-2147483648, 2147483647] -> [0, 255]
-2147483648 ->           0    vs           0   vs         128
-1533916891 ->          36    vs          36   vs         164
 -920350135 ->          73    vs          73   vs         201
 -306783378 ->         109    vs         109   vs         237
  306783377 ->         146    vs         146   vs          18
  920350134 ->         182    vs         182   vs          54
 1533916890 ->         219    vs         219   vs          91
 2147483647 ->         255    vs         255   vs         127

int32_t -> int8_t
[-2147483648, 2147483647] -> [-128, 127]
-2147483648 ->        -128    vs        -128   vs           0
-1533916891 ->         -92    vs         -92   vs          36
 -920350135 ->         -55    vs         -55   vs          73
 -306783378 ->         -19    vs         -19   vs         109
  306783377 ->          18    vs          18   vs        -110
  920350134 ->          54    vs          54   vs         -74
 1533916890 ->          91    vs          91   vs         -37
 2147483647 ->         127    vs         127   vs          -1

int32_t -> uint16_t
[-2147483648, 2147483647] -> [0, 65535]
-2147483648 ->           0    vs           0   vs       32768
-1533916891 ->        9362    vs        9362   vs        4681
 -920350135 ->       18724    vs       18724   vs       42130
 -306783378 ->       28086    vs       28086   vs       14043
  306783377 ->       37449    vs       37449   vs       51492
  920350134 ->       46811    vs       46811   vs       23405
 1533916890 ->       56173    vs       56173   vs       60854
 2147483647 ->       65535    vs       65535   vs       32767

int32_t -> int16_t
[-2147483648, 2147483647] -> [-32768, 32767]
-2147483648 ->      -32768    vs      -32768   vs           0
-1533916891 ->      -23406    vs      -23406   vs      -28087
 -920350135 ->      -14044    vs      -14044   vs        9362
 -306783378 ->       -4682    vs       -4682   vs      -18725
  306783377 ->        4681    vs        4681   vs       18724
  920350134 ->       14043    vs       14043   vs       -9363
 1533916890 ->       23405    vs       23405   vs       28086
 2147483647 ->       32767    vs       32767   vs          -1

int32_t -> uint32_t
[-2147483648, 2147483647] -> [0, 4294967295]
-2147483648 ->           0    vs           0   vs  2147483648
-1533916891 ->   613566757    vs   613566757   vs  2761050405
 -920350135 ->  1227133513    vs  1227133513   vs  3374617161
 -306783378 ->  1840700270    vs  1840700270   vs  3988183918
  306783377 ->  2454267025    vs  2454267025   vs   306783377
  920350134 ->  3067833782    vs  3067833782   vs   920350134
 1533916890 ->  3681400538    vs  3681400538   vs  1533916890
 2147483647 ->  4294967295    vs  4294967295   vs  2147483647

int32_t -> int32_t
[-2147483648, 2147483647] -> [-2147483648, 2147483647]
-2147483648 -> -2147483648    vs -2147483648   vs -2147483648
-1533916891 -> -1533916891    vs -1533916891   vs -1533916891
 -920350135 ->  -920350135    vs  -920350135   vs  -920350135
 -306783378 ->  -306783378    vs  -306783378   vs  -306783378
  306783377 ->   306783377    vs   306783377   vs   306783377
  920350134 ->   920350134    vs   920350134   vs   920350134
 1533916890 ->  1533916890    vs  1533916890   vs  1533916890
 2147483647 ->  2147483647    vs  2147483647   vs  2147483647

int32_t -> float
[-2147483648, 2147483647] -> [-1, 1]
-2147483648 ->          -1    vs          -1   vs           0
-1533916891 -> -0.71428573    vs -0.71428573   vs           0
 -920350135 -> -0.42857143    vs -0.42857143   vs           0
 -306783378 -> -0.14285715    vs -0.14285715   vs           0
  306783377 ->  0.14285715    vs  0.14285715   vs           0
  920350134 ->  0.42857143    vs  0.42857143   vs           0
 1533916890 ->  0.71428573    vs  0.71428573   vs           0
 2147483647 ->           1    vs           1   vs          -0

uint32_t -> uint8_t
[0, 4294967295] -> [0, 255]
          0 ->           0    vs           0   vs           0
  613566756 ->          36    vs          36   vs          36
 1227133512 ->          73    vs          73   vs          73
 1840700269 ->         109    vs         109   vs         109
 2454267025 ->         146    vs         146   vs         146
 3067833782 ->         182    vs         182   vs         182
 3681400538 ->         219    vs         219   vs         219
 4294967295 ->         255    vs         255   vs         255

uint32_t -> int8_t
[0, 4294967295] -> [-128, 127]
          0 ->        -128    vs           0   vs           0
  613566756 ->         -92    vs          36   vs          36
 1227133512 ->         -55    vs          73   vs          73
 1840700269 ->         -19    vs         109   vs         109
 2454267025 ->          18    vs        -110   vs        -110
 3067833782 ->          54    vs         -74   vs         -74
 3681400538 ->          91    vs         -37   vs         -37
 4294967295 ->         127    vs          -1   vs          -1

uint32_t -> uint16_t
[0, 4294967295] -> [0, 65535]
          0 ->           0    vs           0   vs           0
  613566756 ->        9362    vs        9362   vs       37449
 1227133512 ->       18724    vs       18724   vs        9362
 1840700269 ->       28086    vs       28086   vs       46811
 2454267025 ->       37449    vs       37449   vs       18724
 3067833782 ->       46811    vs       46811   vs       56173
 3681400538 ->       56173    vs       56173   vs       28086
 4294967295 ->       65535    vs       65535   vs       65535

uint32_t -> int16_t
[0, 4294967295] -> [-32768, 32767]
          0 ->      -32768    vs           0   vs           0
  613566756 ->      -23406    vs        9362   vs      -28087
 1227133512 ->      -14044    vs       18724   vs        9362
 1840700269 ->       -4682    vs       28086   vs      -18725
 2454267025 ->        4681    vs      -28087   vs       18724
 3067833782 ->       14043    vs      -18725   vs       -9363
 3681400538 ->       23405    vs       -9363   vs       28086
 4294967295 ->       32767    vs          -1   vs          -1

uint32_t -> uint32_t
[0, 4294967295] -> [0, 4294967295]
          0 ->           0    vs           0   vs           0
  613566756 ->   613566756    vs   613566756   vs   613566756
 1227133512 ->  1227133512    vs  1227133512   vs  1227133512
 1840700269 ->  1840700269    vs  1840700269   vs  1840700269
 2454267025 ->  2454267025    vs  2454267025   vs  2454267025
 3067833782 ->  3067833782    vs  3067833782   vs  3067833782
 3681400538 ->  3681400538    vs  3681400538   vs  3681400538
 4294967295 ->  4294967295    vs  4294967295   vs  4294967295

uint32_t -> int32_t
[0, 4294967295] -> [-2147483648, 2147483647]
          0 -> -2147483648    vs -2147483648   vs           0
  613566756 -> -1533916892    vs -1533916892   vs   613566756
 1227133512 ->  -920350136    vs  -920350136   vs  1227133512
 1840700269 ->  -306783379    vs  -306783379   vs  1840700269
 2454267025 ->   306783377    vs   306783377   vs -1840700271
 3067833782 ->   920350134    vs   920350134   vs -1227133514
 3681400538 ->  1533916890    vs  1533916890   vs  -613566758
 4294967295 ->  2147483647    vs  2147483647   vs          -1

uint32_t -> float
[0, 4294967295] -> [-1, 1]
          0 ->          -1    vs          -1   vs           0
  613566756 -> -0.71428573    vs -0.71428573   vs           0
 1227133512 -> -0.42857143    vs  -0.4285714   vs           0
 1840700269 -> -0.14285715    vs -0.14285713   vs           0
 2454267025 ->  0.14285715    vs   0.1428572   vs           0
 3067833782 ->  0.42857143    vs  0.42857146   vs           0
 3681400538 ->  0.71428573    vs  0.71428573   vs           0
 4294967295 ->           1    vs           1   vs           0

float -> uint8_t
[-1, 1] -> [0, 255]
         -1 ->           0    vs           0   vs           0
-0.71428573 ->          36    vs          36   vs           0
-0.42857143 ->          72    vs          73   vs         128
-0.14285715 ->         109    vs         109   vs         191
 0.14285715 ->         145    vs         146   vs           0
 0.42857143 ->         182    vs         182   vs           0
 0.71428573 ->         218    vs         219   vs         128
          1 ->         255    vs           0   vs         191

float -> int8_t
[-1, 1] -> [-128, 127]
         -1 ->        -128    vs        -127   vs           0
-0.71428573 ->         -92    vs         -90   vs           0
-0.42857143 ->         -56    vs         -54   vs           0
-0.14285715 ->         -19    vs         -18   vs           0
 0.14285715 ->          17    vs          18   vs           0
 0.42857143 ->          54    vs          54   vs           0
 0.71428573 ->          90    vs          90   vs           0
          1 ->         127    vs         127   vs           0

float -> uint16_t
[-1, 1] -> [0, 65535]
         -1 ->           0    vs           0   vs           0
-0.71428573 ->        9362    vs        9362   vs       49024
-0.42857143 ->       18724    vs       18724   vs           0
-0.14285715 ->       28086    vs       28086   vs       16126
 0.14285715 ->       37448    vs       37449   vs       56320
 0.42857143 ->       46810    vs       46811   vs       16182
 0.71428573 ->       56172    vs       56173   vs       55296
          1 ->       65535    vs           0   vs       16124

float -> int16_t
[-1, 1] -> [-32768, 32767]
         -1 ->      -32768    vs      -32767   vs           0
-0.71428573 ->      -23406    vs      -23405   vs           0
-0.42857143 ->      -14044    vs      -14043   vs           0
-0.14285715 ->       -4682    vs       -4681   vs      -16639
 0.14285715 ->        4680    vs        4681   vs       18432
 0.42857143 ->       14042    vs       14043   vs      -16750
 0.71428573 ->       23404    vs       23405   vs      -27648
          1 ->       32767    vs       32767   vs      -16639

float -> int32_t
[-1, 1] -> [-2147483648, 2147483647]
         -1 -> -2147483648    vs -2147483648   vs    -8388608
-0.71428573 -> -1533916929    vs -1533916928   vs    -8388608
-0.42857143 ->  -920350145    vs  -920350144   vs    -8388608
-0.14285715 ->  -306783393    vs  -306783392   vs    -8388608
 0.14285715 ->   306783391    vs   306783392   vs  2139095040
 0.42857143 ->   920350143    vs   920350144   vs  2139095040
 0.71428573 ->  1533916927    vs  1533916928   vs  2139095040
          1 ->  2147483647    vs -2147483648   vs  2139095040

float -> uint32_t
[-1, 1] -> [0, 4294967295]
         -1 ->           0    vs           0   vs  2139095040
-0.71428573 ->   613566719    vs   613566720   vs  2139095040
-0.42857143 ->  1227133503    vs  1227133440   vs  2139095040
-0.14285715 ->  1840700255    vs  1840700288   vs  2139095040
 0.14285715 ->  2454267039    vs  2454267136   vs  2139095040
 0.42857143 ->  3067833791    vs  3067833856   vs  2139095040
 0.71428573 ->  3681400575    vs  3681400576   vs  2139095040
          1 ->  4294967295    vs           0   vs  2139095040

float -> float
[-1, 1] -> [-1, 1]
         -1 ->          -1    vs          -1   vs           0
-0.71428573 -> -0.71428573    vs -0.71428573   vs           0
-0.42857143 -> -0.42857143    vs -0.42857143   vs           0
-0.14285715 -> -0.14285715    vs -0.14285715   vs           0
 0.14285715 ->  0.14285715    vs  0.14285715   vs           0
 0.42857143 ->  0.42857143    vs  0.42857143   vs           0
 0.71428573 ->  0.71428573    vs  0.71428573   vs           0
          1 ->           1    vs           1   vs           0

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants