Metal to OpenGL interop is 8-bits only

[jerry-huxtable-foundry]

Description of Issue

hdStorm cannot display extended dynamic range content on macOS. This is due to the Metal/GL interop code in metal.mm which uses an 8 bit texture.

Steps to Reproduce

1. This is difficult to reproduce as it requires a host application with a floating point OpenGL framebuffer and EDR enabled. usdview will not display EDR content because it only creates a 8 bit framebuffer. It may be possible to modify usdview to do this. The problem was discovered using Nuke, which uses a half-float framebuffer and calls code like this to enable EDR:

    CAMetalLayer *metalLayer = (CAMetalLayer *)view.layer;
    metalLayer.wantsExtendedDynamicRangeContent = YES;
  
    const CFStringRef colorspaceName = kCGColorSpaceExtendedSRGB;
    CGColorSpaceRef colorspace = CGColorSpaceCreateWithName(colorspaceName);
    metalLayer.colorspace = colorspace;
    CGColorSpaceRelease(colorspace);

2. Assuming you have such a host application, then displaying the USD file below will show the issue. This has a material with emissiveColor set to 10, which on an EDR display should come out very bright. What happens is that you'll just get plain old red of value 1.

The cause of the issue is the code in imaging/hgiInterop/metal.mm which creates an 8 bit path for the interop. Changing these two pieces of code to use half-floats fixes it. Of course, you might only want to do this if the GL framebuffer is deeper than 8 bits.

    CVPixelBufferCreate(
        kCFAllocatorDefault,
        width,
        height,
        **kCVPixelFormatType_64RGBAHalf**,
        (__bridge CFDictionaryRef)cvBufferProperties,
        &_pixelBuffer);

and

  cvret = CVMetalTextureCacheCreateTextureFromImage(
       kCFAllocatorDefault,
       _cvmtlTextureCache,
       _pixelBuffer,
       (__bridge CFDictionaryRef)metalTextureProperties,
       **MTLPixelFormatRGBA16Float**,
       width,
       height,
       0,
       &_cvmtlColorTexture);

Note also that in order to display EDR, you may need this code (which we have in Nuke):

  glClampColor(GL_CLAMP_VERTEX_COLOR, GL_FALSE);
  glClampColor(GL_CLAMP_FRAGMENT_COLOR, GL_FALSE);

USD file to illustrate the issue:

#usda 1.0
def Material "Red"
{
    token outputs:surface.connect      = </Red/Surface.outputs:surface>

    def Shader "Surface"
    {
        uniform token info:id = "UsdPreviewSurface"
        token outputs:surface
        color3f  inputs:emissiveColor        = (10, 0, 0)
    }
}

def Cube "cube" (
    apiSchemas = ["MaterialBindingAPI"]
)
{
    rel material:binding = </Red>
}

System Information (OS, Hardware)

macOS, any version

Package Versions

Any USD up to and including 23.05

Build Flags

-imaging -usdview

[jesschimein]

Filed as internal issue #USD-8411