Qt Canvas3D: Calling gl.colorMask or gl.depthMask, even with gl.TRUE for all args, breaks all further rendering

Stefan Monov76

I've distilled this to a completely trivial testcase. First I setup all low-level stuff like shaders, vertex buffers and matrices. Then I enable all four arguments of gl.colorMask. Then I draw a white triangle. The results is a window that alternates between all black and white (flickers) at a high frequency. I.e. completely buggy.

Removing the gl.colorMask call fixes it, and, in my real app, lets all other GL functionality work fine.

Side note: the described result is from my minimal testcase. In my real app calling gl.colorMask breaks all further rendering too, but does not cause flicker - instead, the entire Canvas3D becomes transparent (there I have background item behind the Canvas3D, and I clear the GL context to a transparent color, unlike in this testcase, in which I clear to black). The items outside the Canvas3D in my real app are not broken by the gl.colorMask call - its influence is limited within the Canvas3D.

Platforms note: This has been reproduced on:

Windows 7 with an Intel GPU
Ubuntu Linux with an nVidia GPU
custom built Linux with an ARM Mali GPU

My code:

I based my testcase on the textureandlight sample app bundled with Qt. Since mine is a derived work, I have to first post the license terms here:

/****************************************************************************
**
** Copyright (C) 2016 The Qt Company Ltd.
** Contact: https://www.qt.io/licensing/
**
** This file is part of the QtCanvas3D module of the Qt Toolkit.
**
** $QT_BEGIN_LICENSE:BSD$
** Commercial License Usage
** Licensees holding valid commercial Qt licenses may use this file in
** accordance with the commercial license agreement provided with the
** Software or, alternatively, in accordance with the terms contained in
** a written agreement between you and The Qt Company. For licensing terms
** and conditions see https://www.qt.io/terms-conditions. For further
** information use the contact form at https://www.qt.io/contact-us.
**
** BSD License Usage
** Alternatively, you may use this file under the terms of the BSD license
** as follows:
**
** "Redistribution and use in source and binary forms, with or without
** modification, are permitted provided that the following conditions are
** met:
**   * Redistributions of source code must retain the above copyright
**     notice, this list of conditions and the following disclaimer.
**   * Redistributions in binary form must reproduce the above copyright
**     notice, this list of conditions and the following disclaimer in
**     the documentation and/or other materials provided with the
**     distribution.
**   * Neither the name of The Qt Company Ltd nor the names of its
**     contributors may be used to endorse or promote products derived
**     from this software without specific prior written permission.
**
**
** THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
** "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
** LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
** A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
** OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
** SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
** LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
** DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
** THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
** (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
** OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE."
**
** $QT_END_LICENSE$
**
****************************************************************************/

And now for the actual source.

main.cpp:

#include <QtGui/QGuiApplication>
#include <QtQuick/QQuickView>

int main(int argc, char *argv[])
{
    // the following line enables Google's ANGLE layer,
    // since Qt Quick is really buggy on my Intel GPU unless I enable it.
    // But that's not what is causing the problem, because the problem
    // also occurs on non-Windows platforms, on which this line is a no-op
    // and OpenGL calls are executed directly rather than being translated
    // to DirectX.
    QGuiApplication::setAttribute(Qt::AA_UseOpenGLES);
    QGuiApplication app(argc, argv);

    QQuickView viewer;
    viewer.setSource(QUrl("qrc:/main.qml"));
    viewer.setResizeMode(QQuickView::SizeRootObjectToView);
    viewer.show();

    return app.exec();
}

main.qml:

Canvas3D {
    width: 800
    height: 500
    
    id: canvas3d
    
    onInitializeGL: {
        GLCode.initializeGL();
    }

    onPaintGL: {
        GLCode.paintGL();
    }

    onResizeGL: {
        GLCode.resizeGL();
    }
}

GLCode.js:

Qt.include("gl-matrix.js")

var gl;
var vertexPositionAttribute;
var mvMatrix = mat4.create();
var pMatrix  = mat4.create();
var pMatrixUniform;
var mvMatrixUniform;
var colorUniform; // unused. removing it produced a weird error. please ignore.

function initializeGL() {
    gl = canvas3d.getContext("canvas3d");

    gl.disable(gl.DEPTH_TEST);
    gl.disable(gl.CULL_FACE);
    gl.clearColor(0.0, 0.0, 0.0, 1.0);

    initShaders();

    initBuffers();
}

function resizeGL()
{
    gl.viewport(0, 0, canvas3d.width, canvas3d.height);
    mat4.perspective(pMatrix, degToRad(45), canvas3d.width / canvas3d.height, 0.1, 500.0);
    gl.uniformMatrix4fv(pMatrixUniform, false, pMatrix);
}

function degToRad(degrees) {
    return degrees * Math.PI / 180;
}

function paintGL() {
    gl.clear(gl.COLOR_BUFFER_BIT);
    
    mat4.identity(mvMatrix);
    mat4.translate(mvMatrix, mvMatrix, [0, 0, -10.0]);
    
    gl.colorMask(gl.TRUE, gl.TRUE, gl.TRUE, gl.TRUE);
    gl.uniformMatrix4fv(mvMatrixUniform, false, mvMatrix);
    gl.drawElements(gl.TRIANGLES, 3, gl.UNSIGNED_SHORT, 0);
}

function initBuffers()
{
    var cubeVertexPositionBuffer = gl.createBuffer();
    cubeVertexPositionBuffer.name = "cubeVertexPositionBuffer";
    gl.bindBuffer(gl.ARRAY_BUFFER, cubeVertexPositionBuffer);
    gl.bufferData(gl.ARRAY_BUFFER,
                new Float32Array([
                                        -1.0, -1.0,
                                        1.0, -1.0,
                                        1.0,  1.0
                                 ]),
                gl.STATIC_DRAW);
    
    gl.vertexAttribPointer(vertexPositionAttribute, 2, gl.FLOAT, false, 0, 0);

    var cubeVertexIndexBuffer = gl.createBuffer();
    gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, cubeVertexIndexBuffer);
    gl.bufferData(gl.ELEMENT_ARRAY_BUFFER,
                  new Uint16Array([0,  1,  2]),
                  gl.STATIC_DRAW);
}

function initShaders()
{
    var vertexShader = getShader(gl,
                                  "attribute highp vec3 aVertexPosition;  \
                                                                         \
                                  uniform mat4 uMVMatrix;                \
                                  uniform mat4 uPMatrix;                 \
                                                                         \
                                  void main(void) {                      \
                                      gl_Position = uPMatrix * uMVMatrix * vec4(aVertexPosition, 1.0); \
                                  }"
                                 , gl.VERTEX_SHADER);
    var fragmentShader = getShader(gl,
                                   "void main(void) {                  \
                                        gl_FragColor = vec4(1.0);           \
                                    }"
                                   , gl.FRAGMENT_SHADER);
    var shaderProgram = gl.createProgram();

    gl.attachShader(shaderProgram, vertexShader);
    gl.attachShader(shaderProgram, fragmentShader);

    gl.linkProgram(shaderProgram);

    if (!gl.getProgramParameter(shaderProgram, gl.LINK_STATUS)) {
        console.log("Could not initialise shaders");
        console.log(gl.getProgramInfoLog(shaderProgram));
    }

    gl.useProgram(shaderProgram);

    vertexPositionAttribute = gl.getAttribLocation(shaderProgram, "aVertexPosition");
    gl.enableVertexAttribArray(vertexPositionAttribute);

    pMatrixUniform = gl.getUniformLocation(shaderProgram, "uPMatrix");
    mvMatrixUniform = gl.getUniformLocation(shaderProgram, "uMVMatrix");
}

function getShader(gl, str, type) {
    var shader = gl.createShader(type);
    gl.shaderSource(shader, str);
    gl.compileShader(shader);

    if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
        console.log("JS:Shader compile failed");
        console.log(gl.getShaderInfoLog(shader));
        return null;
    }

    return shader;
}

Required js libary: "glMatrix 2.2.2". I can't find this exact version from its official site, but you can get almost the same version (2.2.1) here.

I've also uploaded the complete testcase project here, if you want to run it more easily. (the download button is at the top-right corner of that page). Please ignore the confusing project name, it's a leftover.

My theory is that this is a bug in the Canvas3D's OpenGL JavaScript bindings but I'm not sure.

Edit: found a workaround:

I managed to workaround this by replacing my call of:

gl.colorMask(gl.FALSE, gl.FALSE, gl.FALSE, gl.FALSE);

(which is what it looks like in my real app) with this:

gl.enable(gl.BLEND);
gl.blendFunc(gl.ZERO, gl.ONE);

I'd still like to know why gl.colorMask breaks stuff this way though.

Edit: found the same problem with the depth mask:

As a separate but related bug, replacing the gl.colorMask call in my testcase with a gl.depthMask(gl.FALSE) (or even gl.depthMask(gl.TRUE)) also causes rendering problems. In that case it's not the whole screen that's flickering - instead, only the triangle flickers. Of course to reproduce the bug one should also enable (rather than disable) depth testing in initializeGL, and to clear DEPTH_BUFFER_BIT too, rather than just COLOR_BUFFER_BIT.

That one might be harder to find a proper workaround for, although in my specific case I may get away with disabling gl.DEPTH_TEST altogether for this rendering stage in my algorithm (which has the side effect of disabling depth writes as well)