piglit (IV): How to write binary test programs

Posted by Samuel Iglesias on June 11, 2015

Last post I talked about how to develop GLSL shader tests and how to add them to piglit. This is a nice way to develop simple tests but sometimes you need to do something more complex. For that case, piglit can run binary test programs.

Introduction

Binary test programs in piglit are written in C language taking advantage of the piglit framework to facilitate the test development (there is no main() function, no need to setup GLX (or WGL or EGL or whatever), no need to check manually the available extensions or call to glXSwapBuffers() or similar functions...), however you can still use all OpenGL C-language API.

Simple example

Piglit framework is mostly undocumented but easy to understand once you start reading some existing tests. So I will start with a simple example explaining how a typical binary test looks like and then show you a real example.

/*
 * Copyright © 2015 Intel Corporation
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the next
 * paragraph) shall be included in all copies or substantial portions of the
 * Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 */

/** @file test-example.c
 *
 * This test shows an skeleton for creating new tests.
 */

#include "piglit-util-gl.h"

PIGLIT_GL_TEST_CONFIG_BEGIN
    config.window_width = 100;
    config.window_height = 100;
    config.supports_gl_compat_version = 10;
    config.supports_gl_core_version = 31;
    config.window_visual = PIGLIT_GL_VISUAL_DOUBLE | PIGLIT_GL_VISUAL_RGBA;

PIGLIT_GL_TEST_CONFIG_END

static const char vs_pass_thru_text[] =
    "#version 330n"
    "n"
    "in vec4 piglit_vertex;n"
    "n"
    "void main() {n"
    "   gl_Position = piglit_vertex;n"
        "}n";

static const char fs_source[] =
    "#version 330n"
    "n"
    "void main() {n"
    "       color = vec4(1.0, 0.0. 0.0, 1.0);n"
    "}n";

GLuint prog;

void
piglit_init(int argc, char **argv)
{
    bool pass = true;

    /* piglit_require_extension("GL_ARB_shader_storage_buffer_object"); */

    prog = piglit_build_simple_program(vs_pass_thru_text, fs_source);

    glUseProgram(prog);

    glClearColor(0, 0, 0, 0);

    /* <-- OpenGL commands to be done --> */

    glViewport(0, 0, piglit_width, piglit_height);

    /* piglit_draw_* commands can go into piglit_display() too */
    piglit_draw_rect(-1, -1, 2, 2);

    if (!piglit_check_gl_error(GL_NO_ERROR))
       pass = false;

    piglit_report_result(pass ? PIGLIT_PASS : PIGLIT_FAIL);
}

enum piglit_result piglit_display(void)
{
    /* <-- OpenGL drawing commands, if needed --> */

    /* UNREACHED */
    return PIGLIT_FAIL;
}

As you see in this example, there are four different parts:

  1. License header and description of the test
    • The license details should be included in each source file. There is one agreed by most contributors and it's a MIT license assigning the copyright to Intel Corporation. More information in COPYING file.
    • It includes a brief description of what the test does.
    /*
     * Copyright © 2015 Intel Corporation
     *
     * Permission is hereby granted, free of charge, to any person obtaining a
     * copy of this software and associated documentation files (the "Software"),
     * to deal in the Software without restriction, including without limitation
     * the rights to use, copy, modify, merge, publish, distribute, sublicense,
     * and/or sell copies of the Software, and to permit persons to whom the
     * Software is furnished to do so, subject to the following conditions:
     *
     * The above copyright notice and this permission notice (including the next
     * paragraph) shall be included in all copies or substantial portions of the
     * Software.
     *
     * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
     * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
     * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
     * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
     * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
     * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
     * DEALINGS IN THE SOFTWARE.
     */
    
    /** @file test-example.c
     *
     * This test shows an skeleton for creating new tests.
     */
    
  2. Piglit setup. This is needed to check if a test can be executed by a given driver (minimum supported GL version), or to create a window of a specific size, or even to define if we want double buffering.
    PIGLIT_GL_TEST_CONFIG_BEGIN
        config.window_width = 100;
        config.window_height = 100;
        config.supports_gl_compat_version = 10;
        config.supports_gl_core_version = 31;
        config.window_visual = PIGLIT_GL_VISUAL_DOUBLE | PIGLIT_GL_VISUAL_RGBA;
    
    PIGLIT_GL_TEST_CONFIG_END
    
  3. piglit_init(). This is the function that it is going to be called for configuring the test itself. Some tests implement all their code inside piglit_init() because it doesn't need to draw anything (or it doesn't need to update the drawing frame by frame). In any case, you usually put here the following code:
    • Check for needed extensions.
    • Check for limits or maximum values of different variables like GL_MAX_SHADER_STORAGE_BLOCKS, GL_UNIFORM_BLOCK_SIZE, etc.
    • Setup constant data, upload it.
    • All the initialization setup you need for drawing commands: compile and link shaders, set clear color, etc.
    void
    piglit_init(int argc, char **argv)
    {
        bool pass = true;
    
        /* piglit_require_extension("GL_ARB_shader_storage_buffer_object"); */
    
        prog = piglit_build_simple_program(vs_pass_thru_text, fs_source);
    
        glUseProgram(prog);
    
        glClearColor(0, 0, 0, 0);
    
        /* <-- OpenGL commands to be done --> */
    
        glViewport(0, 0, piglit_width, piglit_height);
    
        /* piglit_draw_* commands can go into piglit_display() too */
        piglit_draw_rect(-1, -1, 2, 2);
    
        if (!piglit_check_gl_error(GL_NO_ERROR))
           pass = false;
    
        piglit_report_result(pass ? PIGLIT_PASS : PIGLIT_FAIL);
    }
    
  4. piglit_display(). This is the function that it is going to be executed periodically to update each frame of the rendered window. In some tests, you will find it almost empty (it returns PIGLIT_FAIL) because it is not needed by the test program.
    enum piglit_result piglit_display(void)
    {
        /* <-- OpenGL drawing commands, if needed --> */
    
        /* UNREACHED */
        return PIGLIT_FAIL;
    }
    

Notice that you are free to add any helper functions you need like any other C program but the aforementioned parts are required by piglit.

Piglit API

Piglit provides a lot of functions under its API to be used by the test program. They are usually often-used functions that substitute one or several OpenGL function calls and other code that accompany them.

The available functions are listed in piglit-util.gl.h file, which must be included in every binary test source code.

/*
 * Copyright (c) The Piglit project 2007
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * on the rights to use, copy, modify, merge, publish, distribute, sub
 * license, and/or sell copies of the Software, and to permit persons to whom
 * the Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the next
 * paragraph) shall be included in all copies or substantial portions of the
 * Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.  IN NO EVENT SHALL
 * VA LINUX SYSTEM, IBM AND/OR THEIR SUPPLIERS BE LIABLE FOR ANY CLAIM,
 * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
 * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
 * USE OR OTHER DEALINGS IN THE SOFTWARE.
 */

#pragma once
#ifndef __PIGLIT_UTIL_GL_H__
#define __PIGLIT_UTIL_GL_H__

#ifdef __cplusplus
extern "C" {
#endif

#include "piglit-util.h"

#include <piglit/gl_wrap.h>
#include <piglit/glut_wrap.h>

#define piglit_get_proc_address(x) piglit_dispatch_resolve_function(x)

#include "piglit-framework-gl.h"
#include "piglit-shader.h"

extern const uint8_t fdo_bitmap[];
extern const unsigned int fdo_bitmap_width;
extern const unsigned int fdo_bitmap_height;

extern bool piglit_is_core_profile;

/**
 * Determine if the API is OpenGL ES.
 */
bool piglit_is_gles(void);

/**
 * \brief Get version of OpenGL or OpenGL ES API.
 *
 * Returned version is multiplied by 10 to make it an integer.  So for
 * example, if the GL version is 2.1, the return value is 21.
 */
int piglit_get_gl_version(void);

/**
 * \precondition name is not null
 */
bool piglit_is_extension_supported(const char *name);

/**
 * reinitialize the supported extension List.
 */
void piglit_gl_reinitialize_extensions();

/**
 * \brief Convert a GL error to a string.
 *
 * For example, given GL_INVALID_ENUM, return "GL_INVALID_ENUM".
 *
 * Return "(unrecognized error)" if the enum is not recognized.
 */
const char* piglit_get_gl_error_name(GLenum error);

/**
 * \brief Convert a GL enum to a string.
 *
 * For example, given GL_INVALID_ENUM, return "GL_INVALID_ENUM".
 *
 * Return "(unrecognized enum)" if the enum is not recognized.
 */
const char *piglit_get_gl_enum_name(GLenum param);

/**
 * \brief Convert a string to a GL enum.
 *
 * For example, given "GL_INVALID_ENUM", return GL_INVALID_ENUM.
 *
 * abort() if the string is not recognized.
 */
GLenum piglit_get_gl_enum_from_name(const char *name);

/**
 * \brief Convert a GL primitive type enum value to a string.
 *
 * For example, given GL_POLYGON, return "GL_POLYGON".
 * We don't use piglit_get_gl_enum_name() for this because there are
 * other enums which alias the prim type enums (ex: GL_POINTS = GL_NONE);
 *
 * Return "(unrecognized enum)" if the enum is not recognized.
 */
const char *piglit_get_prim_name(GLenum prim);


/**
 * \brief Check for unexpected GL errors.
 *
 * If glGetError() returns an error other than \c expected_error, then
 * print a diagnostic and return GL_FALSE.  Otherwise return GL_TRUE.
 */
GLboolean
piglit_check_gl_error_(GLenum expected_error, const char *file, unsigned line);

#define piglit_check_gl_error(expected) \
 piglit_check_gl_error_((expected), __FILE__, __LINE__)

/**
 * \brief Drain all GL errors.
 *
 * Repeatly call glGetError and discard errors until it returns GL_NO_ERROR.
 */
void piglit_reset_gl_error(void);

void piglit_require_gl_version(int required_version_times_10);
void piglit_require_extension(const char *name);
void piglit_require_not_extension(const char *name);
unsigned piglit_num_components(GLenum base_format);
bool piglit_get_luminance_intensity_bits(GLenum internalformat, int *bits);
int piglit_probe_pixel_rgb_silent(int x, int y, const float* expected, float *out_probe);
int piglit_probe_pixel_rgba_silent(int x, int y, const float* expected, float *out_probe);
int piglit_probe_pixel_rgb(int x, int y, const float* expected);
int piglit_probe_pixel_rgba(int x, int y, const float* expected);
int piglit_probe_rect_r_ubyte(int x, int y, int w, int h, GLubyte expected);
int piglit_probe_rect_rgb(int x, int y, int w, int h, const float* expected);
int piglit_probe_rect_rgb_silent(int x, int y, int w, int h, const float *expected);
int piglit_probe_rect_rgba(int x, int y, int w, int h, const float* expected);
int piglit_probe_rect_rgba_int(int x, int y, int w, int h, const int* expected);
int piglit_probe_rect_rgba_uint(int x, int y, int w, int h, const unsigned int* expected);
void piglit_compute_probe_tolerance(GLenum format, float *tolerance);
int piglit_compare_images_color(int x, int y, int w, int h, int num_components,
                const float *tolerance,
                const float *expected_image,
                const float *observed_image);
int piglit_probe_image_color(int x, int y, int w, int h, GLenum format, const float *image);
int piglit_probe_image_rgb(int x, int y, int w, int h, const float *image);
int piglit_probe_image_rgba(int x, int y, int w, int h, const float *image);
int piglit_compare_images_ubyte(int x, int y, int w, int h,
                const GLubyte *expected_image,
                const GLubyte *observed_image);
int piglit_probe_image_stencil(int x, int y, int w, int h, const GLubyte *image);
int piglit_probe_image_ubyte(int x, int y, int w, int h, GLenum format,
                 const GLubyte *image);
int piglit_probe_texel_rect_rgb(int target, int level, int x, int y,
                int w, int h, const float *expected);
int piglit_probe_texel_rgb(int target, int level, int x, int y,
               const float* expected);
int piglit_probe_texel_rect_rgba(int target, int level, int x, int y,
                 int w, int h, const float *expected);
int piglit_probe_texel_rgba(int target, int level, int x, int y,
                const float* expected);
int piglit_probe_texel_volume_rgba(int target, int level, int x, int y, int z,
                 int w, int h, int d, const float *expected);
int piglit_probe_pixel_depth(int x, int y, float expected);
int piglit_probe_rect_depth(int x, int y, int w, int h, float expected);
int piglit_probe_pixel_stencil(int x, int y, unsigned expected);
int piglit_probe_rect_stencil(int x, int y, int w, int h, unsigned expected);
int piglit_probe_rect_halves_equal_rgba(int x, int y, int w, int h);

bool piglit_probe_buffer(GLuint buf, GLenum target, const char *label,
             unsigned n, unsigned num_components,
             const float *expected);

int piglit_use_fragment_program(void);
int piglit_use_vertex_program(void);
void piglit_require_fragment_program(void);
void piglit_require_vertex_program(void);
GLuint piglit_compile_program(GLenum target, const char* text);
GLvoid piglit_draw_triangle(float x1, float y1, float x2, float y2,
                float x3, float y3);
GLvoid piglit_draw_triangle_z(float z, float x1, float y1, float x2, float y2,
                  float x3, float y3);
GLvoid piglit_draw_rect_custom(float x, float y, float w, float h,
                   bool use_patches);
GLvoid piglit_draw_rect(float x, float y, float w, float h);
GLvoid piglit_draw_rect_z(float z, float x, float y, float w, float h);
GLvoid piglit_draw_rect_tex(float x, float y, float w, float h,
                            float tx, float ty, float tw, float th);
GLvoid piglit_draw_rect_back(float x, float y, float w, float h);
void piglit_draw_rect_from_arrays(const void *verts, const void *tex,
                  bool use_patches);

unsigned short piglit_half_from_float(float val);

void piglit_escape_exit_key(unsigned char key, int x, int y);

void piglit_gen_ortho_projection(double left, double right, double bottom,
                 double top, double near_val, double far_val,
                 GLboolean push);
void piglit_ortho_projection(int w, int h, GLboolean push);
void piglit_frustum_projection(GLboolean push, double l, double r, double b,
                   double t, double n, double f);
void piglit_gen_ortho_uniform(GLint location, double left, double right,
                  double bottom, double top, double near_val,
                  double far_val);
void piglit_ortho_uniform(GLint location, int w, int h);

GLuint piglit_checkerboard_texture(GLuint tex, unsigned level,
    unsigned width, unsigned height,
    unsigned horiz_square_size, unsigned vert_square_size,
    const float *black, const float *white);
GLuint piglit_miptree_texture(void);
GLfloat *piglit_rgbw_image(GLenum internalFormat, int w, int h,
                           GLboolean alpha, GLenum basetype);
GLubyte *piglit_rgbw_image_ubyte(int w, int h, GLboolean alpha);
GLuint piglit_rgbw_texture(GLenum internalFormat, int w, int h, GLboolean mip,
            GLboolean alpha, GLenum basetype);
GLuint piglit_depth_texture(GLenum target, GLenum format, int w, int h, int d, GLboolean mip);
GLuint piglit_array_texture(GLenum target, GLenum format, int w, int h, int d, GLboolean mip);
GLuint piglit_multisample_texture(GLenum target, GLenum tex,
                  GLenum internalFormat,
                  unsigned width, unsigned height,
                  unsigned depth, unsigned samples,
                  GLenum format, GLenum type, void *data);
extern float piglit_tolerance[4];
void piglit_set_tolerance_for_bits(int rbits, int gbits, int bbits, int abits);
extern void piglit_require_transform_feedback(void);

bool
piglit_get_compressed_block_size(GLenum format,
                 unsigned *bw, unsigned *bh, unsigned *bytes);

unsigned
piglit_compressed_image_size(GLenum format, unsigned width, unsigned height);

unsigned
piglit_compressed_pixel_offset(GLenum format, unsigned width,
                   unsigned x, unsigned y);

void
piglit_visualize_image(float *img, GLenum base_internal_format,
               int image_width, int image_height,
               int image_count, bool rhs);

float piglit_srgb_to_linear(float x);
float piglit_linear_to_srgb(float x);

extern GLfloat cube_face_texcoords[6][4][3];
extern const char *cube_face_names[6];
extern const GLenum cube_face_targets[6];

/**
 * Common vertex program code to perform a model-view-project matrix transform
 */
#define PIGLIT_VERTEX_PROGRAM_MVP_TRANSFORM        \
    "ATTRIB iPos = vertex.position;\n"      \
    "OUTPUT oPos = result.position;\n"      \
    "PARAM  mvp[4] = { state.matrix.mvp };\n"   \
    "DP4    oPos.x, mvp[0], iPos;\n"        \
    "DP4    oPos.y, mvp[1], iPos;\n"        \
    "DP4    oPos.z, mvp[2], iPos;\n"        \
    "DP4    oPos.w, mvp[3], iPos;\n"

/**
 * Handle to a generic fragment program that passes the input color to output
 *
 * \note
 * Either \c piglit_use_fragment_program or \c piglit_require_fragment_program
 * must be called before using this program handle.
 */
extern GLint piglit_ARBfp_pass_through;

static const GLint PIGLIT_ATTRIB_POS = 0;
static const GLint PIGLIT_ATTRIB_TEX = 1;

/**
 * Given a GLSL version number, return the lowest-numbered GL version
 * that is guaranteed to support it.
 */
unsigned
required_gl_version_from_glsl_version(unsigned glsl_version);


#ifdef __cplusplus
} /* end extern "C" */
#endif

#endif /* __PIGLIT_UTIL_GL_H__ */

Most functions are undocumented although there are a lot of examples of how to use it in other piglit tests. Furthermore, once you know which function you need, it is usually straightforward to learn how to call it.

Just to mention a few: you can request extensions (piglit_require_extension()) or a GL version (piglit_require_gl_version()), compile program (piglit_compile_program()), draw a rectangle or triangle, read pixel values and compare them with a list of expected values, check for GL errors, etc.

piglit-shader.h has all the shader-related functions: compile a shader, link a simple program, build (i.e. compile and link) a simple program with vertex and fragment shaders, require a specific GLSL version, etc.

/*
 * Copyright (c) The Piglit project 2007
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * on the rights to use, copy, modify, merge, publish, distribute, sub
 * license, and/or sell copies of the Software, and to permit persons to whom
 * the Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the next
 * paragraph) shall be included in all copies or substantial portions of the
 * Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT.  IN NO EVENT SHALL
 * VA LINUX SYSTEM, IBM AND/OR THEIR SUPPLIERS BE LIABLE FOR ANY CLAIM,
 * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
 * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
 * USE OR OTHER DEALINGS IN THE SOFTWARE.
 */

#pragma once

/**
 * Null parameters are ignored.
 *
 * \param es Is it GLSL ES?
 */
void piglit_get_glsl_version(bool *es, int* major, int* minor);

GLuint piglit_compile_shader(GLenum target, const char *filename);
GLuint piglit_compile_shader_text_nothrow(GLenum target, const char *text);
GLuint piglit_compile_shader_text(GLenum target, const char *text);
GLboolean piglit_link_check_status(GLint prog);
GLboolean piglit_link_check_status_quiet(GLint prog);
GLint piglit_link_simple_program(GLint vs, GLint fs);
GLint piglit_build_simple_program(const char *vs_source, const char *fs_source);
GLuint piglit_build_simple_program_unlinked(const char *vs_source,
                        const char *fs_source);
GLint piglit_link_simple_program_multiple_shaders(GLint shader1, ...);
GLint piglit_build_simple_program_unlinked_multiple_shaders_v(GLenum target1,
                                 const char*source1,
                                 va_list ap);
GLint piglit_build_simple_program_unlinked_multiple_shaders(GLenum target1,
                               const char *source1,
                               ...);
GLint piglit_build_simple_program_multiple_shaders(GLenum target1,
                          const char *source1,
                          ...);

extern GLboolean piglit_program_pipeline_check_status(GLuint pipeline);
extern GLboolean piglit_program_pipeline_check_status_quiet(GLuint pipeline);

/**
 * Require a specific version of GLSL.
 *
 * \param version Integer version, for example 130
 */
extern void piglit_require_GLSL_version(int version);
/** Require any version of GLSL */
extern void piglit_require_GLSL(void);
extern void piglit_require_fragment_shader(void);
extern void piglit_require_vertex_shader(void);

There are more header files such as piglit-glx-util.h, piglit-matrix.h, piglit-util-egl.h, etc.

Usually, you only need to add piglit-util-gl.h to your source code, however I recommend you to browse through tests/util/ so you find out all the available functions that piglit provides.

Example

A complete example of how a piglit binary test looks like is ARB_uniform_buffer_object rendering test.

/*
 * Copyright (c) 2014 VMware, Inc.
 *
 * Permission is hereby granted, free of charge, to any person obtaining a
 * copy of this software and associated documentation files (the "Software"),
 * to deal in the Software without restriction, including without limitation
 * the rights to use, copy, modify, merge, publish, distribute, sublicense,
 * and/or sell copies of the Software, and to permit persons to whom the
 * Software is furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice (including the next
 * paragraph) shall be included in all copies or substantial portions of the
 * Software.
 *
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
 * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
 * DEALINGS IN THE SOFTWARE.
 */

/** @file rendering.c
 *
 * Test rendering with UBOs.  We draw four squares with different positions,
 * sizes, rotations and colors where those parameters come from UBOs.
 */

#include "piglit-util-gl.h"

PIGLIT_GL_TEST_CONFIG_BEGIN

    config.supports_gl_compat_version = 20;
    config.window_visual = PIGLIT_GL_VISUAL_DOUBLE | PIGLIT_GL_VISUAL_RGBA;

PIGLIT_GL_TEST_CONFIG_END

static const char vert_shader_text[] =
    "#extension GL_ARB_uniform_buffer_object : require\n"
    "\n"
    "layout(std140) uniform;\n"
    "uniform ub_pos_size { vec2 pos; float size; };\n"
    "uniform ub_rot {float rotation; };\n"
    "\n"
    "void main()\n"
    "{\n"
    "   mat2 m;\n"
    "   m[0][0] = m[1][1] = cos(rotation); \n"
    "   m[0][1] = sin(rotation); \n"
    "   m[1][0] = -m[0][1]; \n"
    "   gl_Position.xy = m * gl_Vertex.xy * vec2(size) + pos;\n"
    "   gl_Position.zw = vec2(0, 1);\n"
    "}\n";

static const char frag_shader_text[] =
    "#extension GL_ARB_uniform_buffer_object : require\n"
    "\n"
    "layout(std140) uniform;\n"
    "uniform ub_color { vec4 color; float color_scale; };\n"
    "\n"
    "void main()\n"
    "{\n"
    "   gl_FragColor = color * color_scale;\n"
    "}\n";

#define NUM_SQUARES 4
#define NUM_UBOS 3

/* Square positions and sizes */
static const float pos_size[NUM_SQUARES][3] = {
    { -0.5, -0.5, 0.1 },
    {  0.5, -0.5, 0.2 },
    { -0.5, 0.5, 0.3 },
    {  0.5, 0.5, 0.4 }
};

/* Square color and color_scales */
static const float color[NUM_SQUARES][8] = {
    { 2.0, 0.0, 0.0, 1.0,   0.50, 0.0, 0.0, 0.0 },
    { 0.0, 4.0, 0.0, 1.0,   0.25, 0.0, 0.0, 0.0 },
    { 0.0, 0.0, 5.0, 1.0,   0.20, 0.0, 0.0, 0.0 },
    { 0.2, 0.2, 0.2, 0.2,   5.00, 0.0, 0.0, 0.0 }
};

/* Square rotations */
static const float rotation[NUM_SQUARES] = {
    0.0,
    0.1,
    0.2,
    0.3
};

static GLuint prog;
static GLuint buffers[NUM_UBOS];
static GLint alignment;
static bool test_buffer_offset = false;


static void
setup_ubos(void)
{
    static const char *names[NUM_UBOS] = {
        "ub_pos_size",
        "ub_color",
        "ub_rot"
    };
    static GLubyte zeros[1000] = {0};
    int i;

    glGetIntegerv(GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT, &alignment);
    printf("GL_UNIFORM_BUFFER_OFFSET_ALIGNMENT = %d\n", alignment);

    if (test_buffer_offset) {
        printf("Testing buffer offset %d\n", alignment);
    }
    else {
        /* we use alignment as the offset */
        alignment = 0;
    }

    glGenBuffers(NUM_UBOS, buffers);

    for (i = 0; i < NUM_UBOS; i++) {
        GLint index, size;

        /* query UBO index */
        index = glGetUniformBlockIndex(prog, names[i]);

        /* query UBO size */
        glGetActiveUniformBlockiv(prog, index,
                      GL_UNIFORM_BLOCK_DATA_SIZE, &size);

        printf("UBO %s: index = %d, size = %d\n",
               names[i], index, size);

        /* Allocate UBO */
        /* XXX for some reason, this test doesn't work at all with
         * nvidia if we pass NULL instead of zeros here.  The UBO data
         * is set/overwritten in the piglit_display() function so this
         * really shouldn't matter.
         */
        glBindBuffer(GL_UNIFORM_BUFFER, buffers[i]);
        glBufferData(GL_UNIFORM_BUFFER, size + alignment,
                             zeros, GL_DYNAMIC_DRAW);

        /* Attach UBO */
        glBindBufferRange(GL_UNIFORM_BUFFER, i, buffers[i],
                  alignment,  /* offset */
                  size);
        glUniformBlockBinding(prog, index, i);

        if (!piglit_check_gl_error(GL_NO_ERROR))
            piglit_report_result(PIGLIT_FAIL);
    }
}


void
piglit_init(int argc, char **argv)
{
    piglit_require_extension("GL_ARB_uniform_buffer_object");

    if (argc > 1 && strcmp(argv[1], "offset") == 0) {
        test_buffer_offset = true;
    }

    prog = piglit_build_simple_program(vert_shader_text, frag_shader_text);
    assert(prog);
    glUseProgram(prog);

    setup_ubos();

    glClearColor(0.2, 0.2, 0.2, 0.2);
}


static bool
probe(int x, int y, int color_index)
{
    float expected[4];

    /* mul color by color_scale */
    expected[0] = color[color_index][0] * color[color_index][4];
    expected[1] = color[color_index][1] * color[color_index][4];
    expected[2] = color[color_index][2] * color[color_index][4];
    expected[3] = color[color_index][3] * color[color_index][4];

    return piglit_probe_pixel_rgba(x, y, expected);
}


enum piglit_result
piglit_display(void)
{
    bool pass = true;
    int x0 = piglit_width / 4;
    int x1 = piglit_width * 3 / 4;
    int y0 = piglit_height / 4;
    int y1 = piglit_height * 3 / 4;
    int i;

    glViewport(0, 0, piglit_width, piglit_height);

    glClear(GL_COLOR_BUFFER_BIT);

    for (i = 0; i < NUM_SQUARES; i++) {
        /* Load UBO data, at offset=alignment */
        glBindBuffer(GL_UNIFORM_BUFFER, buffers[0]);
        glBufferSubData(GL_UNIFORM_BUFFER, alignment, sizeof(pos_size[0]),
                pos_size[i]);
        glBindBuffer(GL_UNIFORM_BUFFER, buffers[1]);
        glBufferSubData(GL_UNIFORM_BUFFER, alignment, sizeof(color[0]),
                color[i]);
        glBindBuffer(GL_UNIFORM_BUFFER, buffers[2]);
        glBufferSubData(GL_UNIFORM_BUFFER, alignment, sizeof(rotation[0]),
                &rotation[i]);

        if (!piglit_check_gl_error(GL_NO_ERROR))
            return PIGLIT_FAIL;

        piglit_draw_rect(-1, -1, 2, 2);
    }

    pass = probe(x0, y0, 0) && pass;
    pass = probe(x1, y0, 1) && pass;
    pass = probe(x0, y1, 2) && pass;
    pass = probe(x1, y1, 3) && pass;

    piglit_present_results();

    return pass ? PIGLIT_PASS : PIGLIT_FAIL;
}

The source starts with the license header and describing what the test does. Following those, it's the config setup for piglit: request doube buffer, RGBA pixel format and GL compat version 2.0.

Furthermore, this test defines GLSL shaders, the global data and helper functions as any other OpenGL C-language program. Notice that setup_ubos() include calls to OpenGL API but also calls to piglit_check_gl_error() and piglit_report_result() which are used to check for any OpenGL error and tell piglit that there was a failure, respectively.

Following the structure introduced before, piglit_init() indicates that ARB_uniform_buffer_object extension is required, it builds the program with the aforementioned shaders, setups the uniform buffer objects and sets the clear color.

Finally in piglit_display() is where the relevant content is placed. Among other things, it loads UBO's data, draw a rectangle and check that the rendered pixels have the same values as the expected ones. Depending of the result of that check, it will report to piglit that the test was a success or a failure.

How to add a binary test to piglit

How to build it

Now that you have written the source code of your test, it's time to learn how to build it. As we have seen in an earlier post, piglit uses cmake for generating binaries.

First you need to check if cmake tracks the directory where your source code is. If you create a new extension subdirectory under tests/spec, modify this CMakeLists.txt file and add yours.

Once you have done that, cmake needs to know that there are binaries to build and where to get its source code. For doing that, create a CMakeLists.txt file in your test's directory with the following content:

piglit_include_target_api()

If you have binaries inside directories pending the former, you can add them to the same CMakeLists.txt file you are creating:

add_subdirectory (compiler)
add_subdirectory (execution)
add_subdirectory (linker)
piglit_include_target_api()

But this is not enough to compile your test as we have not specified which binary is and where to get its source code. We do that by creating a new file CMakeLists.gl.txt with a similar content than this one.

include_directories(
    ${GLEXT_INCLUDE_DIR}
    ${OPENGL_INCLUDE_PATH}
)

link_libraries (
    piglitutil_${piglit_target_api}
    ${OPENGL_gl_LIBRARY}
)

piglit_add_executable (arb_shader_storage_buffer_object-minmax minmax.c)

# vim: ft=cmake:

As you see, first we declare where to find the needed headers and libraries. Then we define the binary name (arb_shader_storage_buffer_object-minmax) and which is its source code file (minmax.c).

And that's it. If everything is fine, next time you run cmake . && make in piglit's directory, piglit will build the test and place it inside bin/ directory.

Example in piglit repository

Let's review a real example of how a test was added for a new extension in piglit (this commit). As you see, it added tests/spec/arb_robustness/ subdirectory to tests/spec/CMakeLists.txt, create a tests/spec/arb_robustness/CMakeLists.txt to indicate cmake to track this directory, tests/spec/arb_robustness/CMakeLists.gl.txt to compile the binary test file and add its source code file tests/spec/arb_robustness/draw-vbo-bounds.c.

 tests/spec/CMakeLists.txt | 1 +
 tests/spec/arb_robustness/CMakeLists.gl.txt | 19 +++++++++++++++++++
 tests/spec/arb_robustness/CMakeLists.txt | 1 +
 tests/spec/arb_robustness/draw-vbo-bounds.c | 205 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 226 insertions(+)

If you run git log tests/spec/<dir> command in other directories, you will find similar commits.

How to run it with all.py profile

Once you successfully build the binary test program you can run it standalone. However, it's better to add it to tests/all.py profile so it is executed automatically by this profile.

Open tests/all.py  and look for your extension name, then add your test by looking at  how other tests are defined and adapt it to your case. In case you are developing tests for a new extension, you have to add more lines to tests/all.py. For example this is how was done for ARB_shader_storage_buffer_object extension:

with profile.group_manager(
        PiglitGLTest,
        grouptools.join('spec', 'arb_shader_storage_buffer_object')) as g:
    g(['arb_shader_storage_buffer_object-minmax'], 'minmax')

These lines make several things: indicate under which extension you will find the results, which are the binaries to run and which short name will appear in the summaries for each binary.

ARB_shader_storage_buffer_object's minmax test result in HTML summary ARB_shader_storage_buffer_object's minmax test in HTML summary

Conclusions

This post and last one were about how to create your own piglit tests. I hope you will find them useful and I will be glad to see new contributors because of them :-)

Next post will talk about how to send your contributions to piglit and a table of contents so you can easily jump to any post of this series.