Introduction

bazelizing, bazelization, to bazelize, bazelized: The act of converting some existing software artifact to be easily consumed by the Bazel build tool

Embree is middleware for ray tracing that computes intersections points between rays and different shape types. The source code of Embree is under the Apache 2.0 license and hosted on GitHub. Embree targets CPU ray tracing and contains special code to make use of SSE, AVX, AVX2, and AVX-512 instructions if the target CPU supports it.

This post will cover how I bazelized Embree 3.13.0.

What is this time different?

There is an article where I described how I did bazelize Embree 3.12.1.

Embree lets you select an tasking system. There you can choose between three different options:

  • TBB
  • INTERNAL
  • PPL

In the last post about Embree, I only described how to use the INTERNAL tasking system. In this post I want to test Embree 3.13.0 using the internal tasking system first. Afterward, I will test Embree using oneTBB as a tasking system. I will also do performance measurements to find out if and what the difference is here.

Embree has also support for different ISAs:

  • AVX
  • AVX2
  • AVX512
  • SSE2
  • SSE42

In the last post I only tested SSE2. In this post I also want to test SSE42 and AVX.

There is also an option to use ispc. I guess this is not needed for using the SSE42 and AVX, but I am not entirely sure here.

Embree can also be configured to support different geometry types. Last time I only supported triangles:

/* #undef EMBREE_RAY_MASK */
/* #undef EMBREE_STAT_COUNTERS */
/* #undef EMBREE_BACKFACE_CULLING */
/* #undef EMBREE_BACKFACE_CULLING_CURVES */
/* #undef EMBREE_FILTER_FUNCTION */
/* #undef EMBREE_IGNORE_INVALID_RAYS */
#define EMBREE_GEOMETRY_TRIANGLE
/* #undef EMBREE_GEOMETRY_QUAD */
/* #undef EMBREE_GEOMETRY_CURVE */
/* #undef EMBREE_GEOMETRY_SUBDIVISION */
/* #undef EMBREE_GEOMETRY_USER */
/* #undef EMBREE_GEOMETRY_INSTANCE */
/* #undef EMBREE_GEOMETRY_GRID */
/* #undef EMBREE_GEOMETRY_POINT */
#define EMBREE_RAY_PACKETS
/* #undef EMBREE_COMPACT_POLYS */

This time I consider more geometry types:

"#cmakedefine EMBREE_GEOMETRY_TRIANGLE": "#define EMBREE_GEOMETRY_TRIANGLE",
"#cmakedefine EMBREE_GEOMETRY_QUAD": "#define EMBREE_GEOMETRY_QUAD",
"#cmakedefine EMBREE_GEOMETRY_CURVE": "#define EMBREE_GEOMETRY_CURVE",
"#cmakedefine EMBREE_GEOMETRY_SUBDIVISION": "#define EMBREE_GEOMETRY_SUBDIVISION",
"#cmakedefine EMBREE_GEOMETRY_USER": "#define EMBREE_GEOMETRY_USER",
"#cmakedefine EMBREE_GEOMETRY_INSTANCE": "#define EMBREE_GEOMETRY_INSTANCE",
"#cmakedefine EMBREE_GEOMETRY_GRID": "#define EMBREE_GEOMETRY_GRID",
"#cmakedefine EMBREE_GEOMETRY_POINT": "#define EMBREE_GEOMETRY_POINT",
"#cmakedefine EMBREE_RAY_PACKETS": "#define EMBREE_RAY_PACKETS",
"#cmakedefine EMBREE_COMPACT_POLYS": "/* #undef EMBREE_COMPACT_POLYS */",

Bazalization of Embree 3.13.0

Embree has some common libs

  • math
  • simd
  • sys
  • tasking

Last time I tried to introduce some dependencies between those libs. But I dropped this approach this time. Those libs use partially the same source files or at least have dependencies to the same source files. Even worse those dependencies are circular. I think that is an architectural problem within Embree. Nevertheless, I made of the above libraries for individual libraries using cc_library without having any dependencies on any other library (except implicit one).

Embree 3.13.3 contains three header files that can be configured:

  • kernels/rtcore_config.h.in
  • kernels/config.h.in
  • kernels/hash.h.in

For those ones I use within the Bazel build know template_rule that was taken from TensorFlow.

One problem I stumbled over was that the introduction of template_header did cause some issues with relative includes.

Here is a picture how the Bazel build of Embree 3.13.0 looks like:

embree3.13.0_graph.png

You can generate this image by:

bazel query --noimplicit_deps 'deps(//:embree)' --output graph > graph.in
dot -Tpng < graph.in > graph.png

I had to change in the files embree-3.13.0/kernels/common/default.h and embree-3.13.0/kernels/common/device.cpp the includes from #include "../config.h" and #include "../hash.h" to #include kernels/config.h and #include "kernels/hash.h". Unfortunately, relative include pathes of generated headers seem not to work with Bazel (see next section for a detailed discussion about this). Even more, worse is the fact that these two changes break the CMake based build.

Include a generated header file using a relative path

I started to setup a minimal demo for this problem.

As already mentioned, I am running into a problem when trying to include a generated a header file using a relative path (in the CMake build it uses configure_file):

WORKSPACE.bazel:

workspace(name = "TemplateRule")

main.cpp:

#include "kernel/some_header.h"
#include <iostream>

int main() {
    std::cout << VERSION_STR << std::endl;
}

kernel/some_header.h:

#pragma once

#include "../config.h"   // <---- include config.h using a relative path

config.h.in:

#pragma once

#define VERSION_STR "@VERSION_STR@"

BUILD.bazel

load("//bazel:template_rule.bzl", "template_rule")

template_rule(
    name = "config_h",
    src = "config.h.in",
    out = "config.h",
    substitutions = {
        "@VERSION_STR@": "1.0.3",
    },
)

cc_binary(
    name = "HelloWorld",
    srcs = [
        "main.cpp",
        "kernel/some_header.h",
        ":config_h",
    ],
)

bazel/BUILD.bazel: < empty >

bazel/template_rule.bzl:

# Copied from https://github.com/tensorflow/tensorflow/blob/master/third_party/common.bzl

'''
    Copyright 2019 The TensorFlow Authors. All Rights Reserved.
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.
    You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
'''

def template_rule_impl(ctx):
    out = ctx.outputs.out
    ctx.actions.expand_template(
        template = ctx.file.src,
        output = ctx.outputs.out,
        substitutions = ctx.attr.substitutions,
    )
    return [CcInfo(
        compilation_context = cc_common.create_compilation_context(
            includes = depset([out.dirname]),
            headers = depset([out]),
        ),
    )]

template_rule = rule(
    attrs = {
        "src": attr.label(
            mandatory = True,
            allow_single_file = True,
        ),
        "substitutions": attr.string_dict(mandatory = True),
        "out": attr.output(mandatory = True),
    },
    # output_to_genfiles is required for header files.
    output_to_genfiles = True,
    implementation = template_rule_impl,
)

When I run bazel build //...

I get the error:

In file included from main.cpp:1:
kernel/some_header.h:3:10: fatal error: ../config.h: No such file or directory
    3 | #include "../config.h"
      |          ^~~~~~~~~~~~~

When I include config.h in main.cpp and remove it from kernel/some_header.h everything works as expected.

Creating the file config.h and compiling the code manually works as expected:

g++  main.cpp kernel/some_header.h config.h

According to the Best Practices relative paths using .. should be avoided, but you can find such things in legacy code that uses CMake to build. Not sure if this is a restriction of Bazel. If you know of any workaround please let me know.

I posted about this also a question on StackOverflow.

Embree 3.13.0 performance

In the following you will find measurements of different configurations:

  • Embree 3.12.1 with internal tasking system
  • Embree 3.13.0 with internal tasking system
  • Embree 3.13.0 with oneTBB

Details to the test:

  • Using oneTBB 2021.3.0
  • only SSE2 ISA is tested
  • test performed on Windows 10 x64 (Version 21H1) and Visual Studio 2019 (Version 16.10.2) as a compiler.
  • Test CPU: AMD Ryzen 9 3900X 12-Core processor.
  • For Embree 3.12.1 I enabled as a geometry type only triangles
  • For Embree 3.13.1 I enabled almost all geometry types but test scene contains only tirangles
  • 512 samples per pixel, 768x768 pixels

I performed these measurements with Okapi Renderer. I used an ambient occlusion test scene of the Ajax model.

I repeated the experiment for every test configuration four times:

/images/embree3.13.0_render_time.png

It seems that Embree 3.12.1 with internal tasking system has the best performance. I am not sure why Embree 3.13.0 is slower. Maybe because of the support of different geometry types or any additions to Embree 3.13.0. Maybe for other test scenes, it performs better. What makes me also wonder is that oneTBB brings no big speed improvement.

I tested also under Ubuntu 20.04. Test results from Embree 3.13.0 with internal tasking lib:

/images/render_time_ajax_ubuntu.png

Embree 3.13.0 with oneTBB:

/images/render_time_ajax_ubuntu.png