PPX Support Last updated May 5, 2022

Changes

all ppx_* rules execpt ppx_executable rule removed
ppx executables compiled in bytecode mode always statically linked (using -custom).

Overview

PPX = PreProcessor eXtension.

Demos

Building PPX resources

See also [PPX Optimizations](optimization.md#ppx).

[modules](#modules)
[libraries](#libraries)
[archives](#archives)
[executables](#executables)

Building PPX modules

Rule: [ppx_module](../refman/rules_ppx.md#ppx_module)

Building:

load("@obazl_rules_ocaml//ocaml:rules.bzl", "ppx_module")
...
ppx_module(
    name      = "ppx_greeting",
    struct    = ":ppx_greeting.ml",
    deps_opam = ["ppxlib"]
)

Demo: [demos/ppx/rewriter/greeting](https://github.com/obazl/dev_obazl/tree/main/demos/ppx/rewriter/greeting)

Building PPX libraries

Rule: [ppx_library](../refman/rules_ppx.md#ppx_library)

Building PPX archives

Rule: [ppx_archive](../refman/rules_ppx.md#ppx_archive)

Building PPX executables

Rule: [ppx_executable](../refman/rules_ppx.md#ppx_executable)

[WARNING This documentation assumes you are using

A PPX (ppxlib) executable requires a driver, which can easily be created using a Bazel [genrule](https://docs.bazel.build/versions/master/be/general.html#genrule):

load("@obazl_rules_ocaml//ocaml:rules.bzl", "ppx_module")
...
ppx_module(
    name = "Driver",
    struct = ":ppxlib_driver.ml",
    deps_opam = ["ppxlib"]
)
genrule(
    name = "gendriver",
    outs = ["ppxlib_driver.ml"],
    # In the cmd string, '$@' == outs file, here ppxlib_driver.ml
    cmd = "\n".join([
        "echo \"(* GENERATED FILE - DO NOT EDIT *)\" > \"$@\"",
        "echo \"let () = Ppxlib.Driver.standalone ()\" >> \"$@\"",
    ]),
)

The PPX executable will depend on any PPX modules it needs, and the driver must be placed last in the dependency list. OBazl provides an optional convenience attribute to support this: pass the driver via the main attribute of ppx_executable, and OBazl will arrange for it to come last in the dep list:

load("@obazl_rules_ocaml//ocaml:rules.bzl", "ppx_executable")
...
ppx_executable(
    name = "ppx.exe",
    main = ":Driver",
    deps = [":ppx_greeting"] # ppx_module containing handler for [%greeting ...] extension points
)

equivalently:

ppx_executable(
    name = "ppx.exe",
    deps = [":ppx_greeting", ":Driver"]
)

 **IMPORTANT** Remember that compilation of an executable can succeed
even if you omit critical dependencies, since OCaml does not define
a required 'main' routine. The purpose of the `main` attribute is to
minimize the probability of putting the `deps` in the wrong order or
inadvertently omitting a driver.

Demo: [demos/ppx/rewriter/greeting](https://github.com/obazl/dev_obazl/tree/main/demos/ppx/rewriter/greeting)

Preprocessing v. build dependencies

Every OCaml module (archive, executable) has its own build dependency graph, which is a tree containing the modules upon which it directly and indirectly depends.

OCaml extension points introduce a second dependency graph we call a preprocessing dependency graph. A source file that contains an extension point, such as [%greeting "Hello"], must be preprocessed by code that is capable of handling the extension point. This preprocessing dependency is orthogonal to any build dependencies the source file may have; normally it is a single PPX executable containing PPX modules that implement handle extension point handlers.

Thus any module that contains OCaml extension points has two distinct dependency graphs, one for build dependencies and one for preprocessing dependencies. In OBazl rules, ordinary build dependencies are usually expressed using a deps attribute, and preprocessing dependencies are expressed using the ppx attribute and a few additional ppx_* attributes to parameterize the PPX executable.

Codependencies

Sometimes PPX processing injects code that induces compile-time dependencies; such dependencies must be listed as deps in the ocaml_module or ppx_module rule that compiles the transformed source file. These are often erroneously called "runtime" dependencies, but Runtime dependencies is a different concept. Runtime dependencies of a module or executable are needed when that module or executable is executed, not when it is built. These dependencies do not fit that description, so OBazl calls them ppx_codeps.

In other words, codependencies are build dependencies that are attached to a preprocessing dependency graph and passed on to preprocessing outputs.

One way to support such codependencies is to list them in the deps attribute of the ocaml_module or ppx_module rule instances that use the PPX executable and compile its output, as noted above. However this requires maintenance of the deps attribute for each rule instance using the PPX executable in question. Since PPX executables may be shared by many targets, this is cumbersome and error-prone.

attribute: ppx_codeps

As a convenience, OBazl supports an attribute, ppx_codeps, on ppx_module and ppx_executable rules. Dependencies listed in this attribute will be automatically propagated through the preprocessing dependency graph to the build rule of the transformed source. For example, if an ocaml_module rule instance lists a ppx dependency (referring to a ppx_executable, then any codependencies listed in the dependency graph of that ppx will be added as build dependencies of the module being compiled by the rule.

See demos/ppxlib/ppx_codeps for an example.

Runtime dependencies

Runtime dependencies are files that are required by modules and/or executables at runtime. For example, a common pattern is to have a module read a file of configuration data at runtime; such a data file constitutes a runtime dependency of the module. Another common case is dynamic loading and linking using dlopen; the library to be loaded is a runtime dependency.

For non-PPX modules and executables, such files must be passed using the data attribute; for PPX modules and executables, they must be passed using the ppx_data attribute, as [described below](#ppx_data). The rules will arrange for the files to be included in the generated command line with the appropriate option flags.

PPX executables

Main Module

Unlike many compiled languages, OCaml does not define a main entry point for executables. The modules used to construct an executable are organized in the executable binary in the order in which they were passed as arguments to the compiler. When control is passed to an OCaml executable, the (top-level) code of the component modules is executed in order.

This means it is possible to successfully compile and run an OCaml executable that lacks critical modules. Since there is no main entry point, the compiler has no way of knowing that something is missing.

The main attribute of the ppx_executable rule is an optional convenience attribute, intended to reduce the likelihood of inadvertently omitting the critical piece of code that drives PPX processing. A module passed as main will automatically be added as the last module in the dependency list, thereby ensuring that it will receive control after all other modules.

Demo code: [demos/ppx/hello](https://github.com/obazl/dev_obazl/blob/aed0ce898b480c109ccd9b42fddc6f6c1640277c/demos/ppx/hello/BUILD.bazel#L53)

The Ppxlib Driver module

Here is one way to implement a driver for a ppx_executable:

ppx_executable( name = "_ppx.exe", main = ":_Driver", ...etc... )
ppx_module(
    name = "_Driver",
    src = ":ppxlib_driver.ml",
    deps = ["@opam//pkg:ppxlib"],
)
genrule(
    name = "gendriver",
    outs = ["ppxlib_driver.ml"],
    cmd = "\n".join([
        "echo \"(* GENERATED FILE - DO NOT EDIT *)\" > \"$@\"",
        "echo \"let () = Ppxlib.Driver.standalone ()\" >> \"$@\"",
    ]),
)

PPX attributes

These attributes apply to rules [ocaml_module](../refman/rules_ocaml.md#ocaml_module), [ocaml_interface](../refman/ocaml_rules.md#ocaml_interface), [ppx_module](../refman/rules_ppx.md#ppx_module).

Attributes applicable to ppx_* rules are documented in the [Reference Manual](../refman/rules_ppx.md)

ppx

The ppx attribute takes a ppx_executable target. The rule will generate several actions - see [Action Queries](transparency.md#action_queries) to see how to inspect the actions.

ppx_codeps

See above.

ppx_args

Use ppx_args to pass options to the ppx_executable that is passed via the ppx attribute.

ppx_data

Bazel uses a data attribute for runtime file dependencies; OBazl follows this convention. For rules ocaml_executable, ocaml_module, ocaml_interface, ppx_executable, and ppx_module, the data attribute is for files that will be needed at runtime.

The ppx_data attribute is for files that are needed by the ppx executable when it transforms source files. For example, [ppx_optcomp]() supports an extension, import, that acts like the #include directive of the C preprocessor language: it allows you to include the content of one file in another. This induces a runtime dependency: if foo.ml contains e.g. [%import "config.mlh"], then the file config.mlh must be available to ppx_optcomp when it runs (as part of the ppx_executable tasked with transforming foo.ml). So this is a genuine runtime dependency, and it must be listed in the ppx_data attribute of the ppx_executable rule instance that lists ppx_optcomp as a dependency.

See [ppx/ppx_optcomp](https://github.com/obazl/dev_obazl/blob/c0f01d6ae66ecdebbbfac687120ef734886542d4/demos/ppx/ppx_optcomp/BUILD.bazel#L27) for an example.

ppx_print

PPX executables can emit the AST they produce in binary or text form.

Rules that support PPX processing ([ocaml_interface](../refman/rules_ocaml.md#ocaml_interface), [ocaml_module](../refman/rules_ocaml.md#ocaml_module), [ppx_module](../refman/rules_ppx.md#ppx_module)) also support the ppx_print attribute, which controls output format.

The ppx_print attribute takes a label, which must be either @ppx//print:binary or @ppx//print:text. The former tells OBazl to add -dump-ast as a command line option when running the ppx_executable that is passed by the ppx attribute; the latter just omits the argument.

The default print output format is determined by the [config rules](configrules.md) target @ppx//print, which in turn defaults to binary. You can change the global default to print by passing --@ppx//print:text on the command line. Use the ppx_print attribute to override this global default.

PPX Testing & Troubleshooting

Ppx libraries are notoriously under-documented. It is often the case that their authors use Dune and dispense with documentation.

This means that to use a Ppx library you may need to do some deciphering. The Dune build files that may accompany any test cases in a library distrib may not contain sufficient information.

case study: ppx_assert

ppx_assert contains no build documentation at all. The only thing to go on is one test case, whose Dune file looks like this:

(library
 (name ppx_assert_test_lib)
 (libraries sexplib str)
 (preprocess
  (pps ppx_compare ppx_sexp_conv ppx_here ppx_assert ppx_inline_test)))

There are two major problems here, both involving "hidden" build configurations.

One is that using ppx_inline_test requires that the ppx executable be run with arguments -inline-test-lib and an "tag" to identify the test.

The other problem is that the list of dependencies is incomplete. The source file uses a [%here] extension from ppx_here; this has the effect of injecting a dependency on Ppx_here_lib into the transformed source file. This means that ppx_here is both a direct dependency and a codependency of the ppx executable, but you would not know that from the Dune file, and of course it is not documented. Dune discovers this dynamically at build time, but since dynamically added dependencies compromise replicability, Bazel disallows them.

So if we write a Bazel target using only the information in this Dune stanza, we’ll get something that won’t build due to missing dependencies. The following shows what we need to add:

ocaml_module(
    name          = "Ppx_assert_test",
    struct        = "ppx_assert_test.ml",
    deps          = [
        "@sexplib//lib/sexplib",
        "@ocaml//lib/str",
    ],
    ppx           = ":ppx.exe",
    ppx_args      = [                 (1)
        "-inline-test-lib",
        "ppx_assert_test_lib"
    ],
    # ppx_print = "@rules_ocaml//ppx/print:text!",  (2)
)

ppx_executable(
    name    = "ppx.exe",
    main    = "@ppxlib//lib/runner",
    prologue = [
        "@ppx_compare//lib/ppx_compare",
        "@ppx_sexp_conv//lib/ppx_sexp_conv",
        "@ppx_here//lib/ppx_here",
        "//src:ppx_assert",
        "@ppx_inline_test//lib/ppx_inline_test"
    ],
    ppx_codeps = ["@ppx_here//lib/ppx_here"]   (3)
)

1	Args to `ppx.exe` required by `ppx_inline_test`.
2	Optional - see comments below.
3	This codependency will be passed to the `ocaml_module` rule for use as a build dependency.

If we omit the ppx_args, we get this error:

$ bazel build test:Ppx_assert_test
Error: ppx_inline_test: extension is disabled because the tests would be ignored (the build system didn't pass -inline-test-lib...

If we omit ppx_codeps:

Error: Unbound module Ppx_here_lib

Now the problem is that the original source file makes no mention of Ppx_here_lib; rather it contains [%here] in several places, and the ppx executable replaces this with code that does reference Ppx_here_lib. To inspect the result we can add the following to the ocaml_module target:

ppx_print = "@rules_ocaml//ppx/print:text!",

This will direct the ppx executable to emit text rather than binary data. Then we can open the output file and inspect the generated code.

Rules

[ocaml_test](../refman/rules_ocaml.md#ocaml_test)
[ppx_test](../refman/rules_ppx.md#ppx_test)

Troubleshooting

Case: ppx_assert.

resources

Am I missing some comprehensive Ppxlib resource somewhere? forum msg, Feb 2022