Skip to content

Customizing the Search Space

The search space used for tuning can be completely overwritten by the user, if desired. Before we describe how to do that, let us quickly recap some foundational concepts.

At its core, a search space is simply a set of all points that we are willing to consider during the optimization process. In our particular case, each point in a search space should relate to a particular engine configuration, whereas each configuration is described by a set of environmental variables and their values, and another set of command-line arguments with their values. Both sets can be potentially empty.

For example, the following could be considered a valid search space of 3 distinct points:

  • command line: "--foo=1 --bar", environment: ""
  • command line: "test", environment: "MAKO='is_great'"
  • command line: "", environment: ""

In practice, however, exhaustively listing all points like in the example above is not how we define search spaces. Instead, a more common way of doing that, which MakoOptimize also uses, is to list all available options for each parameter, using a mapping like: parameter name: [list of possible value]. The final search space is then implicitly defined as a cartesian product of all parameters.

The following section outlines the specific format used by MakoOptimize to describe search spaces, and how to deal with some edge cases.

Search space file format

A search space can be defined with a YAML file, by following the generic format:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
cli: # (1)!
    argument: [ # (2)!
        value 1,
        value 2,
        !missing, # (3)!
        null # (4)!
    ]

    another: !flag # (5)!

env: # (6)!
    SOME_ENV_VAR: ["true", "false"] # (7)!
    MAKO: ["is_great", "is_awesome"]
  1. Command-line argument and their possible values should listed under the cli key.

  2. Argument names should be keys, after removing leading --. The value associated with the name should be a list of possible values the argument can take. In here, the following snippets can be added to the engine arguments lines:

    • --argument="value 1"
    • --argument="value 2"
    • ``
    • --argument

  3. !missing is a special tag that can be used to indicate an option of not including the argument at all. Note that this is technically different from specifying "" as value - !missing will result in not including --argument in the command line at all, while "" will result in --argument="" being appended, which might be interpreted differently.

  4. null can be used to indicate the desire to not pass any extra value for the argument. It will result in simply --argument being appended. Note that this is again different than "", because just --argument is potentially different than --argument="". Instead of null, one can also use !present, which has the same behavior.
  5. !flag can be used as a shorthand for an argument that should either be included without any value, or should be excluded. It is functionally the same as having [!present, !missing] as possible choices.
  6. Environmental variables should be listed under the env key.
  7. The same convention follows as for command-line arguments. However, remember that environmental variables have to be strings!