docker-mql-compiler: reading compile_one, the function that runs MetaEditor under Wine

Written by

These are the lines the whole project exists to run. Everything else in docker-mql-compiler is setup for them, or cleanup after them:

    # MetaEditor's exit code is backwards (1 usually means success); the log's
    # Result line is the verdict, and its stderr is Wine noise.
    wine "$editor" \
        "/compile:$(winepath -w "$STAGED_SOURCE")" \
        "/log:$(winepath -w "$log_file")" \
        "/inc:$(winepath -w "$MERGED_ROOT")" >/dev/null 2>&1 || true

That’s MetaQuotes’ MetaEditor, the only thing on the planet that turns an .mq5 into an .ex5, a Windows GUI that never heard of Linux, being run by a bash script inside a Wine container with Xvfb as a fake display nobody looks at. Note the || true. The exit code gets thrown away on purpose, because it lies. My editor, my agents and my CI all live on Linux, and I got sick of somebody poking a Windows box to click “Compile” and copying errors out of a log pane by hand. So I’m going to read entrypoint.sh the way it runs: from the first download to the JSON line an agent reads at the end.

Some context first. mt5-httpapi already has a POST /compile route, MQL5 source in as JSON, a base64 .ex5 out. But it lives inside a whole Windows VM under KVM that runs live terminals and trades money, it’s MQL5 only, it refuses includes with absolute paths or .., and you upload shared libraries to its compile tree first. This one compiles a directory of MT4 and MT5 code on a laptop or a GitHub runner without booting Windows at all.

verify_installer: signed by MetaQuotes or refused

The image is Debian trixie with Wine (64-bit plus the 32-bit half, because MT4’s ancient MetaEditor is 32-bit Windows code) and not a single MetaQuotes byte. MetaTrader isn’t mine to redistribute, so the first run downloads the installers (RoboForex’s by default, override with MQL_COMPILER_MT4_URL and MQL_COMPILER_MT5_URL). It only fetches what the run needs: nothing but .mq5 files means only MT5 gets installed. About a minute and a half the first time, seconds after that.

Before wine touches an installer, osslsigncode checks its Authenticode signature against the image’s CA bundle, and then this checks who actually signed it:

require_signer() {
    local name="$1" report="$2" subject
    subject="$(awk '/^Signer.s certificate:/ { found = 1 } found && /Subject:/ { print; exit }' <<<"$report")"
    if [[ "$subject" != *"/CN=$SIGNER_PATTERN"* ]]; then
        log ERROR "installer is validly signed but not by $SIGNER_PATTERN file=$name signer=${subject#*Subject: }"
        return 1
    fi
    log INFO "installer signature ok signer=$SIGNER_PATTERN file=$name"
}

SIGNER_PATTERN is MetaQuotes. Only the signer’s certificate counts, not the chain or timestamp certificates sitting in the same report. A valid signature from somebody else is still a refusal. If the CA’s revocation server is down, verify_installer retries three times, then checks digest, chain and signer without the revocation lookup and logs a warning with reason=crl_unreachable so you know it happened.

unpack_stdlib: boot the terminal once so Trade.mqh exists

The installer doesn’t put the standard library on disk. The terminal unpacks it the first time it starts. Install MetaEditor under Wine, skip that, and every #include <Trade/Trade.mqh> dies. So setup starts each terminal headless in portable mode and waits for the marker file:

    (cd "$terminal" && wine "$(terminal_name "$platform")" /portable >/dev/null 2>&1 &)
    while [[ ! -f "$marker" ]] && ((waited < STDLIB_UNPACK_TIMEOUT_SECONDS)); do
        sleep "$POLL_SECONDS"
        waited=$((waited + POLL_SECONDS))
    done

The marker is Include/Trade/Trade.mqh for MT5 and stdlib.mqh for MT4. Once it shows up, wineserver -k kills the terminal, which never exits on its own. The installed terminals then get moved out of Program Files into plain portable directories, drive_c/terminals/mt4 and drive_c/terminals/mt5, inside the Docker volume mql-compiler-cache. Roughly 2 GB, one subdirectory per user ID. The container runs as your UID, so the compiled files are yours, not root’s.

build_merged_root: four include layers, yours on top

MetaEditor reads #include <...> from exactly one Include directory. You need at least two, the MetaQuotes standard library and your own headers. This is the function that makes the tool actually usable:

    cp -rs "$(stdlib_root "$platform")/Include" "$MERGED_ROOT/Include"
    for include_dir in ${EXTRA_INCLUDES[@]+"${EXTRA_INCLUDES[@]}"}; do
        cp -rs --remove-destination "$include_dir/." "$MERGED_ROOT/Include/"
    done
    MERGED_ROOTS[$key]="$MERGED_ROOT"
    [[ -n "$user_root" ]] || return 0
    for entry in "$user_root"/*; do
        [[ -e "$entry" ]] || continue
        name="$(basename "$entry")"
        if [[ "${name,,}" == "$INCLUDE_FOLDER" && -d "$entry" ]]; then
            cp -rs --remove-destination "$entry/." "$MERGED_ROOT/Include/"
            continue
        fi
        ln -s "$entry" "$MERGED_ROOT/$name"
    done

Read it top to bottom and you have the layer order:

  1. The terminal’s standard Include (Trade/, Arrays/, stdlib.mqh, the whole lot).
  2. Every -I directory, in the order you gave them.
  3. Your own Include. --remove-destination means the later layer wins a clash, and yours is last. Patched Trade/Trade.mqh in your tree? Yours gets compiled.
  4. Links to the rest of your MQL4/MQL5 directory (Files, Images and friends), so #resource "\\Files\\data.txt" resolves.

It’s all symlinks (cp -rs) in a staging area, built once per platform and user directory per pass. Nothing gets written into your directories and it’s gone when the run ends. Which MQL4 or MQL5 directory counts as yours is just the nearest one above the source file. The extension alone picks the compiler, .mq4 to MT4’s metaeditor.exe, .mq5 to MT5’s MetaEditor64.exe.

stage_source: a link around the spaces

Your terminal lives in MT5 - SomeBroker? MetaEditor’s /compile and /inc both choke on spaces. So a spaced source gets compiled through a link:

    [[ "$source" =~ [[:space:]] ]] || return 0
    link="${SOURCE_LINKS[$root]:-}"
    if [[ -z "$link" ]]; then
        link="$STAGING_DIR/sources/${#SOURCE_LINKS[@]}"
        mkdir -p "$STAGING_DIR/sources"
        ln -s "$root" "$link"
        SOURCE_LINKS[$root]="$link"
    fi
    STAGED_SOURCE="$link${source#"$root"}"

The link points at the whole MQL4/MQL5 directory, not the file, so "../" includes inside it still resolve. Same reason the merged include root lives in the staging area: a mounted terminal’s directory often has spaces in it.

compile_one: delete first, then don’t trust the exit code

Right before the wine call from the top of this post, there’s this:

    # An old .ex4/.ex5 must never outlive a compile that did not produce a new
    # one. MetaEditor removes it on compile errors, but not when it crashes or
    # writes no result.
    rm -f "$output"

Whatever .ex4/.ex5 is sitting there after the run came out of this run. No stale binary pretending to be fresh after MetaEditor fell over. Files compile one at a time, never in parallel, because two MetaEditors under Wine at once is how you summon demons.

After the call, the verdict comes from the log, not the exit code:

    result="$(grep -E '^Result: ' <<<"$lines" | tail -n 1)" || result=""

MetaEditor’s command-line mode usually returns 1 when everything went fine. Wire that into CI naively and you get a pipeline that fails on success. The Result: line carries the error and warning counts, two sed calls pull them out, and --werror turns warnings into a failure. No Result: line at all means MetaEditor crashed or wrote nothing, and the record carries its raw log lines instead of pretending.

read_log: Windows paths back into yours

MetaEditor writes its log as UTF-16 with Windows paths pointing into a staging area you never asked about. read_log pipes it through iconv, turns Z:\ back into /, and resolves every staging path to the real file behind it with readlink. Your header for a user header, the toolchain’s for a standard one, and the current directory gets dropped off the front. It’s plain bash string replacement, not sed, so no character in a path can break it. The payoff: an error in a header points at the header, not at the EA that included it.

parse_messages: one regex, one JSON record

Every error and warning line in the log looks like path(line,column) : error 256: text, and one jq capture turns them into objects:

        | capture("^(?<file>.*)\\((?<line>[0-9]+),(?<column>[0-9]+)\\) : (?<severity>error|warning) (?<code>[0-9]+): (?<text>.*)$")?

Without --text, stdout is JSON Lines: a file record per source as it finishes, then a summary. Progress goes to stderr, also as JSON lines, so 2>/dev/null leaves nothing but results.

{"type":"file","source":"MQL5/Experts/Scalper.mq5","platform":"mt5","status":"failed","output":null,"errors":1,"warnings":0,"messages":[{"severity":"error","file":"MQL5/Experts/Scalper.mq5","line":41,"column":7,"code":256,"text":"undeclared identifier 'lots'"}]}
{"type":"summary","total":1,"compiled":0,"failed":1,"werror":false}

Every message is {severity, file, line, column, code, text}, which is what an LLM writing EAs actually needs instead of a screenshot of a log pane. --format gnu renders the same record as path:line:column: error: text [code] lines that VS Code, vim and emacs jump straight to. Exit codes are boring on purpose: 0 everything compiled, 1 at least one failed, 2 bad arguments or a setup that couldn’t install.

mql-compiler 2>/dev/null | jq -r 'select(.status == "failed") | .source'

That’s the whole “what broke” report. The repo also ships an agent skill, published to ClawHub on every release, that teaches Claude Code, Codex, OpenClaw and peen to install the command, compile and actually read those records instead of hallucinating that the EA built fine.

The wrapper that calls all of it

None of the above runs until something starts the container. You need Docker on an x86-64 box. There’s no ARM image, because MT4’s MetaEditor needs Wine’s i386 half, so Apple Silicon is out unless Docker emulates amd64 for you.

# per user, lands in ~/.local/bin
curl -fsSL https://raw.githubusercontent.com/psyb0t/docker-mql-compiler/master/install.sh | bash
# system wide, /usr/local/bin
curl -fsSL https://raw.githubusercontent.com/psyb0t/docker-mql-compiler/master/install.sh | sudo bash -s -- --system

The installer pins the newest release image and writes one file, the mql-compiler command, which is a thin docker run. Run it again to upgrade, --rolling pins :latest instead, --uninstall removes the command and its image. The cache volume stays because every user on the box shares it.

mql-compiler                            # every .mq4 and .mq5 under the current directory
mql-compiler MQL5/Experts/Scalper.mq5   # one source
mql-compiler --text MQL5 MQL4           # both platforms, human output
mql-compiler --watch MQL5/Experts       # recompile on save until Ctrl-C
mql-compiler --werror MQL5              # warnings fail the run
mql-compiler -I ~/libs/mql5 MQL5        # shared headers outside your tree
mql-compiler --mt5-terminal ~/mt5 MQL5  # use a portable terminal already on disk
mql-compiler --setup                    # install the toolchain, compile nothing
mql-compiler --version                  # image version and MetaEditor builds
mql-compiler --update                   # reinstall from the newest installers

Keep sources under a directory called MQL4 or MQL5, same layout as a terminal’s data directory, headers in its Include. Arguments can be files or directories, relative or absolute, anywhere on the box, mixed platforms. The wrapper mounts each needed directory at the same path inside the container and starts one container for the whole run. Run it bare from / and it refuses, because scanning your entire disk for EAs is a stupid idea.

–watch

--watch holds the container open, compiles, then sits on inotifywait and runs another full pass whenever a .mq4, .mq5 or .mqh changes under your paths or an -I directory. Between passes it lets go of the cache lock, so a one-off compile from another terminal still gets its turn. Ctrl-C exits 130, cleanly.

–mt4-terminal and –mt5-terminal

Your broker’s terminal runs a specific MetaEditor build and you want exactly that one? Point these at a portable terminal directory already on disk and compile_one runs its MetaEditor against its standard library. That platform never gets downloaded, the terminal is mounted read-only, and leaving the flag off next time puts you back on the cached build. The terminal must have been started once so it unpacked its headers, otherwise it’s exit 2 before anything compiles. Since the cached terminals are moved into the same portable shape, there’s no difference between mine and yours.

The cache keeps whatever MetaEditor build it installed first, forever. --version tells you which builds you’re on, --update reinstalls from the newest installers, and if that fails the old toolchain comes back untouched.

The same entrypoint in a GitHub runner

- uses: actions/checkout@v6
- uses: psyb0t/docker-mql-compiler@master
  with:
    paths: |
      MQL5/Experts
      MQL4/Indicators/Bands.mq4

Those JSON records are what the Action reads. Compile errors fail the job and land as annotations on the exact source lines in the PR. Inputs are paths, include, werror, image, cache and cache-dir. With cache on (the default) the 2 GB toolchain sits in the Actions cache, so only the first run eats the 90-second install. Outputs are results (the JSON Lines file), total, compiled and failed, and the job summary gets a table of every file. An EA that doesn’t compile never reaches main again, and nobody touched Windows to find out.

The regex that got its own fix commit

The first release shipped without it. The very next fix commit in the history exists for this one line:

# osslsigncode's words when it could not get a CRL (its own typo included).
readonly CRL_FETCH_FAILURE="Fai(le)?d to (decode|download|get|fetch) CRL"

On GitHub’s runners, osslsigncode’s fetch of the Sectigo revocation list came back truncated, and every first run failed the signature check on a perfectly good MetaQuotes installer, the v0.1.0 test job included. Retrying that without also retrying a genuinely bad signature meant matching osslsigncode’s error text, which on that path reads “Faild to decode CRL”. That (le)? is the most honest piece of code in the repo, a regex that bends to someone else’s typo so the check doesn’t have to bend at all. And the whole pipeline it guards compiles your EA. It doesn’t make your EA any good. If it builds clean and then martingales your account into the ground, that one’s on you.

Code’s here: github.com/psyb0t/docker-mql-compiler. WTFPL, do what the fuck you want.