fff's file finder library in Tellop's engine: its source, and how to rebuild it with a modified xdiff
====================================================================================================

Tellop (by Pathnomic Labs FZ-LLC) carries an engine, OpenCode v1.19.0 built
with Bun 1.3.14 into one program for Macs with Apple silicon. The engine embeds
the native library of fff 0.9.4, a file finder (github.com/dmtrKovalenko/fff,
tag v0.9.4, commit 7d7910b6ba78ea8406671eaac8cdb5eb9850d9d1): the file
libfff_c.dylib of the npm package @ff-labs/fff-bin-darwin-arm64 0.9.4 (SHA-256
7e371b5d655e6737ed24147cdcb477ab6e362b338de9b0f7aaf785b6636fa6cd), built by
fff's own release workflow.

That library statically links libgit2 1.9.2, as the Rust crate libgit2-sys
0.18.3+1.9.2 compiles it, and with it xdiff, libgit2's diff code (its folder
deps/xdiff), licensed under the GNU Lesser General Public License, version 2.1
or later. libgit2's linking exception does not cover xdiff. This archive is
the means to build the library again with a modified copy of xdiff and to
build the engine with that library: it is used together with the engine's
source kit, engine-89e502e3-tellop-source.tar.gz (SHA-256
7e9d47f6489928e6e48458c14701cdeddcd1573b9a2a15c7193353886c778c16), published
at https://updates.tellop.ai/source/engine-89e502e3-tellop-source.tar.gz,
which holds the engine's source and builds it.

Pathnomic Labs made no changes to fff, to libgit2, to xdiff or to any crate
here, and has not run the steps below (see section 6).


1. What is in this archive
--------------------------

  README.txt
      This file.

  relink-fff.sh
      The steps below, as a script.

  relink-engine-notes.txt
      Corrections to the engine's source kit's relink-engine.sh, which
      relinks the engine's other LGPL libraries (JavaScriptCore and TinyCC):
      that kit is published and never changes, so they travel here.

  fff/fff-v0.9.4.tar.gz
      fff's source at tag v0.9.4, as GitHub archives it (codeload.github.com,
      SHA-256 6c7d7624a636e9a751f7861abac248c0901d5aec6ef55dbf558294e5756c1c7e;
      its header names commit 7d7910b6ba78ea8406671eaac8cdb5eb9850d9d1). Its
      Cargo.lock (SHA-256
      69161124f1392810cdfd50347125cd0e12a41a4f3497a3585af503a13e7b5514)
      names every crate the build uses.

  crates/
      Every one of the 324 crates.io crates that Cargo.lock names, each the
      .crate file crates.io serves, with the SHA-256 the lockfile records for
      it. Among them, crates/libgit2-sys-0.18.3+1.9.2.crate (SHA-256
      c9b3acc4b91781bb0b3386669d325163746af5f6e4f73e6d2d630e09a35f3487)
      carries libgit2 1.9.2's source in its folder libgit2/, and xdiff in
      libgit2/deps/xdiff/; crates/zlob-1.4.1.crate carries zlob's Zig source
      (zig-src/, build.zig), which its build script compiles with Zig.

  licences/libgit2-COPYING.txt
      libgit2's COPYING, unchanged from libgit2-sys's libgit2/COPYING: the
      terms of libgit2 and of the code it bundles. The text of the GNU Lesser
      General Public License, version 2.1, is its lines 462 to 963: it
      follows the section on the bundled winhttp files (lines 441 to 458),
      which points to it, and the licences of other bundled code follow it.
      xdiff's own notice of that licence heads the file
      libgit2/deps/xdiff/xdiff.h (lines 1 to 21) in
      crates/libgit2-sys-0.18.3+1.9.2.crate.

  licences/fff-LICENSE.txt
      fff's own licence (MIT), unchanged from its LICENSE.

  SHA256SUMS
      The SHA-256 of every other file here. Check them with:
          shasum -a 256 -c SHA256SUMS


2. What you need first (not included)
-------------------------------------

  - A Mac with Apple silicon, and Apple's Command Line Tools
    (xcode-select --install, or https://developer.apple.com/download/all/):
    clang and a macOS SDK, which the crates' build scripts compile C with;
    libclang, which zlob's build script reads its C header with (through the
    bindgen crate; set LIBCLANG_PATH to its folder if it is not found); and
    codesign. The library Tellop's engine carries records macOS SDK 15.5.
  - Rust 1.96.0, the compiler the library Tellop's engine carries was built
    with (it names the Rust source commit
    ac68faa20c58cbccd01ee7208bf3b6e93a7d7f96, Rust 1.96.0 of 2026-05-28):
    install rustup from https://rustup.rs, then
        rustup toolchain install 1.96.0
  - Zig 0.16.0, as zig on PATH, from https://ziglang.org/download/ (the
    aarch64-macos build). zlob's build script runs `zig build`, and fff's own
    build script checks that zig is on PATH.
  - For the engine (step 4): the engine's source kit named above, and the
    folder it builds in, /opt/tellop-engine, which an administrator makes
    once:
        sudo mkdir -p /opt/tellop-engine && sudo chown "$USER" /opt/tellop-engine && chmod 700 /opt/tellop-engine

Allow several GB of free space: the engine's build alone takes about 3 GB
(its kit's README). Nothing is read from the network: Cargo runs offline,
against this archive's crates.


3. The steps
------------

From this folder, with a work folder of your own (outside your home folder
is safest: Cargo also reads any .cargo/config.toml in a folder above the one
it builds in, such as ~/.cargo/config.toml, which must set nothing for this
build). Give a new or empty folder: the steps delete and remake folders
inside it, so the script refuses this kit's folder or a folder holding it,
and a folder that holds files but not the file .relink-fff-work, which it
writes there on first use. It also refuses a work folder whose path holds a
double quote, a backslash or a line break, as it writes that path into
Cargo's settings file as it is.

  Step 1. Lay out the source.

        sh relink-fff.sh prepare <work folder>

      It checks every file against SHA256SUMS, unpacks fff's source into
      <work folder>/fff, checks each crate against the checksum fff's
      Cargo.lock names for it, unpacks each into
      <work folder>/vendor/<name>-<version>/ with a .cargo-checksum.json of
      its own, and writes <work folder>/cargo-home/config.toml, which tells
      Cargo to take every crate from <work folder>/vendor and never to use
      the network. Running step 1 again deletes and remakes all of these,
      so it discards any change you made to xdiff: keep a copy of your
      changes outside the work folder.

  Step 2. Change xdiff.

      xdiff is <work folder>/vendor/libgit2-sys-0.18.3+1.9.2/libgit2/deps/xdiff/.
      libgit2-sys's build script compiles every .c file in that folder (its
      build.rs: add_c_files(&mut cfg, "libgit2/deps/xdiff")), so a file you
      add there is compiled too.

      Cargo refuses a vendored crate whose files differ from the list in its
      .cargo-checksum.json. Step 1 writes each crate's as
          {"files":{},"package":"<the checksum fff's Cargo.lock names>"}
      that is, with an empty list: Cargo then checks no single file, and
      accepts your changes. Leave "package" as it is, and leave fff's
      Cargo.lock unchanged: Cargo compares the two and refuses a difference,
      and step 3 builds with --frozen. (If you lay the crates out with
      `cargo vendor` instead, which lists every file, delete the entries of
      the files you changed from the "files" list of
      vendor/libgit2-sys-0.18.3+1.9.2/.cargo-checksum.json, or make that list
      empty.)

  Step 3. Build the library.

        sh relink-fff.sh library <work folder>

      It checks for Rust 1.96.0, Zig 0.16.0 and the Command Line Tools,
      removes any earlier build (so a changed xdiff is always compiled), and
      runs, in <work folder>/fff:

        CARGO_HOME=<work folder>/cargo-home RUSTUP_TOOLCHAIN=1.96.0 \
        MACOSX_DEPLOYMENT_TARGET=13 CI=true \
        cargo build --frozen --profile ci --target aarch64-apple-darwin -p fff-c --features zlob

      with these unset, as each would change the build: every variable whose
      name begins CARGO_ (Cargo takes any of its settings from one, such as
      CARGO_PROFILE_CI_LTO, CARGO_TARGET_DIR, CARGO_BUILD_TARGET or
      CARGO_ENCODED_RUSTFLAGS; CARGO_HOME is then set as above), RUSTFLAGS,
      RUSTC, RUSTC_WRAPPER, RUSTC_WORKSPACE_WRAPPER, ZIG, and
      LIBGIT2_NO_VENDOR (section 4). The variables for C code (CC, CFLAGS,
      AR, SDKROOT and the like, which the cc crate and the compiler read) are
      left to you: leave them unset for the build that matches Tellop's.

      After the build it checks that the library holds xdiff as compiled
      here: otool -L must name no libgit2 among the libraries it loads, and
      nm must list "t _xdl_diff" (xdiff's entry point, defined inside the
      library), as both hold for the library Tellop's engine carries;
      otherwise it stops. It then copies
      target/aarch64-apple-darwin/ci/libfff_c.dylib to
      <work folder>/c-lib-aarch64-apple-darwin.dylib, signs it ad hoc as fff's
      release does (codesign --force --sign -), and copies it to
      <work folder>/libfff_c.dylib. Section 5 says where each setting comes
      from.

  Step 4. Build the engine with it.

        sh relink-fff.sh engine <work folder> <engine-89e502e3-tellop-source.tar.gz>

      It checks the engine's source kit by its SHA-256 and unpacks it into
      <work folder>/engine-kit (it checks every file against that kit's own
      SHA256SUMS), then changes three things in that copy, and nothing else:

        - your library is stored as engine/installed/<its SHA-256>;
        - in engine/installed.tsv (SHA-256
          5dc8d3293c147b40ff48ebdbffe22ba603a4f897370dbe933240bb5b20599de7,
          the packages the engine's build uses), the one line for
          node_modules/.bun/@ff-labs+fff-bin-darwin-arm64@0.9.4/node_modules/@ff-labs/fff-bin-darwin-arm64/libfff_c.dylib
          gets your library's SHA-256 and size in place of
          7e371b5d655e6737ed24147cdcb477ab6e362b338de9b0f7aaf785b6636fa6cd
          and 4822976;
        - build.json's "installed" record gets that listing's new SHA-256.

      The kit's own tool, tools/rebuild-engine.mjs, run with the kit's own Bun
      (bun-runtime/bun-darwin-aarch64-1.3.14.tgz), then checks every input
      against build.json as it always does (the listing by its SHA-256 and
      every stored file by the SHA-256 it is named by), lays the engine's
      tree out in /opt/tellop-engine/opencode-v1.19.0 with your library at
      that path, builds the engine and writes it to
      <work folder>/engine/opencode. It runs with --relinked, the option the
      kit's relink-engine.sh uses, so it writes an engine that is not the one
      Tellop ships rather than refusing it (its message then speaks of a
      relinked Bun; here the Bun is the kit's own and the library is yours).
      The engine embeds the library unchanged, and the script checks that
      the engine carries your library byte for byte.

The rebuilt library is not byte for byte the one Tellop's engine carries,
even with xdiff unchanged: a library records the folder it was built in (the
one the engine carries names /Users/runner/work/fff/fff, GitHub's build
folder) and the SDK, and is then signed. Tellop checks that the engine it runs
is the engine it shipped. These scripts build an engine; they do not change
an installed Tellop.


4. What the build scripts do
----------------------------

Pathnomic Labs searched the build script (build.rs) of every crate here for
anything that downloads, and read those of the crates that compile code. None
of those that run for this library reads the network:

  - libgit2-sys builds libgit2 1.9.2 from its own libgit2/ folder (fff's
    Cargo.toml asks for git2 with the vendored-libgit2 feature): libgit2's
    sources, xdiff, PCRE and llhttp, with clang through the cc crate. It
    would run `git submodule update --init libgit2` only if its libgit2/src
    were missing; the crate carries it. One variable overrides the
    vendored-libgit2 feature: with LIBGIT2_NO_VENDOR set (to anything but
    0), its build.rs links a libgit2 installed on the Mac (1.9.2 or a
    later 1.9.x, found through pkg-config) and compiles neither libgit2
    nor xdiff, so your changes to xdiff would silently not be built. The
    script unsets it for the build, and stops if the library loads a
    libgit2 or lacks xdiff's xdl_diff.
  - libz-sys links the zlib macOS provides (/usr/lib/libz.1.dylib, as the
    library Tellop's engine carries does), after compiling a small test
    program against it. Its build.rs first asks pkg-config for "zlib":
    if pkg-config finds another zlib (Homebrew's, say), Cargo is told to
    search that zlib's folder, and the library may load that zlib rather
    than macOS's. To keep macOS's, set ZLIB_NO_PKG_CONFIG=1 before step 3
    (export ZLIB_NO_PKG_CONFIG=1): the pkg-config crate 0.3.32, which
    libz-sys asks through, then does not run pkg-config for zlib (its
    src/lib.rs, in probe). Afterwards, otool -L on
    <work folder>/libfff_c.dylib should name /usr/lib/libz.1.dylib.
  - lmdb-master-sys compiles LMDB from its own copy; blake3 compiles its C
    code for the Arm processor; both with clang through the cc crate.
  - zlob reads its C header with bindgen (which loads libclang) and runs
    `zig build` on its own Zig source, with Zig's caches inside Cargo's build
    folder. Its build.zig.zon names no dependencies, so Zig fetches nothing.
  - fff-search (fff's crates/fff-core) checks that zig is on PATH.
  - The others only ask rustc its version, or set flags for Cargo.


5. Where each setting comes from
--------------------------------

  - Target aarch64-apple-darwin, profile ci, package fff-c, feature zlob,
    MACOSX_DEPLOYMENT_TARGET=13: fff's .github/workflows/release.yaml at the
    tag, job build-c, step "Build for macOS", on the row whose npm_package
    is fff-bin-darwin-arm64:
        MACOSX_DEPLOYMENT_TARGET="13" cargo build --profile ci --target ${{ matrix.target }} -p fff-c --features zlob
    The library the engine carries names target/aarch64-apple-darwin/ci/ and
    records macOS 13.0 as its least version.
  - The ci profile: fff's Cargo.toml, [profile.ci] inherits [profile.release]
    (opt-level 3, codegen-units 1, strip "debuginfo") with lto "thin".
  - The linker's -undefined dynamic_lookup: fff's .cargo/config.toml, for
    every macOS target. Cargo reads it from fff's folder by itself;
    RUSTFLAGS would replace it, so the script unsets it.
  - Signing: release.yaml's step "Ad-hoc sign macOS binary",
    codesign --force --sign - "c-lib-${{ matrix.target }}.${{ matrix.ext }}",
    then "Prepare npm package" copies that file into
    packages/fff-bin-darwin-arm64/libfff_c.dylib. The library the engine
    carries is signed ad hoc, as c-lib-aarch64-apple-darwin.
  - CI=true: GitHub sets it for every workflow run. zlob's build.rs then
    builds its Zig code for the plain aarch64-macos target rather than for
    the building Mac's own processor.
  - Rust 1.96.0: fff's rust-toolchain.toml says only "stable"; the library
    the engine carries names Rust 1.96.0's commit (section 2), so the script
    names the version (RUSTUP_TOOLCHAIN).
  - Zig 0.16.0: release.yaml installs it (mlugg/setup-zig@v2, version:
    0.16.0), and zlob 1.4.1's build.zig.zon names it as the least it builds
    with.
  - The engine: the engine's source kit (its README.txt and build.json).


6. Not run by Pathnomic Labs
----------------------------

These steps are written from the sources above and have not been run:
Tellop's founder decided against installing these compilers on Tellop's
release Mac. The library step is the build fff's own release runs with the
same settings: its run for tag v0.9.4 (GitHub Actions run 27240394393,
https://github.com/dmtrKovalenko/fff/actions/runs/27240394393) built the very
library the engine carries, as its npm package and its release asset
c-lib-aarch64-apple-darwin.dylib (both SHA-256
7e371b5d655e6737ed24147cdcb477ab6e362b338de9b0f7aaf785b6636fa6cd). The engine
step is the engine's source kit's own build, which Pathnomic Labs did run,
with the library put in its place. Read the script before you run it; a step
may need adjusting on your Mac.

Questions: support@pathnomic.com.
