Java target

The Java backend (packages/compiler/src/backends/java) turns the IR into Java 21 sources that use the runtime in packages/runtime-java (package org.yafflelang.runtime, no dependencies; Maven coordinates org.yafflelang:yaffle-runtime). Generated code and the runtime compile with javac -Xlint:all -Werror.

Generated API

Every struct, enum and union type becomes one file in one package (targets.java.package, default yaffle.gen). An exported struct also gets the root API:

// export struct Doc(u32 version) { … }  →  yaffle/gen/Doc.java
Doc d = Doc.parse(bytes, new Doc.Options().version(2));     // Options: endian, strict, root params
byte[] out = Doc.serialize(d, new Doc.Options().version(2));
SafeResult<Doc> r = Doc.safeParse(bytes, options);          // Ok(value) | Err(YaffleException)
CompletableFuture<Doc> f = Doc.parseAsync(bytes, options);  // also serializeAsync, parseAsync(AsyncSource)
Object json = Doc.toJson(d);                                // Map/List/String/Long/Double/Boolean/null
Doc back = Doc.fromJson(json);
Doc.Options o = Doc.Options.fromJson(argsJson);

Doc.View v = Doc.view(bytes);                               // also view(Source, options)
v.getEntries().get(3).setAlign(0x200);                      // same size: patched in place
v.getEntries().add(Entry.View.of(newEntry));                // structural edit
List<Patch> ps = v.patches();                               // [{at, remove, insert}]
byte[] committed = v.commit(Layout.Grow.SPLICE);            // DEFAULT | APPEND | SPLICE | ERROR

AsyncView<Doc.View> av = Doc.viewAsync(AsyncSource.of(asyncChannel), options);
long size = av.get(w -> w.getEntries().get(3).getSize()).join();
av.set(w -> w.getEntries().get(2).setHandler(5)).join();
byte[] out2 = av.commit().join();

Schemas with an async codec get parseAsync and serializeAsync instead of parse, safeParse and serialize. parse and serialize without options use the defaults. Errors are YaffleExceptions with a stable ErrorCode, a path (field names and indices) and a byte offset (-1 on write).

Struct classes are public final with public fields: one class is both the parse result and the serialize input (derived and computed fields are filled by parse and ignored by serialize). They implement YValue (toJsonValue(), what == on composite values compares) and have a nested View class. @hidden fields are package-private $name fields.

Type mapping

Switch cases flatten into the struct’s fields (nullable), like the canonical JSON. A field with different Java types in different branches is an Object. A conditional nullable pointer has a presence flag, so a taken branch with a null pointer is null in JSON and an untaken one has no key.

Names: keywords get a _ suffix (open-enum constants named value/member too). User types keep their names; generated code refers to JDK, java.lang and runtime classes by simple name only when no user type shadows them, and fully qualified otherwise. The options class is Options, or Options_ if a user type is called Options.

Generator structure

Java lambdas only capture effectively final locals, so generated code keeps mutable state in objects: the error path in a Frame, read-side extents in an int[], write-side extents in Extent objects created up front.

Runtime structure

Serialization encodes bottom-up into blocks with fixups; each base region is laid out when it is complete, then assembled, then the fixups run. A value decoded through a codec remembers its encoded bytes, so serializing it unchanged reuses them (byte-exact rebuilds of lossy codecs).

Views and async

A generated scan method reads a struct’s scalars into a plain object and records byte ranges; composite fields become view values: child nodes, ListViews, lazy bytes, lazy pointer targets, codec regions decoded on access and lazy streams. Fields that later expressions need are read eagerly. Setters patch the source in place when the new encoding has the same size and nothing depends on the field; anything else is a structural edit. commit() returns the in-place patches directly when it can. Otherwise it turns the view into a plain value (unchanged parts read from the source), re-encodes it with the generated writer under the sticky layout (targets keep their positions where the grow policy allows) and diffs the result into patches.

Byte spaces can be lazy: a Store loads 64 KiB blocks of a Source or AsyncSource, decodes a @ranged codec region block by block, or decodes the pieces of an arr[*].field stream only when a read touches them (when the pieces’ lengths are known from size fields).

Generated readers and writers are synchronous. parseAsync, serializeAsync and the accesses of an AsyncView run on virtual threads; an async codec’s future or a missing block is awaited there, which parks the virtual thread. Accesses of one async view run in order. The typed view (av.view()) can also be used directly; reads then wait on the calling thread.

Host interfaces

CodecCtx carries the arguments (boxed: u8 → Integer, u64 → Long bit pattern, u8[n] → byte[]) and outputSize() (-1 when unknown). Host exceptions become CODEC errors.

targets.java.externs in yaffle.json maps extern names to path/file.java[#ITEM] (the file is copied into the output under its public class’s name; ITEM defaults to the extern’s name) or to a class on the classpath, com.example.Host[#ITEM]. Relative paths resolve against targets.java.externsRoot when set. targets.java.dependencies lists Maven jars ({"io.airlift:aircompressor": "2.0.2"}) the conformance adapter and tests download.

The conformance externs are in conformance/externs/java.

Tests

JDK 21 comes from JAVA_HOME, PATH or nix shell nixpkgs#jdk21 (its location is cached). YAFFLE_JAVA_CACHE (default $TMPDIR/yaffle-java-cache) holds the compiled runtime, keyed by a hash of its sources, and downloaded jars; YAFFLE_JAVA_TEST_DIR (default $TMPDIR/yaffle-java-tests) holds the test programs. Use private directories when several checkouts run tests at once.

# conformance, parse and view mode (java, java:view)
node --max-old-space-size=12000 --conditions=development conformance/src/cli.ts --targets java
# backend tests: API, views, lazy and async views, IR features, a view property test,
# -Werror compilation of every conformance schema, and the runtime unit tests
npx vitest run --project yaffle packages/compiler/test/backends/java
# runtime unit tests on their own (a self-contained runner, no JUnit)
nix shell nixpkgs#jdk21 --command packages/runtime-java/build.sh test

The conformance adapter generates the case directory’s program, a harness that runs every case in one JVM, compiles both with -Xlint:all -Werror and judges what the harness observed. In view mode the harness reads every field through the view (ViewRead.materialize) and commits without edits; { file } cases go through viewAsync over an AsynchronousFileChannel.

Known gaps

• Piece streams whose piece lengths aren’t known from size fields decode every piece on first access.
• A from stream over a single bytes field forces the whole field (a @ranged source region is still decoded block by block, but all of it).
• Derived and computed fields of an edited view node are stale until commit(); computed fields can’t read derived fields of array elements.
• Fields of the same Java type but different JSON rules in different branches (u32 and u64, both long) use the first branch’s JSON rule.
• In views, edits inside codec regions and streams are always structural, and getters of eagerly read fields return detached views.
• A layout on the root struct of a codec’s decoded side or a stream is ignored at that level.
• Unions are named after their variants (AOrB): the IR has no union names.
• yaffle build passes extern paths relative to the output directory without the directory itself; the backend falls back to the current directory and its ancestors (or targets.java.externsRoot).