Modules: include vs import​

Split your scripts into files without inventing a build system. include is textual composition: it runs the file's contents in your scope every time, may return a value, and accepts dynamic path expressions. import is a module declaration, cached per engine and re-run only when its file changed: the statement form yields nothing, the call form import("f.jai") is an expression that says whether the module ran. That timestamp check is exactly the hook hot reload hangs off. Both resolve through one ladder: the asking file's folder, then the host's add_include_path roots, then the working directory; a host can confine it to chosen folders (Safety Rails). And both are visible to the checker before anything runs, so a multi-file program is checked as one program (below).

include: runs every single time​

include "file.jai" (or include <file.jai>) executes the file's contents in the current scope, every time it is evaluated. Definitions land in your scope; side effects repeat on every include:

include "mod_math.jai";
print("mod_add(2, 3) = {}", mod_add(2, 3));
print("MOD_PI = {}", MOD_PI);
auto load_count = 0;
include "mod_count.jai";
include "mod_count.jai";
include "mod_count.jai";
print("include runs EVERY time: load_count = {}", load_count);

with mod_math.jai:

function mod_add(int a, int b) -> int { return a + b; }
auto MOD_PI = 3.14159;

and mod_count.jai:

load_count = load_count + 1;

mod_add(2, 3) = 5
MOD_PI = 3.14159
include runs EVERY time: load_count = 3

Includes nest (an included file can include another), and the # prefix is accepted and ignored for C++ muscle memory: #include "x.jai";.

include in expression position returns the file's value: var cfg = include "config.jai"; evaluates to the included file's result (its last top-level expression), which makes data-file configs a one-liner. Statement-position include still discards the value.

An error in an included or imported file is an error at the line that reached it. A throw arrives with its value and type (as e.value), a run-time error with its own text, and a file that does not parse or check raises its listing, named by the file's path. A try around an include catches every one of them, and the program goes on. Uncaught, the stack trace lists the file's own frames first, then the frame of each include or import that reached it.

Dynamic paths​

Both may appear wherever a statement may: in a function or lambda body, a block, a loop, a conditional, a coroutine. An include runs where it is written (its declarations are that scope's). An import takes no scope: wherever it stands, the module's declarations are program-scope declarations. Directly in a class or a namespace body an import is a parse error, since there it would read as scoped.

The two differ in where the declarations land. An include is text pasted where it stands, so it runs in the current scope under the current file: inside a function its names are that function's and end with it. An import is a file of its own, one unit however many places import it, so its names are the program's wherever the statement stands:

void load_rules() {
    include "tuning.jai";     // textual: SPEED is a local of load_rules
    print("inside load_rules: SPEED = {}", SPEED);
}
load_rules();

void load_physics() {
    import "physics.jai";     // its own unit: gravity() is declared at program scope
}
load_physics();
print("after load_physics: gravity() = {}", gravity());

with tuning.jai:

int SPEED = 3;

and physics.jai:

float gravity() { return 9.81; }

inside load_rules: SPEED = 3
after load_physics: gravity() = 9.81

An include inside a function body is that function's own text. A literal path is pasted in when the function is parsed, the way C++ pastes an #include: the file reads and writes the function's parameters and locals, the variables it declares are locals, and a function it declares is a nested function of the body, callable below the include and gone when the body returns (like every nested named function it captures nothing: the body's locals are a parse error inside it). A program-scope function of the same name is hidden inside the body and stays as it was everywhere else; the strict checker warns about that one, since a reload is usually what was meant (function 'helper' from the include in function 'reload' is local to it and hides the program-scope 'helper' there without replacing it; import the file to replace program-scope functions). A class is always a program-scope type: one written in a function, method or lambda body is a parse error (a class cannot be declared inside a function body (a class is a program-scope type); declare it at program scope or import its file), and so is one a file included inside a function body declares: an include inside a function body cannot declare class 'Rules' (a class is a program-scope type and the included text is the function's own); import 'rules.jai' instead. To replace program-scope functions and classes from inside a function, import the file: an import is program scope wherever it stands.

The call form takes any string expression, both for include and import:

auto libname = "mod_" + "math" + ".jai";
include(libname);
mod_add(20, 22);

== result: 42

import: cached module declaration​

import "file.jai" executes the file at most once per engine (subject to the mode below). Repeating the import is free:

auto load_count = 0;
import "mod_count.jai"
import "mod_count.jai"
import "mod_count.jai"
print("import caches (default: re-run only if the file changed): load_count = {}", load_count);

import caches (default: re-run only if the file changed): load_count = 1

The statement form yields nothing (auto x = import "mod_math.jai"; does not parse). The call form import("file.jai") is an expression: true when the module, or a module it imports, ran on this reach, false when it had nothing to do:

bool first = import("mod_math.jai");
bool again = import("mod_math.jai");
print("first reach ran it: {}, the next one: {}", first, again);

first reach ran it: true, the next one: false

Imports may form cycles. An import of a module that is still running, one that imported this file directly or through others, does nothing, and the file the program started with counts as one: a module that imports the root file never runs it a second time. Code after an import sees everything the module declared, whichever way the cycle closes.

Reloading from a running program​

A running program reloads its modules by reaching their imports again, and the loop that does it guards the import: a rejected reload raises a catchable error at the import, and the program keeps its old code. An unguarded import in a loop ends the run at the first mistake in an edited file. This is the guarded reload a game loop makes once per frame (the group form is described below):

array<string> MODULES = ["rules.jai", "monsters.jai", "items.jai"];
void tick() {
    if (DEV_MODE) {
        try {
            if (import(MODULES)) { print("reloaded"); }
        } catch (e) {
            print("reload failed: {}", e);   // the program keeps the code it had
        }
    }
    step_world();
}

An import re-checks its file each time it is reached, so a running program reloads the modules it is working on by reaching their imports again. A game loop that keeps its module list in one file includes that list where the reload should happen:

import "rules.jai";
import "monsters.jai";

void tick() {
    if (DEV_MODE) {
        try {
            include "modules.jai";   // each import re-runs only a module whose file changed
        } catch (e) {
            print("reload failed: {}", e);
        }
    }
    step_world();
}

An unchanged module costs its stamp check: one per file of its closure, the module and every file its last run included (at the top level, nested, or spliced into a function body), so editing only an included file re-runs the module. The check goes on through the modules that module imported, and the modules they imported, each checked once per pass with its own includes: a changed one re-runs on its own, with the same checks first, and the module that imported it does not, so while (running) { import("game.jai"); } picks up an edit anywhere in the program. The modules checked this way are the ones the importing module's last run actually imported, by the file each path resolved to: an import inside an if counts only when its branch ran, a computed import(p) counts as the file p named. When the module itself re-runs the list is rebuilt from that run; a module it no longer imports stops being checked from there (it stays loaded, since imports never unload). A loaded module whose file disappears raises Failed to open import file once and then waits quietly, keeping the code it had, until a check finds the file again and re-runs it with the usual checks first; a file that never loaded raises every time it is reached. A touched one is parsed and checked, together with the modules it imports, before any of it runs: a mistake raises a catchable error naming the file and line, and the program keeps the code it had. The check covers the code that uses it too: every unit the program has loaded (the root file, the other modules, an executed string that declared functions, classes or variables) is checked again against the new declarations, never run again, and a reload that would break code which is not changing, such as a field dropped while a function elsewhere still reads it, is rejected the same way, the listing naming that code's file and line (root.jai:12:41: error: no member 'a' in class 'C'). Code that is changing counts with its new text: when the file that reads the field has been edited too and its import is not reached yet, the reload is checked against that edit, and each module still re-runs at its own import. A rejected reload waits until a file it involved changes: the module, a file it includes, or the code it would break, so editing either side tries it again. When the checks pass the module runs as its own unit: its functions and classes replace the old ones in place (live instances migrate, callers see the new bodies), and its top-level statements run again. A variable it declares with an initializer (int spawn_rate = 3;) takes the new value; one declared without (int kills;, array<Item> pack;) keeps the value it holds, unless its declared type changed, which resets it to the new type's default. So state lives in a reloaded module simply by being declared without an initializer.

An import in a game loop costs nanoseconds, and a frame sees all of its edits or none. Asking the filesystem for a module's stamp takes microseconds, so imports check in passes. At most once per import_recheck_interval (0.2 seconds by default) the first import reached opens a check pass, and every import reached during the pass asks for its modules' stamps, each module once. The pass closes when an import reaches a module it already checked in this pass (the frame loop came around) or when the call from the host returns. Between passes an import does nothing: it reads no clock and asks the filesystem nothing, it compares the engine's clock sample against the window and branches. The engine samples the clock where it reads it anyway: at each call from the host, every 1024 loop iterations and calls (the execution budget's check, budget or not), and whenever a time:: function reads the system clock, so a frame loop that waits with a blocking sleep refreshes it every frame, as does every resume of a suspended run. The window runs on the real clock: a host's virtual time source never moves it. One frame therefore checks every import it reaches and the frames after it check none, so a touched file is picked up within about 0.2 seconds, and never half of a set of edits in one frame and the rest in the next. import_recheck_interval(seconds) sets the window from a script (the host spells it engine::import_recheck_interval), import_recheck_interval() reads it, and 0 checks on every reach, whichever import reaches the module, so an edit is seen at the next reach even inside the same run. A loop that keeps reaching its imports inside one long run sees an edit once the window has passed: the engine's clock sample moves with every time:: read of the system clock and with the execution budget's periodic samples. A module that has never run loads when it is first reached, pass or not.

Modules that must change together go in one import: import("rules.jai", "monsters.jai"), or import(MODULES) with an array of paths. The group is one decision. Every changed module is read, parsed and checked, together with the modules it imports and knowing the names the others declare, before any of them runs: a mistake anywhere is one catchable error and nothing is applied, and the edits then wait until a file involved in the failure changes again. When every check passes the changed modules run in the listed order, and the value is true when any of them ran, which is what the guarded loop at the top of this section prints on.

A reload trigger that fires now and then (an F5 key, an editor's ping) asks for a pass at once with the jai::force directive, the last argument of the call form:

if (key_pressed(F5)) { import(MODULES, jai::force); }   // checks now: touched files re-run, unchanged ones do nothing

A forced import opens a pass whatever the window: a moved stamp re-runs its module with the usual checks first, an unchanged file does not re-run. import("rules.jai") loads exactly like import "rules.jai"; and also yields whether the file ran (true when it ran, false when the import had nothing to do), and a directive other than jai::force fails the load with a parse error naming it.

A file's own reload hook​

A file may declare, at its top level, void hot_reload_migrate(map<string, var> previous). When that same file runs again (an import that re-runs it, execute_file of the same file, the same string executed again, a file included again by the same unit), the hook runs once, after the re-run's top-level code, with every program-scope variable the file declared as it was before the re-run, including one the new text removed, retyped or initializes again. It never runs on a first load. This is how a reload converts data whose shape changed:

map<string, int> scores = {};             // was: array<int> scores = [12, 40, 7];

void hot_reload_migrate(map<string, var> previous) {
    array<int> old = previous["scores"];  // the array the last run left
    for (int i = 0; i < old.size(); i++) { scores["player" + to_string(i)] = old[i]; }
}

The hook belongs to its file, not to the program: it is no global function, nothing can call it (a call to hot_reload_migrate is an undefined name, at load and at run time), two files each have their own, and each run takes it from that run's text, so a run whose text no longer declares it has none. At a file's top level the name is reserved for the hook, and any other declaration of it there is a load error naming the signature. An error inside the hook is caught like a run-time error in the re-run: the program stays as the re-run left it. The snapshot is taken only when the new text declares a hook.

Import modes (engine::import_behavior)​

Set with engine->set_import_behavior(engine::import_behavior::once);.

Invalidation and tracking (host API)​

eng->add_include_path("scripts/");        // search roots; FIRST added path wins on collisions
eng->reset_import(canonical_path);        // forget one module -> next import re-runs it
eng->reset_imports();                     // forget all
eng->is_imported(canonical_path);         // query (canonical/absolute path)
eng->get_imported_files();                // list

With filesystem access enabled (the default), a path resolves the same way everywhere a file is named: import, include, the checker's gather across files, and the resolve_path builtin all walk one ladder. An absolute path stands or falls as given. A relative one is tried against the folder of the file doing the asking first (a module imports its sibling by bare name, wherever the program was started from), then against each include path in the order they were added, then against the working directory. The first hit wins and its canonical spelling is the key the unit is cached and re-imported under. When no rung has the file, the include or import raises the catchable Failed to open include file: helpers.jai (Failed to open import file: ... for an import, the path as written) when it is reached (a file the checker cannot find is simply not part of its gather); resolve_path instead returns "".

Program-scope redeclarations​

An engine runs many sources over its life: files, imports, strings from the host, a hook line every frame. Each is a unit: a file by its path, a string by its content (the same string executed every frame is one unit), a jaibite object by itself. A program-scope variable belongs to the unit that declared it.

• Another unit's declaration gets its own cell. When a second unit declares a name the first one declared, the new declaration shadows the old one the way a nested block shadows in C++. Code reads its own unit's cell when its unit declared the name; otherwise it reads the newest cell declared no later than the code was compiled (top-level code: the start of its current run; a function: the run that declared it). Code compiled before the redeclaration keeps its binding, and nothing is rewritten under running code. A name only a later unit declares resolves to the newest cell, so a function may read a global a later file declares.
• The same unit run again writes in place. A file reloaded at its path, a module an import re-runs, the same string executed again: the declaration writes the existing cell. One that assigns (int kills = 0;) runs its initializer again; one written without an initializer (int kills;, var loose;) keeps the value it holds, unless its declared type changed, which resets it to the new type's default.
• Functions and classes replace in place whatever the unit: a redefinition reaches every caller and every live instance, which is what hot reload is.
• { } blocks scope as in C++, at program scope too. A declaration inside a bare block, an if, a loop body or a try shadows the outer name for the block and vanishes at its end. A function called from inside the block still sees the binding of the scope it was declared in. Only a declaration at the unit's own top level binds the name for the unit.

An include is textual, so what it declares is the includer's own; an import is a unit of its own. Here rival.jai is imported, so its G is a second cell:

int G = 1;
int main_g() { return G; }
import "rival.jai";           // another unit: its G is a cell of its own
print(main_g(), " ", rival_g(), " ", G);

int hp = 10;
void show() { print("show: {}", hp); }
{
    int hp = 99;              // shadows for the block only
    print("block: {}", hp);
    show();                   // a callee keeps the outer binding
}
print("after: {}", hp);

with rival.jai:

int G = 5;
int rival_g() { return G; }

1 5 1
block: 99
show: 10
after: 10

The host reads and assigns the newest cell (add_global after a redeclaration writes it), and a reference bound before a redeclaration stays on the cell it bound. A unit that should use a global rather than own one declares it extern; one variable shared by every unit that declares it is inline int score;, the C++17 inline variable (extern and inline).

Where files are​

A program that reads or writes files beyond its own imports should not depend on the working directory: a script dropped onto the runner, launched from a shortcut or started by another program gets whatever directory the shell chose. Every engine carries these builtins. Location queries use absolute paths with forward slashes and no trailing separator; file_name returns just the last component, and the documented empty results remain empty:

All of these queries require jai::filesystem, including the lexical file_name operation. A host that calls eng->disable(jai::filesystem) makes them, include and import raise the catchable error Insufficient privilege: filesystem.. Script can inspect the flag with reflect::is_enabled(jai::filesystem) or reflect::is_enabled("filesystem"), but only the host can enable it. Hosts can additionally require custom named permissions on their own file APIs; those do not replace the built-in filesystem requirement. See Filesystem access is a host choice.

Loading is one question, so it gets one ladder. Saving is a choice, so the builtins hand you the anchors and leave the choice explicit: a save beside the program goes under root_script_dir(), a per-user save under app_data_dir(name), a regenerable cache under app_cache_dir(name), a dump for whoever is watching under working_dir(). The core engine does filesystem work for source loading, path queries and disk caching. General file operations such as read_file, write_file, list_dir and writable_dir are host functions: the standalone runner provides them and attaches the same filesystem privilege requirement; an embedding host chooses which to expose and must check or register the requirement for its own bindings. JaiDoom chooses its folder this way: the folder above scripts/ when it is writable, otherwise app_data_dir("JaiDOOM").

A missing file raises. The runner's read_file(path) returns the file's bytes exactly ("" for an existing empty file) and raises a catchable error when the file is missing or cannot be read: Cannot read file 'saves/world.gqw': not found, and not a file (a folder), permission denied, cannot be opened or read failed in the same shape. A program that expects absence asks first, if (file_exists(path)) { text = read_file(path); }, or catches: try { text = read_file(path); } catch (e) { text = ""; }. An uncaught one ends the program with the usual trace, naming the path.

What the checker sees across files​

The checker runs before anything executes, and it does not stop at the file boundary. A unit (one file, or one string handed to execute()) whose top-level import and include lines carry literal paths is the root of a program. With filesystem access enabled, before the root runs, every file those paths reach is gathered (recursively, each file once, cycles included), and the top-level names of all of them, functions, globals, classes, enums, namespaces, are pooled. Every unit is then checked against the pool. Two consequences follow.

When filesystem access is disabled, the checker skips discovery on disk. It still checks the source supplied in memory, but cannot gather declarations from those files; an attempted runtime include or import raises the privilege error. Neither a filename label for an in-memory program nor an extern declaration grants file access.

First, import order does not matter for names. A function in one file may use a global another file declares, even a file imported after it:

import "rules.jai";
import "config.jai";        // imported AFTER rules.jai, which reads its names
print("damage = {}", damage(3));

with rules.jai:

function damage(int hits) -> int { return hits * BASE_DAMAGE; }

and config.jai:

int BASE_DAMAGE = 7;

damage = 21

rules.jai reads BASE_DAMAGE, which is declared in a file imported after it. The checker knew the name from the pool; the runtime finds the value because damage() is not called until both imports have run. (Execution order still matters for values: a top-level statement in rules.jai that read BASE_DAMAGE would find it undefined at run time. The checker is a superset by design; it never fails a program that runs clean, and it cannot see the clock.)

Second, a mistake anywhere fails the root. Misspell the name in rules.jai and the root's load fails before any unit runs, with the listing named by the file's resolved path (shortened here):

jaiscript: static check failed:
scripts/rules.jai:1:50: error: use of undeclared identifier 'BASE_DAMGE'
    function damage(int hits) -> int { return hits * BASE_DAMGE; }
                                                     ^

A file that does not parse stops the program at that file: the load fails with its parse error, named by its path, line and column exactly as running the file directly reports it, and nothing is checked until it parses (the names it would declare are unknown, so checking the other files would only call them undeclared).

Class knowledge crosses files the same way: a subclass whose parent lives in another unit resolves inherited fields and methods, and a typed mismatch against a cross-file field is the same error the same-file case reports.

Three escape valves​

Sometimes a name genuinely is not in the program. It comes from the host, from a file whose path is computed at run time, or from a module only some programs import. Three spellings tell the checker so, each as narrow as its case:

• A dynamic path. import(expr) and include(expr) cannot be followed at check time, so a unit that contains one drops undeclared identifier from error to warning for that unit. It can define anything; the checker admits that and stays quiet.
• extern. Declare the name without defining it: extern function net_relay; (a callable of unknown signature), extern int tick_rate; (typed: checked as an int), extern config; (untyped, behaves like var). The program's pool carries it to every unit, and at run time it declares nothing: the name resolves when it is used, and a name nothing has defined by then raises 'net_relay' is declared extern but never defined (for a variable, with (an 'inline' declaration creates it) after it). This is also how a shared leaf calls into a module only some programs import: in a program that does import it, the extern is inert and the real definition wins.
• [[warn_mode]] at the top of a unit. The whole file is checked but reported as warnings, and a warn-mode leaf never fails its program. Coarser than extern; reach for it when a file is deliberately dynamic.

The extern shape, in a leaf that waits for a host-registered function:

import "leaf.jai";
print("loaded; on_packet waits for the host: {}", leaf_value());

with leaf.jai:

extern function net_relay;          // registered by the host before any packet arrives
function leaf_value() -> int { return 1; }
function on_packet(var p) { return net_relay(p); }

loaded; on_packet waits for the host: 1

A variable that several files declare, each wanting it to exist whichever of them a program imports, is an inline variable, as in C++17: the first inline declaration to run creates it (from its initializer, or its type's default), and every later one adopts it without running its initializer. A plain declaration and an inline one of the same name never mix: the checker reports 'frags' is already defined inline; declare it 'inline int frags;' to share it at load when it sees both.

import "hud.jai";
import "netplay.jai";
award_frag();
award_frag();
print("frags = {}", frags);

with hud.jai:

inline int frags;                   // exists whether or not netplay.jai is imported
void award_frag() { frags += 1; }

and netplay.jai:

inline int frags = 100;             // adopts hud.jai's variable: this initializer does not run

frags = 2

All three, plus inline, [[strict_mode]] and the diagnostic format, are covered in Types That Fail Early.

Which one do I want? (the real-world idiom)​

A program declares its module graph with import lines at program scope, and reloads by reaching those imports again. The dungeon crawler's entry point (examples/crawler/scripts/main.jai, abridged) is one import per module:

// GRUBWELL entry point. The host executes: import "main.jai"; crawler_main();
import "state.jai";     // the persistent globals; the dev reload leaves this file out
import "util.jai";
import "data.jai";
// ... one import per module ...

An include written inside a function body is not a reload: its text becomes part of that function (Dynamic paths), so the functions it declares are that function's nested functions, a class there is a parse error, and the program's own definitions stay as they were. Reaching an import again is what re-runs a changed module at program scope, with its checks first; the crawler's dev-mode reload is one group import inside a function, import("util.jai", "data.jai", ...), parsed and checked as a whole before any of it runs.

Reload is pull-based: it flows through normal execution. An import acts only when it is reached, so the canonical host pattern is to reach the imports every frame or tick, from the entry point or a function the loop calls: in each check pass (above) every import reached asks for its file's stamp once, a changed file re-imports and an unchanged one does nothing more. There is no watcher thread and no out-of-band reload sweep, deliberately: re-running a module has side effects, and pull-based reload means those side effects land exactly where control flow puts them, never asynchronously under your feet. engine::reset_import(s) is a narrow escape hatch for a file regenerated with an unchanged mtime (timestamp granularity), or for forcing a re-run under once mode. It is not the reload mechanism.

Below all of this sits the jaibite disk cache, and it is parse-level and side-effect-free: every file-based load keeps .cache/foo.jai.jaibite (pre-parsed AST) beside foo.jai, loading it instead of parsing only when the source it records (its size and a hash of its bytes) is exactly the text just read and the host registrations match. It never changes whether a file runs (the import gate and include semantics above decide that), only how cheaply its text turns back into an AST. Any other text reparses (and refreshes the cache), whatever its mtime: a revision copied into place with an older timestamp, or a file restored with its old one, runs its own code. A cache is written to a temporary file and renamed over the old one, so no reader sees part of one; a module re-runs only from text read whole, its stamp queried before and after the read agreeing (a save that lands mid-read waits for the next pass). Writes are silent best-effort (a read-only script dir just skips them); engine->jaibite_cache(false) disables it. Disabling jai::filesystem also skips automatic disk-cache reads, writes and metadata checks; explicitly loading or saving a file is refused instead. In-memory source and jaibite bytes remain usable.

Rules of thumb:

• Declaring your module graph: import at the top of each file: cached, cheap to repeat, mtime-aware by default; edits are picked up the next time the import executes.
• Reloading while running: reach the imports again, from wherever the reload belongs (a function the loop calls, a dev-mode branch). Only touched modules re-run.
• Forcing a re-run (mtime aside): include. It always executes, in the scope where it is written.
• Persistent state in a module you reload: declare it without an initializer, and a re-run keeps it; an initializer resets it on every re-run.