Serialization​

Two layers, one system.

Script side: the to_json / from_json builtins turn any script value into text and back: one line each way.

C++ side: an archive framework that writes JSON or a compact binary through one interface (include/jaiscript/serialization/). It is a drop-in for Cereal-shaped code: the same member/free serialize / save / load dispatch.

The part worth knowing before you write any of it: binding a class for scripting does not make it archivable by itself. The C++ archives need what the class declares for them: a member or free serialize / save / load, or JAI_PROPERTY fields on a property_owner. What the dynamic_binder chain adds is the class's name for the script-value path, and a jai::registrar around the same chain gives the identity polymorphic saves need (below).

Script side: to_json / from_json​

class Player {
    int hp = 0;
    float speed = 0.0;
    string name = "";
}
auto p = Player();
p.hp = 42; p.speed = 1.5; p.name = "hero";

var j = to_json(p);
print(j);

var q = from_json(j);          // reconstructs a Player via the $type discriminator
print(type_of(q.hp) + " " + to_string(q.hp) + ", " + q.name);

// A float payload landing in an int field: the load converts, it does not trust.
var r = from_json("{\"$type\": \"Player\", \"$val\": {\"hp\": 7.9, \"speed\": 2, \"name\": \"clone\"}}");
print(type_of(r.hp) + " " + to_string(r.hp) + ", " + type_of(r.speed));

// Plain values and containers round-trip too.
var m = from_json(to_json({"a": [1, 2.5, "three", true, null]}));
print(to_json(m));

{"$type":"Player","$val":{"hp":42,"speed":1.5,"name":"hero"}}
int 42, hero
int 7, float
{"a":[1,2.5,"three",true,null]}

The interesting line is the third one. The payload said "hp": 7.9 and the field is declared int, so the load converted it (through the very same table = uses) and you got 7. Nothing about a save file is trusted more than an assignment is.

A second argument pretty-prints: to_json(v, n) indents n spaces per level, and to_json(v) (or n = 0) writes one line.

class Player { int hp = 42; string name = "hero"; array<int> bag = [1, 2]; }
print(to_json(Player(), 2));

{
  "$type": "Player",
  "$val": {
    "hp": 42,
    "name": "hero",
    "bag": [
      1,
      2
    ]
  }
}

Key facts:

• Typed-field enforcement on deserialize: declared field types are stored on the class definition, and every write path, including from_json, converts through the same field kernel that = uses (float→int truncates, int→string via to_string; var fields stay dynamic). A hostile or sloppy save file cannot sneak a string into an int hp.
• A class instance is {"$type": "Name", "$val": {fields}}, its fields in the class's declaration order (base first). The class must be defined (or registered) in the engine before from_json sees its $type. An unknown $type is data, not an error: from_json("{\"$type\": \"NoSuchClass\", \"$val\": {\"hp\": 3}}") hands you a plain map with both keys intact.
• A known $type is constructed as Name() would construct it (the default constructor and the field initializers run, then the payload writes its fields). A constructor that throws or raises an error fails the from_json call with that error, catchable like any call's.
• Values a JSON object cannot spell travel tagged, in the same $ family as the archives' $id: a char is {"$char": 122} (its code, 0..255), and a map whose keys are not all strings (or an empty one declared with such a key type) is {"$map": [[1, "sword"], [2, "bow"]]}, each key and value written by the same rules. from_json reads both back, so a char field and a map<int, string> field round-trip. A single-key object of any other shape ({"$char": 300}, a $map whose entries are not pairs) stays a plain map.
• Handles keep their sharing. A shared_ptr<T> (what new T() yields) writes its object once, at its first appearance, as {"$id": 1, "$type": "T", "$val": {...}}, and every later handle to the same object as {"$id": 1}; a weak_ptr is {"$ref": 1} in the same numbering (with the object's "$type" and "$val" beside it when it is the first reference). An empty handle and an expired weak handle are null; a null loads into a shared_ptr<T> or weak_ptr<T> field as = null stores it. These are the C++ archives' spellings (One handle format). The load hands back handles to one object per id, in any order and through cycles, so two slots that shared an object share it again; a weak pointer whose object nothing else in the save holds loads expired. to_binary / to_base64 keep the same graph. An id whose object the save never wrote fails the load with Invalid shared_ptr ID: 4.
• Floats round-trip at full precision (std::to_chars shortest-round-trip; -0.0 keeps its sign). JSON has no spelling for an infinity or a NaN, so to_json writes null for them; to_base64 / from_base64 keep the exact bits.
• A payload nests as deep as memory allows: parsing, rebuilding and writing walk values on work lists, so a save that loaded on one machine loads on another whatever its depth (a host record type's own recursion stops at the host thread's stack or JAI_MAX_HOST_RECORD_DEPTH = 1024 records). Hostile-text hardening: trailing garbage rejected, invalid unicode escapes are serialization errors, lone surrogates become replacement chars.

Saving game state to a file​

A save is to_json written to a file, and a load is the file read back through from_json. File access belongs to the host: the standalone runner provides write_file, read_file and file_exists, and an embedding host registers its own (they run through its virtual filesystem, its save folder, its cloud sync). app_data_dir(name) answers the platform's per-user data folder.

class Hero { string name = "ada"; int gold = 0; array<string> bag = []; }
class SaveGame { int version = 1; Hero hero; map<string, bool> flags; }

SaveGame game;
game.hero.gold = 120;
game.hero.bag.push("lantern");
game.flags["met_the_raccoon"] = true;
write_file("save1.json", to_json(game, 2));

SaveGame loaded = from_json(read_file("save1.json"));
print(loaded.hero.name, " ", loaded.hero.gold, " ", loaded.hero.bag[0], " ", loaded.flags["met_the_raccoon"]);

ada 120 lantern true

A field added to the class after a save loads with its initializer, because from_json builds the object first and then writes the fields the file has. A version field like the one above lets a loader convert older files explicitly.

C++ side: the convenience API​

#include <jaiscript/serialization/convenience.hpp>

struct SaveGame {
    int level = 0; double time_played = 0; std::string hero; std::vector<int> inventory;
    template<typename Archive>
    void serialize(Archive& ar) {          // one method, both directions, both formats
        ar("level", level);
        ar("time_played", time_played);
        ar("hero", hero);
        ar("inventory", inventory);
    }
};

SaveGame save{7, 123.5, "Alice", {1, 2, 3}};
std::string json = jai::to_json(*eng, save);                  // human-readable, indented 2 spaces
SaveGame loaded  = jai::from_json<SaveGame>(*eng, json);
std::cout << json << "\n";
std::cout << jai::to_json(*eng, save, 0) << "\n";              // indent 0: one line
std::cout << "loaded: level=" << loaded.level << " hero=" << loaded.hero << " items=" << loaded.inventory.size() << "\n";

std::vector<uint8_t> blob = jai::to_binary(*eng, save);       // compact little-endian
SaveGame fromBin = jai::from_binary<SaveGame>(*eng, blob);
std::cout << "binary: " << blob.size() << " bytes, hero=" << fromBin.hero << "\n";

{
  "level": 7,
  "time_played": 123.5,
  "hero": "Alice",
  "inventory": [
    1,
    2,
    3
  ]
}
{"level": 7,"time_played": 123.5,"hero": "Alice","inventory": [1,2,3]}
loaded: level=7 hero=Alice items=3
binary: 114 bytes, hero=Alice

The third argument of to_json is the indent, 2 by default; 0 writes one line. A plain struct's fields sit flat in the root record, which to_json opens with no type name, and a C++ record nested inside it carries no $type. (An owning pointer at the root sits under a value0 slot in its pointer shape instead: $id/$type/$val for a shared_ptr, the shape the binder chapter shows, and $val alone for a unique_ptr.)

Also available: to_binary_string/from_binary_string, to_base64/from_base64, and Context& overloads of every one (jai::from_json<T>(*eng, json, ctx), jai::to_binary(*eng, value, ctx)) that hand a user context pointer to nested save and load calls (ar.get_user_context<Context>()). The binary format is little-endian on every platform, so a blob written on one machine loads on another.

The loads that return T by value (from_json<T>, from_binary<T>, from_binary_string<T>, from_base64<T>) build a local T and return it, so T must be movable. A property_owner deletes its move constructor (its property manager points at its members), so it loads as jai::from_json<std::shared_ptr<T>>, which also keeps a derived object's $type.

Dispatch priority (Cereal-style: provide either serialize OR save/load)​

For a type T the archive takes the first of these that applies:

1. a primitive: an arithmetic type, an enum or std::string
2. a member save (writing) or load (reading)
3. a member serialize
4. a property_owner<T>: every level's JAI_PROPERTY fields, automatically
5. a built-in standard type: std::vector, map, unordered_map, array, pair, optional, variant, tuple, the sets, shared_ptr / weak_ptr / unique_ptr
6. a free serialize found by ADL
7. a free save (writing) or load (reading) found by ADL

Members come first. A property_owner type that also declares save / load uses them, and can call save_owned_properties(ar) / load_owned_properties(ar) inside them to write its JAI_PROPERTY fields beside the hand-written ones. A type with a member function never reaches a free one. A type with save but no load loads through its serialize, and the reverse.

Free functions are found only in their two-parameter form: serialize(Archive&, T&), save(Archive&, const T&), load(Archive&, T&). One with a version parameter is not found. A member save, load or serialize with a std::uint32_t version parameter is found, and preferred over the one-parameter form (Versions says what it receives and why a format property serves better).

No match is not a compile error: the archive throws jai::serialization_error when it reaches the value (Type '...' is not serializable. Add save/load, serialize, property_owner, or ADL functions — or mark the property transient., and is not deserializable on a load). A test that saves and loads each type catches it before a shipped save does. JAI_STATIC_BINDER (a compile-time property table some hosts declare instead of property_owner) is not part of this dispatch: its jai_static_type<T>::save / load runs for a statically bound property inside another statically bound type, and a statically bound type handed to the archive directly needs one of the functions above.

A shared_ptr or weak_ptr load builds its object through load_and_construct when the type declares one: a static member static void load_and_construct(Archive&, jai::serialization::construct<T>&) with exactly those two parameters (<jaiscript/serialization/construct.hpp>); a free function or a form with a version parameter is not found. Without one, the archive default-constructs the object and loads it normally: a public default constructor, or a private one when the type declares friend jai::access;. A type with neither raises serialization_error. A unique_ptr always default-constructs, the same two ways.

Private members: friend jai::access.

The archive inside save, load and serialize​

Named properties are the normal form; the unnamed calls exist for hand-written layouts. A writer:

A reader mirrors it:

Every failure in either direction throws jai::serialization_error, a jai::runtime_error.

Versions and missing fields​

Ignore the version parameter; write a format property. A member save, load or serialize may take a std::uint32_t version, the slot Cereal fills with a class version. These archives keep no class version, so the number carries nothing you wrote: a save always receives 0, and a load receives whatever the record happens to hold, usually 1 in JSON and 0 in binary. A format property is an ordinary field instead. You choose its value, it reads back the same from JSON and binary, and a record written before it existed simply lacks it, so the load defaults it:

template<class Archive>
void save(Archive& ar) const {
    ar("format", std::uint32_t(2));
    ar(JAI_NVP(health));
    ar(JAI_NVP(experience));
}

template<class Archive>
void load(Archive& ar) {
    std::uint32_t format = 1;
    if (ar.has_property("format")) { ar("format", format); }
    ar(JAI_NVP(health));
    if (format >= 2) {
        ar(JAI_NVP(experience));
    } else {
        int level = 0;
        ar("level", level);
        experience = level * 100;
    }
}

Most changes need no format number at all. Both formats are self-describing: every record names its properties, so a reader finds a property by name, never by position. The automatic property_owner load reads the properties the record and the class share. A property the record lacks keeps its initializer, and a property the class no longer declares is skipped. Adding or removing a JAI_PROPERTY therefore needs no migration code.

A hand-written load asks for each property it reads, and ar("name", v) throws when the record lacks it. A field newer than some saves is read under ar.has_property("name"), so a record written before the field existed leaves it at its initializer.

A format property earns its place when a field keeps its name but changes its meaning: a unit, an encoding, one field split into two, as in the example above. From Cereal converts a Cereal-versioned type this way.

For code that already takes the version parameter, this is what it receives:

• A save always receives 0.
• A load reached through a named read (ar("name", obj), a JAI_PROPERTY, a non-polymorphic pointer's $val) receives the version stored in that record.
• Every other load (ar(obj), a polymorphic pointer's $val) receives 0.
• A record the archive opens itself stores no version. JSON writes no "$type" or "$version" key for it, which reads back as version 1. Binary stores an empty type name and version 0.
• The root record that to_json / to_binary open has an empty type name and version 1, which JSON leaves unwritten.
• A hand-written save can open a record with begin_object(type_name, version). Binary keeps any number; JSON writes "$version" only above 1, so 0 reads back as 1.
• ar.get_version() answers the version of the record opened last.

A binder chain's .version(n) never reaches the C++ archives. It sets a number (1 by default) that the engine's serialization registry keeps for the class, and only the binary writer of script values reads it: to_binary / to_base64 write it after the class name of each bound object a script value holds. On load, from_binary / from_base64 hand that number to the class's .post_load_hook only when the class also has a .deserialization_factory. Without a factory the object is rebuilt by its class name, and the hook receives 1. Script-side from_json runs the hook too, with the object's "$version" key (beside its "$type") when it has one and 1 otherwise, and to_json writes no "$version".

Wire formats​

Both formats run the same calls; the writer decides the bytes. JSON writes a record as an object of its properties; a record opened with a type name (a hand-written begin_object(type_name, version)) leads with "$type", and with "$version" when the version is above 1. The binary writer is little-endian: a record is the byte 0x01, its type name (a uint32 length, then the bytes; empty for a record the archive opens itself) and a uint32 version (0 for such a record), then, when the record closes, its property count, every property name, every property's byte size, the values, and 0x02. An array is 0x03, a count, the elements and 0x04; a map is 0x05 … 0x06; null is 0x00. The names and sizes up front let a reader jump to a property by name and skip the ones it does not know without parsing them.

Pointers have the same record shapes in both formats, keyed by the archive's per-save object ids:

One handle format​

The script archives write a script value: script to_json / from_json, to_binary / from_binary and to_base64 / from_base64. The C++ archives write a host type through its serialize, save / load or JAI_PROPERTY fields: jai::to_json, jai::from_json<T> and the other C++ entry points. Both spell classes and handles with the same keys, and the binary forms follow the same identity rules: an object travels with the save's first reference to it, and every later reference is its id.

Where the two differ, the difference is the static type:

The same object graph in each. A script class holding two handles to one Node and a weak handle to it:

class Node { int v = 1; }
class Link { shared_ptr<Node> a; shared_ptr<Node> b; weak_ptr<Node> w; }
Link link;
link.a = new Node();
link.b = link.a;
link.w = link.a;
print(to_json(link));

{"$type":"Link","$val":{"a":{"$id":1,"$type":"Node","$val":{"v":1}},"b":{"$id":1},"w":{"$ref":1}}}

The C++ version of the same types:

struct Node {
    int64_t v = 1;
    template<class Archive> void serialize(Archive& ar) { ar("v", v); }
};
struct Link {
    std::shared_ptr<Node> a, b;
    std::weak_ptr<Node> w;
    template<class Archive> void serialize(Archive& ar) { ar("a", a); ar("b", b); ar("w", w); }
};

Link link;
link.a = std::make_shared<Node>();
link.b = link.a;
link.w = link.a;
std::cout << jai::to_json(*eng, link, 0) << "\n";

{"a": {"$id": 1,"$val": {"v": 1}},"b": {"$id": 1},"w": {"$ref": 1}}

A script load has no static type to go by, so every object names its class, and a name the engine does not know leaves the data as a map. A C++ load knows the type it is reading, so a record names its class only when the dynamic type differs from the static one. A script class layered over a C++ object meets both: the C++ archives write it as "$type", "$script" and "$val", and "$script" holds the script fields as a string in the script archives' JSON (Script-Extensible Objects).

Migrating from Cereal​

A Cereal-serialized type moves over with spelling changes: CEREAL_NVP becomes JAI_NVP, friend class cereal::access becomes friend jai::access, member and free serialize / save / load keep their shape, and CEREAL_REGISTER_TYPE becomes a jai::registrar or a property_owner base. The files do not move: these archives do not read Cereal's format, so existing data is converted once, loaded with Cereal and saved with JaiScript. From Cereal, in the Code Migrations, maps every Cereal construct (and the Boost.Serialization spellings), converts a small save game, keeps both libraries in one function during the transition, and covers converting existing files.

Smart pointers, polymorphism, graphs​

• shared_ptr dedupe: aliased pointers serialize once ($id) and re-alias on load. Loaded copies share one object, exactly like the graph you saved.
• weak_ptr reconstruction works even for forward references (the weak_ptr appears in the stream before its owner).
• Polymorphism needs a serialization identity. Saving a shared_ptr<Base> that holds a Derived writes a $type discriminator and loads the Derived back, provided Derived has a name the archives know. A property_owner type has one from its declaration (dynamic_binder). A plain class bound with dynamic_binder gets one from a jai::registrar, whose jai::bases<...> tag records the edge the load checks; .base_class<B>() alone is the script-side edge, and a save through the base pointer without a registrar raises Cannot serialize unregistered polymorphic type 'class Monster' through pointer-to-base 'class Actor'. Register it with jai::registrar<T, Context, jai::bases<Base>>("Name") or derive from jai::property_owner<T, Base>. (the type names are the compiler's typeid spellings, MSVC's here):
  C++

  // examples/embed_quickstart: Monster derives Actor, both plain classes with serialize()
  static jai::registrar<Monster, void, jai::bases<Actor>> monsterRegistrar("Monster", [](jai::dynamic_binder<Monster>& b) {
      b.base_class<Actor>()
       .constructor<std::string, int64_t>()
       .property("hp", &Monster::hp)
       .method("hurt", &Monster::hurt);
  });
  
  jai::bind_registrar(*eng);                            // binds every registrar into this engine
  std::shared_ptr<Actor> boss = std::make_shared<Monster>("Troll", 40);
  auto json = jai::to_json(*eng, boss);                 // "$type": "Monster"
  auto back = jai::from_json<std::shared_ptr<Actor>>(*eng, json);   // a Monster again

  The chain's .version(n), .post_load_hook(...) and .deserialization_factory(...) serve bound objects inside script values: .version(n) and the factory serve from_binary / from_base64, and the hook runs after those and after from_json (Versions). A C++ pointer load constructs through load_and_construct or the default constructor. The chain itself is documented in dynamic_binder.
• A script class layered over a C++ object serializes as itself. An object a script has extended writes $type (the script class), $script (its script fields) and $val (the C++ payload) through every archive path, and a load rebuilds the hybrid. See Script-Extensible Objects.
• Signal connections ride save files. Named script-text receivers on signal properties serialize with their object; pass the engine as the load context (jai::from_json<std::shared_ptr<T>>(*eng, json, *eng)) and they re-arm on load, firing on the next emit. Function-object receivers (script lambdas, C++ callbacks) are runtime-only and skipped, never errored on. The worked round-trip is in the binder chapter.

Security posture (untrusted payloads)​

A dedicated security suite pins that deserialization never executes payload-controlled names: hostile $type/property names in JSON, binary, and base64 are data, not code, and oversized array/map length prefixes in binary data are rejected instead of pre-allocated, so a 12-byte payload can't request a 2 GB allocation, and a count reserves at most 1 MiB ahead of the elements that actually load. Data of another kind where a record, an array, a map or an integer belongs fails the load with a jai::serialization_error (Expected an object in the archive, found an integer), and so does an integer its field cannot hold (Archive value 300 does not fit int8, and 1.5 into an integer field alike): nothing wraps or truncates (the texts). A payload does choose which of the engine's classes a load builds: a type name it carries runs that class's zero-argument constructor and its post_load, as any load of that class would.

The load itself runs no payload text, but two things in a payload reach code. A signal property (JAI_SIGNAL_PROPERTY) saves its script-text receivers as their source, and a load re-arms them, so each one runs as an execute of that text, in the engine's global scope with every API the host registered, the next time its signal fires. A payload carrying signal properties therefore carries script: load such objects only from a trusted source, or into an engine that runs untrusted script under the safety rails. And a payload chooses which classes the load constructs: a $type naming a class the engine defines runs that class's own code (on from_json, a script class's constructor and field initializers, the setters of the properties the payload writes, and its post_load; a hybrid's field initializers and post_load; a C++ type's load_and_construct or default constructor and its load), never code the payload supplies.

Which layer to use​

• Script gameplay state (the crawler's save file is exactly this): script-side to_json(G) / from_json(text). One line each way, typed fields re-enforced on load.
• Host/engine objects, editor documents, network payloads: the C++ archives. JSON while debugging, to_binary when size/speed matter, same serialize method either way. A network payload that holds signal properties carries script (Security posture).
• Pre-parsed script code is its own format: .jaibite via jaibite::save (see jaibite). ASTs, not values.