From Cereal​

Cereal and the JaiScript archives share a shape: a type describes itself once, in a serialize function or a save/load pair, and an archive drives that description in both directions. Most of a migration is spelling. Three things are more than spelling:

• The files. The JaiScript archives write their own format, in JSON and in binary. A file Cereal wrote does not load through them, so existing data is converted once: loaded with Cereal, saved with JaiScript.
• Registration. A polymorphic type gets its identity from a jai::property_owner base or a jai::registrar, in place of CEREAL_REGISTER_TYPE.
• Versions. Both JaiScript formats name every property in every record, so most of what CEREAL_CLASS_VERSION did is unnecessary.

This page maps every Cereal construct in common use, converts a small save game, shows the property route that removes the serialize functions entirely, and lays out a path that keeps both libraries in the build until the last old file is gone. Cereal took its design from Boost.Serialization, so a Boost codebase takes the same path; From Boost.Serialization lists what is spelled differently there. The archive framework itself is documented in Serialization.

The concept map​

Headers: <jaiscript/serialization/convenience.hpp> for the archives and the one-call entry points, <jaiscript/serialization/construct.hpp> for load_and_construct, <jaiscript/core/registrar.hpp> for registrars, and <jaiscript/properties/macros.hpp> with <jaiscript/properties/property_serialization.hpp> for properties.

Archives and entry points​

Names​

Serialization functions​

Versions and polymorphism​

Pointers and identity​

Cereal tracks each shared_ptr by an id so a shared object is written once. The JaiScript archives do the same, in records you can read:

Containers and standard types​

Construction, access, context and errors​

A worked conversion​

A small save game: an Item, an Inventory that owns items, a Player that holds its inventory through a shared_ptr, and status effects behind base pointers.

Leaf types​

#include <cereal/types/string.hpp>
#include <cereal/types/vector.hpp>

struct Item {
    std::string name;
    int count = 1;

    template<class Archive>
    void serialize(Archive& ar) { ar(CEREAL_NVP(name), CEREAL_NVP(count)); }
};

class Inventory {
    friend class cereal::access;
    std::vector<Item> items;
    int capacity = 20;

    template<class Archive>
    void serialize(Archive& ar, std::uint32_t const version) {
        ar(CEREAL_NVP(items));
        if (version >= 2) { ar(CEREAL_NVP(capacity)); }
    }
};
CEREAL_CLASS_VERSION(Inventory, 2);

#include <jaiscript/serialization/convenience.hpp>

struct Item {
    std::string name;
    int count = 1;

    template<class Archive>
    void serialize(Archive& ar) { ar(JAI_NVP(name), JAI_NVP(count)); }
};

class Inventory {
    friend jai::access;
    std::vector<Item> items;
    int capacity = 20;

    template<class Archive>
    void serialize(Archive& ar) { ar(JAI_NVP(items), JAI_NVP(capacity)); }
};

The type headers go, because the standard types are built in. The names change spelling. The version gate goes too: it existed for Cereal files written before capacity did. Every JaiScript file is written by the converted code, so the first JaiScript layout already holds both fields, and the old files reach JaiScript through the converter, which reads them with Cereal and its version branch (below). A field added from here on is read under ar.has_property (Versions).

Owners and shared pointers​

#include <cereal/types/memory.hpp>
#include <cereal/archives/json.hpp>

class Player {
public:
    std::string name;
    int hp = 100;
    std::shared_ptr<Inventory> inventory = std::make_shared<Inventory>();
    float hpFraction = 1.0f;                     // derived on load, never saved

    template<class Archive>
    void save(Archive& ar) const { ar(CEREAL_NVP(name), CEREAL_NVP(hp), CEREAL_NVP(inventory)); }

    template<class Archive>
    void load(Archive& ar) {
        ar(CEREAL_NVP(name), CEREAL_NVP(hp), CEREAL_NVP(inventory));
        hpFraction = hp / 100.0f;
    }
};

std::ofstream os("player.json");
{
    cereal::JSONOutputArchive ar(os);
    ar(player);
}

class Player {
public:
    std::string name;
    int hp = 100;
    std::shared_ptr<Inventory> inventory = std::make_shared<Inventory>();
    float hpFraction = 1.0f;

    template<class Archive>
    void save(Archive& ar) const { ar(JAI_NVP(name), JAI_NVP(hp), JAI_NVP(inventory)); }

    template<class Archive>
    void load(Archive& ar) {
        ar(JAI_NVP(name), JAI_NVP(hp), JAI_NVP(inventory));
        hpFraction = hp / 100.0f;
    }
};

auto eng = jai::engine::make();
std::ofstream("player.json") << jai::to_json(*eng, player);

std::stringstream text;
text << std::ifstream("player.json").rdbuf();
Player loaded = jai::from_json<Player>(*eng, text.str());

The class body changes only in its names. The inventory is written as {"$id": 1, "$val": {"items": [...], "capacity": 20}}; a second shared_ptr to the same inventory in the same save would write {"$id": 1} and load as the same object. A std::vector<Item> needs no help: the archives walk the container and call Item::serialize per element.

A polymorphic hierarchy​

#include <cereal/types/polymorphic.hpp>
#include <cereal/types/base_class.hpp>

struct Effect {
    virtual ~Effect() = default;
    double remaining = 0;

    template<class Archive>
    void serialize(Archive& ar) { ar(CEREAL_NVP(remaining)); }
};

struct Burn : Effect {
    int damagePerTick = 1;

    template<class Archive>
    void serialize(Archive& ar) { ar(cereal::base_class<Effect>(this), CEREAL_NVP(damagePerTick)); }
};

struct Shield : Effect {
    int absorb = 0;

    template<class Archive>
    void serialize(Archive& ar) { ar(cereal::base_class<Effect>(this), CEREAL_NVP(absorb)); }
};

CEREAL_REGISTER_TYPE(Burn)
CEREAL_REGISTER_TYPE(Shield)
CEREAL_REGISTER_POLYMORPHIC_RELATION(Effect, Burn)
CEREAL_REGISTER_POLYMORPHIC_RELATION(Effect, Shield)

#include <jaiscript/core/registrar.hpp>

struct Effect {
    virtual ~Effect() = default;
    double remaining = 0;

    template<class Archive>
    void serialize(Archive& ar) { ar(JAI_NVP(remaining)); }
};

struct Burn : Effect {
    int damagePerTick = 1;

    template<class Archive>
    void serialize(Archive& ar) { Effect::serialize(ar); ar(JAI_NVP(damagePerTick)); }
};

struct Shield : Effect {
    int absorb = 0;

    template<class Archive>
    void serialize(Archive& ar) { Effect::serialize(ar); ar(JAI_NVP(absorb)); }
};

static jai::registrar<Burn, void, jai::bases<Effect>> burnRegistrar("Burn");
static jai::registrar<Shield, void, jai::bases<Effect>> shieldRegistrar("Shield");

Each registrar gives its class the name the archives write as $type and records the edge to Effect, which a load checks before it hands a Burn back through an Effect pointer. It registers at program start; the same object also makes the class bindable into an engine through jai::bind_registrar, with an optional configure lambda that adds methods and properties (Serialization shows one). Give Player a std::vector<std::shared_ptr<Effect>> effects field, archive it like the others, and each element is written as {"$id": N, "$type": "Burn", "$val": {"remaining": 3.5, "damagePerTick": 2}} and loads as a Burn.

The base's fields land in the derived class's record, beside its own, where Cereal nests them in a record of their own. Keep field names distinct across a hierarchy.

Root objects and files​

jai::to_json(*eng, value) writes one value as the root. Cereal code that writes several root values (ar(settings, world)) either gathers them into one struct or opens the root record by hand, the way the one-call functions do:

jai::serialization::json_archive_writer out(2, eng.get());     // indent, engine
out.begin_object("", 1);
out("settings", settings);
out("world", world);
out.end_object();
std::string text = out.str();

jai::serialization::json_archive_reader in(text, eng.get());
std::string typeName;
std::uint32_t version = 0;
in.begin_object(typeName, version);
in("settings", settings);
in("world", world);
in.end_object();

The binary pair is the same calls: binary_archive_writer out(eng.get()) with out.data() for the bytes, and binary_archive_reader in(bytes, eng.get()).

The property route: no serialize functions​

Cereal needs a function per type. A class that derives jai::property_owner and declares its fields with JAI_PROPERTY needs none: the archives save and load every property by name, the same declaration makes the field reflected and script-visible (dynamic_binder), and the class carries its own polymorphic identity. A migration is a good moment to make that move for types whose functions only list their fields.

#include <jaiscript/properties/macros.hpp>
#include <jaiscript/properties/property_serialization.hpp>

class Effect : public jai::property_owner<Effect> {
public:
    JAI_PROPERTY((double), remaining, 0.0);
};

class Burn : public jai::property_owner<Burn, Effect> {
public:
    JAI_PROPERTY((int), damagePerTick, 1);
};

class Player : public jai::property_owner<Player> {
public:
    JAI_PROPERTY((std::string), name, "hero");
    JAI_PROPERTY((int), hp, 100);
    JAI_PROPERTY((std::shared_ptr<Inventory>), inventory, std::make_shared<Inventory>());
    JAI_PROPERTY((std::vector<std::shared_ptr<Effect>>), effects);
    JAI_TRANSIENT_PROPERTY((float), hpFraction, 1.0f);
};

auto player = std::make_shared<Player>();
std::string json = jai::to_json(*eng, player);
auto loaded = jai::from_json<std::shared_ptr<Player>>(*eng, json);

• Nothing to keep in step. A load reads the properties the record and the class share. A property the record lacks keeps its initializer, and one the class no longer declares is skipped, so adding or removing a JAI_PROPERTY needs no migration code.
• JAI_TRANSIENT_PROPERTY is reflected but never saved: the counterpart of a field a Cereal function left out. JAI_NAMED_PROPERTY((type), "saved_name", member) saves under a key other than the member's name, which keeps a key stable through a rename.
• property_owner<Burn, Effect> records Burn's name and its edge to Effect, so polymorphic saves need no registrar. The name is the bare class name; JAI_PROPERTY_OWNER_AS(Burn, "Burn") pins it.
• A property_owner cannot be moved (its property manager points at its members), so a root loads through a std::shared_ptr, as above.
• Members that are not properties are not saved. A class that needs more keeps a save / load pair, which takes precedence over the automatic one; save_owned_properties(ar) and load_owned_properties(ar) write and read every level's properties beside the hand-written part (property_mgr.save(ar) covers only the declaring class's level):

class Player : public jai::property_owner<Player> {
public:
    JAI_PROPERTY((int), hp, 100);
    float hpFraction = 1.0f;

    template<class Archive>
    void save(Archive& ar) const { save_owned_properties(ar); }

    template<class Archive>
    void load(Archive& ar) {
        load_owned_properties(ar);
        hpFraction = hp.get() / 100.0f;
    }
};

Versions in a self-describing format​

Cereal's binary archives are positional: a reader asks for exactly what the writer wrote, in the same order, and CEREAL_CLASS_VERSION tells it which layout it holds. Both JaiScript formats name every property in every record (the binary one through a table of names and sizes ahead of the values), so a reader finds a property by name and skips the ones it does not ask for. Most of what a version number did goes away:

• Reordering fields changes nothing, except among fields that hold the same object: the one the file writes the object in is read first (Pitfalls).
• A removed field is skipped, because nothing asks for it.
• An added field is read under ar.has_property, and keeps its initializer for a record written before it existed (a property_owner does this for every property):

template<class Archive>
void load(Archive& ar) {
    ar(JAI_NVP(items));
    if (ar.has_property("capacity")) { ar(JAI_NVP(capacity)); }
}

What is left for a version is a field whose meaning changes: a unit, an encoding, one field split into two. The archives record no class version for a C++ type, so such a type writes one as a property and defaults it for records that predate it:

template<class Archive>
void save(Archive& ar) const {
    ar("format", std::uint32_t(2));
    ar(JAI_NVP(metersPerSecond));
}

template<class Archive>
void load(Archive& ar) {
    std::uint32_t format = 1;
    if (ar.has_property("format")) { ar("format", format); }
    if (format >= 2) {
        ar(JAI_NVP(metersPerSecond));
    } else {
        double kilometersPerHour = 0;
        ar("speed", kilometersPerHour);
        metersPerSecond = kilometersPerHour / 3.6;
    }
}

A Cereal version history does not carry over. The first JaiScript layout of a type holds every current field, and the version branches stay on the Cereal side for as long as the converter needs them. A versioned member function still compiles and is still called, but the number it receives is not a record version: leave it unused on the JaiScript side.

A migration path​

1. Add JaiScript beside Cereal. Link the library, include <jaiscript/serialization/convenience.hpp> where types are serialized, and make one engine the save code can reach. Nothing changes on disk yet.
2. Make the leaf types dual. Starting with types that hold only primitives, strings and containers of those, give each function a JaiScript branch beside its Cereal one (below). Both libraries drive the same function, and Cereal's output does not change.
3. Move up through the owners. A type is ready when every type it holds is. Containers and smart pointers need nothing beyond their element types.
4. Register the polymorphic hierarchies. Add a registrar, or a property_owner base, beside each CEREAL_REGISTER_TYPE. Keep Cereal's registrations: the converter still needs them.
5. Prove each type round-trips through both JaiScript formats before anything is saved for real: jai::from_json<T>(*eng, jai::to_json(*eng, value)) and jai::from_binary<T>(*eng, jai::to_binary(*eng, value)), compared with the original. The two formats run different code, and a field one of them drops can hide behind the other.
6. Switch the writers. New files go out through JaiScript under a new name (an extension, a folder). The loader reads files with the new name through JaiScript and falls back to Cereal for the old name.
7. Convert the existing data once (below).
8. Remove Cereal. When no file in the old format remains, delete the Cereal branches, registrations, includes and friend class cereal::access lines, and drop the dependency. Types whose functions only list fields can then take the property route.

Both libraries in one function​

jai::serialization::jai_archive<Archive> is true for the JaiScript archives and false for every Cereal archive, so an if constexpr on it keeps each library's spelling on its own side and the branch not taken is never instantiated:

struct Effect {
    virtual ~Effect() = default;
    double remaining = 0;

    template<class Archive>
    void serialize(Archive& ar) {
        if constexpr (jai::serialization::jai_archive<Archive>) { ar(JAI_NVP(remaining)); }
        else { ar(CEREAL_NVP(remaining)); }
    }
};

struct Burn : Effect {
    int damagePerTick = 1;

    template<class Archive>
    void serialize(Archive& ar) {
        if constexpr (jai::serialization::jai_archive<Archive>) {
            Effect::serialize(ar);
            ar(JAI_NVP(damagePerTick));
        } else {
            ar(cereal::base_class<Effect>(this), CEREAL_NVP(damagePerTick));
        }
    }
};

CEREAL_REGISTER_TYPE(Burn)
CEREAL_REGISTER_POLYMORPHIC_RELATION(Effect, Burn)
static jai::registrar<Burn, void, jai::bases<Effect>> burnRegistrar("Burn");

• Where the two bodies share nothing, two overloads say the same thing: one constrained with requires jai::serialization::jai_archive<Archive>, the other with requires jai::serialization::not_jai_archive<Archive>.
• A Cereal function with a version parameter keeps it. The JaiScript archives call the versioned member form too, so the version branches stay inside the Cereal side.
• A type with load_and_construct keeps one per library. They take different construct types, so they overload without constraints.
• Code that already holds a Cereal name-value pair hands it to a JaiScript archive as its two halves:

template<class Archive, class T>
void archive_pair(Archive& ar, cereal::NameValuePair<T>& pair) {
    if constexpr (jai::serialization::jai_archive<Archive>) { ar(pair.name, pair.value); }
    else { ar(pair); }
}

Converting existing files​

A JaiScript archive cannot read a file Cereal wrote: the two formats share no layout. Existing data crosses once, through a program that links both libraries and the dual types. Cereal loads each file, exactly as the old code wrote it (the same archive type, the same root shape, Cereal's registrations compiled in), and JaiScript saves it:

#include <cereal/archives/binary.hpp>
#include <jaiscript/serialization/convenience.hpp>

void convertSave(jai::engine& eng, const std::filesystem::path& from, const std::filesystem::path& to) {
    SaveGame save;
    {
        std::ifstream is(from, std::ios::binary);
        cereal::BinaryInputArchive in(is);
        in(save);
    }
    std::string text = jai::to_json(eng, save);

    SaveGame check = jai::from_json<SaveGame>(eng, text);
    if (jai::to_json(eng, check) != text) {
        throw std::runtime_error("lossy conversion: " + from.string());
    }
    std::ofstream(to) << text;
}

The check loads the new text back and saves it again: a field the JaiScript load misses shows up as a difference. (A type holding an unordered_map or unordered_set can list the same entries in another order after a reload; compare those by content.) Content that ships with the program (levels, prefabs, tuning tables) converts in one batch: run the converter over it, keep the converted files and delete the old ones.

Files you cannot convert in advance​

Saves on players' machines convert on first load. Ship releases that read a file with the old name through Cereal, write it under the new name through JaiScript, and read the new name from then on; remove the Cereal reader once the old saves no longer matter. Tell the formats apart by name, never by trying one reader and catching its error (first pitfall).

The JaiScript formats belong to the library's 0.x surface: data saved by one 0.x release is not promised to load in the next (Stability). Keep the converter and the original files until the format your data depends on has settled.

From Boost.Serialization​

Cereal is modeled on Boost.Serialization, and everything above applies to a Boost codebase: the files convert once through a program that links both libraries, polymorphic types register, and class versions mostly go away. Boost's member functions keep their signatures, because serialize(Archive&, const unsigned int) is the versioned member form the JaiScript archives call. What changes is inside them: Boost streams fields with &, << and >>, and the JaiScript archives take named fields through their call operator, with no stream operators.

The effects from the worked conversion, written for Boost and then for JaiScript:

#include <boost/serialization/base_object.hpp>
#include <boost/serialization/export.hpp>
#include <boost/serialization/nvp.hpp>

struct Effect {
    virtual ~Effect() = default;
    double remaining = 0;

    template<class Archive>
    void serialize(Archive& ar, const unsigned int) { ar & BOOST_SERIALIZATION_NVP(remaining); }
};

struct Burn : Effect {
    int damagePerTick = 1;

    template<class Archive>
    void serialize(Archive& ar, const unsigned int) {
        ar & boost::serialization::make_nvp("Effect", boost::serialization::base_object<Effect>(*this));
        ar & BOOST_SERIALIZATION_NVP(damagePerTick);
    }
};

BOOST_CLASS_EXPORT(Burn)

#include <jaiscript/serialization/convenience.hpp>
#include <jaiscript/core/registrar.hpp>

struct Effect {
    virtual ~Effect() = default;
    double remaining = 0;

    template<class Archive>
    void serialize(Archive& ar, const unsigned int) { ar(JAI_NVP(remaining)); }
};

struct Burn : Effect {
    int damagePerTick = 1;

    template<class Archive>
    void serialize(Archive& ar, const unsigned int version) {
        Effect::serialize(ar, version);
        ar(JAI_NVP(damagePerTick));
    }
};

static jai::registrar<Burn, void, jai::bases<Effect>> burnRegistrar("Burn");

A std::vector<std::shared_ptr<Effect>> holding a Burn saves as {"$id": 1, "$type": "Burn", "$val": {"remaining": 3.5, "damagePerTick": 2}} and loads as a Burn. During the transition one function serves both libraries the way it does for Cereal (above): jai::serialization::jai_archive<Archive> is false for every Boost archive, so the Boost body stays in the other branch, and a save/load pair keeps its BOOST_SERIALIZATION_SPLIT_MEMBER() until Boost leaves the build:

class Player {
    friend class boost::serialization::access;
    friend jai::access;
public:
    std::string name;
    int hp = 100;
    float hpFraction = 1.0f;

private:
    template<class Archive>
    void save(Archive& ar, const unsigned int) const {
        if constexpr (jai::serialization::jai_archive<Archive>) { ar(JAI_NVP(name), JAI_NVP(hp)); }
        else { ar << name << hp; }
    }

    template<class Archive>
    void load(Archive& ar, const unsigned int) {
        if constexpr (jai::serialization::jai_archive<Archive>) { ar(JAI_NVP(name), JAI_NVP(hp)); }
        else { ar >> name >> hp; }
        hpFraction = hp / 100.0f;
    }
    BOOST_SERIALIZATION_SPLIT_MEMBER()
};

A Boost text or binary archive is positional, like Cereal's binary archives, and reads only through the code and the class versions that wrote it. The converter loads each old file with Boost, exactly as the old code wrote it, and saves it with JaiScript (Converting existing files).

Pitfalls​

• Reading an old file with the new loader. A property_owner load reads the properties it recognizes and keeps the initializers of the rest, so a Cereal file handed to it can load as an object of defaults without an error. Keep the formats under different names.
• Positional fields. Cereal names unnamed values value0, value1; JaiScript has no positional form. JAI_NVP takes its argument's text as the key, so JAI_NVP(m_hp) saves "m_hp" and JAI_NVP(this->hp) saves "this->hp"; ar("hp", m_hp) spells the key you want in the file.
• A field the record lacks. ar("name", v) on a missing property throws jai::serialization_error (Property 'name' not found during deserialization). A field a Cereal function wrote conditionally is read under ar.has_property("name").
• A polymorphic unique_ptr. It is written by its static type, so a std::unique_ptr<Effect> holding a Burn saves the Effect fields and loads an Effect, with no error. A polymorphic owner is a std::shared_ptr.
• An unregistered derived class. Saving one through a base pointer throws Cannot serialize unregistered polymorphic type '...' through pointer-to-base '...'; loading a $type the program does not know throws Unknown polymorphic type: Name, and a $type naming a registered class that does not derive from the requested base is rejected.
• The name is in the file. $type stores the registered name, and an automatic name follows the C++ class name, so renaming the class breaks old saves. Pin a name that must outlive a rename (the registrar's argument, JAI_PROPERTY_OWNER_AS) before the first save.
• A registration a binary never sees. A property_owner registers its name when the program constructs or binds the type somewhere; a tool that only loads a type registers it with a jai::registrar. A registrar in a static library's object file that nothing references is dropped by the linker, as Cereal's registrations are.
• A versioned free function. The archives find free functions with two parameters only, so a free serialize(Archive&, T&, std::uint32_t) leaves the type unserializable: the save fails with a jai::serialization_error naming it.
• Base and derived fields share a record. A derived field named like a base field writes the same key twice. (In a property_owner chain the derived level wins, as C++ name hiding does.)
• A property saved twice. A hand-written save that calls save_owned_properties(ar) and also writes a JAI_PROPERTY field by name writes that key twice.
• Shared objects read out of order. A shared_ptr writes its object at the first reference the save reaches ({"$id": N, "$val": {...}}; a weak_ptr that comes first writes it as {"$ref": N, "$val": {...}}) and {"$id": N} at every later one, and the load resolves ids in the order load reads its fields. When the save writes inventory and then equipped, both holding one object, a load that reads equipped first throws jai::serialization_error (Dangling shared_ptr reference: {"$id": N} names an object this archive never defined (...)); a weak_ptr read before the field that carries its object loads empty. Read such fields in the order the save writes them. A property_owner's automatic save and load walk its properties in one order (each level by name, base levels first), so this concerns a hand-written pair.
• One graph per call. Object identity lasts one archive, as in Cereal: a shared_ptr saved by two separate to_json calls loads as two objects.
• Exception types. A catch (const cereal::Exception&) becomes catch (const jai::serialization_error&); std::exception covers both while both libraries run.
• Enums are numbers. Both libraries write the underlying integer, so an enumerator inserted mid-list renumbers every value after it in old files. Give enumerators explicit values, or save a name.