StructTypes.jl

This guide provides documentation around the StructTypes.StructType trait for Julia objects and its associated functions. This package was born from a desire to make working with, and especially constructing, Julia objects more programmatic and customizable. This allows powerful workflows when doing generic object transformations and serialization.

If anything isn't clear or you find bugs, don't hesitate to open a new issue, even just for a question, or come chat with us on the #data slack channel with questions, concerns, or clarifications.

StructTypes.jl
- StructTypes.StructType

`StructTypes.StructType`

In general, custom Julia types tend to be one of: 1) "data types", 2) "interface types" or sometimes 3) "custom types" or 4) "abstract types" with a known set of concrete subtypes. Data types tend to be "collection of fields" kind of types; fields are generally public and directly accessible, they might also be made to model "objects" in the object-oriented sense. In any case, the type is "nominal" in the sense that it's "made up" of the fields it has, sometimes even if just for making it more convenient to pass them around together in functions.

Interface types, on the other hand, are characterized by private fields; they contain optimized representations "under the hood" to provide various features/functionality and are useful via interface methods implemented: iteration, getindex, accessor methods, etc. Many package-provided libraries or Base-provided structures are like this: Dict, Array, Socket, etc. For these types, their underlying fields are mostly cryptic and provide little value to users directly, and are often explictly documented as being implementation details and not to be relied upon directly under warning of breakage.

What does all this have to do with the StructTypes.StructType trait? A lot! There's often a desire to programmatically access the "public" names and values of an object, whether it's a data, interface, custom or abstract type. For data types, this means each direct field name and value. For interface types, this means having an API to get the names and values (ignoring direct fields). Similarly for programmatic construction, we need to specify how to construct the Julia structure given an arbitrary set of key-value pairs.

For "custom" types, this is kind of a catchall for those types that don't really fit in the "data" or "interface" buckets; like wrapper types. You don't really care about the wrapper type itself but about the type it wraps with a few modifications.

For abstract types, it can be useful to "bundle" the behavior of concrete subtypes under a single abstract type; and when serializing/deserializing, an extra key-value pair is added to encode the true concrete type.

Each of these 4 kinds of struct type categories will be now be detailed.

DataTypes

You'll remember that "data types" are Julia structs that are "made up" of their fields. In the object-oriented world, this would be characterized by marking a field as public. A quick example is:

struct Vehicle
    make::String
    model::String
    year::Int
end

In this case, our Vehicle type is entirely "made up" by its fields, make, model, and year.

There are three ways to define the StructTypes.StructType of these kinds of objects:

StructTypes.StructType(::Type{MyType}) = StructTypes.Struct() # an alias for StructTypes.UnorderedStruct()
# or
StructTypes.StructType(::Type{MyType}) = StructTypes.Mutable()
# or
StructTypes.StructType(::Type{MyType}) = StructTypes.OrderedStruct()

`StructTypes.Struct`

StructTypes.Struct — Type

StructTypes.StructType(::Type{T}) = StructTypes.Struct()
StructTypes.StructType(::Type{T}) = StructTypes.UnorderedStruct()
StructTypes.StructType(::Type{T}) = StructTypes.OrderedStruct()

Signal that T is an immutable type who's fields should be used directly when serializing/deserializing. If a type is defined as StructTypes.Struct, it defaults to StructTypes.UnorderedStruct, which means its fields are allowed to be serialized/deserialized in any order, as opposed to StructTypes.OrderedStruct which signals that serialization/deserialization must occur in its defined field order exclusively. This can enable optimizations when an order can be guaranteed, but care must be taken to ensure any serialization formats can properly guarantee the order (for example, the JSON specification doesn't explicitly require ordered fields for "objects", though most implementations have a way to support this).

For StructTypes.UnorderedStruct, if a field is missing from the serialization, nothing should be passed to the StructTypes.construct method.

For example, when deserializing a Struct.OrderedStruct, parsed input fields are passed directly, in input order to the T constructor, like T(field1, field2, field3). This means that field names may be ignored when deserializing; fields are directly passed to T in the order they're encountered.

Another example, for reading a StructTypes.OrderedStruct() from a JSON string input, each key-value pair is read in the order it is encountered in the JSON input, the keys are ignored, and the values are directly passed to the type at the end of the object parsing like T(val1, val2, val3). Yes, the JSON specification says that Objects are specifically un-ordered collections of key-value pairs, but the truth is that many JSON libraries provide ways to maintain JSON Object key-value pair order when reading/writing. Because of the minimal processing done while parsing, and the "trusting" that the Julia type constructor will be able to handle fields being present, missing, or even extra fields that should be ignored, this is the fastest possible method for mapping a JSON input to a Julia structure. If your workflow interacts with non-Julia APIs for sending/receiving JSON, you should take care to test and confirm the use of StructTypes.OrderedStruct() in the cases mentioned above: what if a field is missing when parsing? what if the key-value pairs are out of order? what if there extra fields get included that weren't anticipated? If your workflow is questionable on these points, or it would be too difficult to account for these scenarios in your type constructor, it would be better to consider the StructTypes.UnorderedStruct or StructTypes.Mutable() options.

struct CoolType
    val1::Int
    val2::Int
    val3::String
end

StructTypes.StructType(::Type{CoolType}) = StructTypes.OrderedStruct()

# JSON3 package as example
@assert JSON3.read("{"val1": 1, "val2": 2, "val3": 3}", CoolType) == CoolType(1, 2, "3")
# note how `val2` field is first, then `val1`, but fields are passed *in-order* to `CoolType` constructor; BE CAREFUL!
@assert JSON3.read("{"val2": 2, "val1": 1, "val3": 3}", CoolType) == CoolType(2, 1, "3")
# if we instead define `Struct`, which defaults to `StructTypes.UnorderedStruct`, then the above example works
StructTypes.StructType(::Type{CoolType}) = StructTypes.Struct()
@assert JSON3.read("{"val2": 2, "val1": 1, "val3": 3}", CoolType) == CoolType(1, 2, "3")