Parsing/exporting a truckfile

Status
Not open for further replies.

only_a_ptr

Infamous Developer
Administrator
Developer
Joined
Jan 31, 2018
Messages
167
Location
Prague, CZ
Our Truck Definition Fileformat (TDF) (which I call just truckfile) is designed as sequential (order matters), which works nicely in a text editor, but once you start thinking about creating an interactive editor, it suddenly gets ugly. @Max98 is currently struggling with it in rorEditor, I struggled before in RigEditor (never released). To move ahead, I'd like to share my experience and make an overview of all the truck-parsing code we have around.

What's the deal: Truckfile consists of sections (i.e. nodes/beams/wheels/flares) which are simple to work with. However, those are interleaved by directives (set*defaults) which affect all following lines. Directives don't respect section boundaries, wherever they are, they affect all the rest of the truckfile. Sometimes they invoke special behavior (meshwheels2). To make it even more complicated, there are special group comments ";grp: " which are technically just comments but modder would expect them to survive a load/save in editor.

Truckfile in detail: it's big and hairy, it has it's own readme (my work, long and not entirely up to date), so I'll re-iterate from my head here:
  • data come in sections (nodes/beams/wheels) and inline-sections (fileformatversion/extcamera). Order in section matters; flares are referenced by position (starting from 0) in materialflarebindings, axles are referenced by interaxles (starting from 1 for a change)...
  • Top-level directives modify default values for all lines which come after them (until the same directive appears again, of course): set_beam_defaults, set_node_defaults, set_default_minimass, set_inertia_defaults, set_managedmaterials_options, detacher_group
  • Special directive: set_beam_defaults_scale - sets scale factors for set_beam_defaults values, does not reset with next set_beam_defaults, only with another _scale.
  • "Section config" feature (keywords: section/end_section) - I call these "Modules" in my readme and RoR codebase because 'section' is ambiguous.
How RoR parses truckfile: Up to RoR v0.4.0x, RoR parsed the truckfile and spawned the truck at the same time. There were fixed-size arrays for nodes[1000] and beams[5000]. You were supposed to defined nodes with sequential numbers, but it was only loosely checked and referencing a non-existent node (maybe defined later) was completely unchecked (and also may have passed un-noticed due to the fixed-size arrays). Overall it was one huge slab of code (single function parsing the file and creating physics objects/OGRE objects). Take a look in my code archive. This was the first thing I tackled when I joined RoR as developer, and although it didn't go smoothly, my parser infrastructure is the foundation of modern RoR.
  • RoR parses truckfile separately from all other activities, data are kept in `RigDef::File` object, which consists of `RigDef::File::Module` objects (one default, other for each sectionconfig) and some global attributes. I called it `RigDef` because I didn't like 'truck' (may as well be a wooden box).
  • Sections are simple: each has corresponding `RigDef::` object which represents a single line in truckfile (`RigDef::Node` is one node, `RigDef::Engine` is engine, `RigDef::Engoption`...)
  • Directives are mostly called *Defaults, but I really think of them as "presets" - I use this terminology in my unfinished RigEditor. Each directive has it's object (`RigDef::BeamDefaults` etc...) and all sections which care have smart pointers to these objects (examples: Node references NodeDefaults, Rope references BeamDefaults). The pointers are never empty, parser creates default objects at the start.
  • The tricky set_beam_defaults_scale is merged into `RigDef::BeamDefaults`. When _scale line is read, new object is created with updated scale params.
  • Support for ;grp: grouping comments was only coded recently, only present in AngelHUD branch: https://github.com/RigsOfRods/rigs-...mits/20a8ea10c342f4ff38bda2440e2d1c79e53ef53b
  • When RoR spawns the truck, it processes the sections in fixed order, so all the "this must go before that" rules in the doc (and most often undocumented, only known to old IRC users) are now obsolete.
  • The node array is now dynamically sized to fit, and there is a fixed ordering of node types. I wrote a lot of commentary on the topic in code
How RoR exports truckfile: it actually doesn't, except in AngelHUD branch. The exporter was already written back in 2014 for the RigEditor, but it was just collecting dust until now.
  • code structure and terminology is the same as in parser: document(truckfile) consists of modules (sectionconfigs), sections are written in fixed order: https://github.com/only-a-ptr/rigs-.../rig_def_fileformat/RigDef_Serializer.cpp#L95
  • nodes are handled primitively: just loop over the RigDef::Node array and if current ;grp () or set_node_defaults is different from previous line, write a ;grp line or set_node_defaults line. Since RoR doesn't modify nodes at the moment, nothing smarter is needed. Code
  • beams are smarter: they are first grouped by set_beam_defaults (code openly says "by presets"), but ;grp handling is just as crude as in nodes. For now. Code
  • other beam types repeat the same logic - code for shocks
 
(continued)

How my modified Blender addon does it: I coded this last year but never published it, it's an extended version of Ulteq's import/export scripts, it adds GUI to manipulate set_beam_defaults/set_node_defaults and also calls it "presets". see https://github.com/only-a-ptr/ror-blender
How RigEditor would do it, if it was functional: this is technically the most interesting part, unfortunately it references frozen code. see https://github.com/only-a-ptr/rigs-of-rods/tree/rig-editor-2017/source/main/rig_editor
Other code: I'm out of time to write this post, there's also python truckparser in https://github.com/CuriousMike56/RoRServerScripts and some older stuff is referenced in https://github.com/only-a-ptr/ror-blender README.
 
Last edited:
Status
Not open for further replies.
Back
Top