🌶️ Eyeshot Proprietary File Format

Eyeshot 11 introduces a new proprietary file format that is versioned. It provides options to save geometry only, tessellation only, or both, allowing users to balance file size and loading speed according to their needs. Additionally, the file format supports entity types extension and file compression.

This new file format ensures backward compatibility by enabling the reading of older version files whilst only saving files in the current version. To determine the latest supported version, you can check the field Serializer.LastVersion.

Serialization strategy

The Eyeshot file format is a binary format with a header and a body, based on the protobuf-net open-source library.

General concepts

  • The surrogate refers to a set of fields used for storing the state of an object. Every object in Eyeshot has its own surrogate and implements the method ConvertToSurrogate() to convert its state into the surrogate representation.

  • The model holds the hierarchical collection of objects and defines the fields for each surrogate. It provides the structure and organization for the serialization process.

  • The serializer handles the serialization and deserialization operations based on the specified model. It is responsible for converting objects and their states into a format that can be stored or transmitted, and for reconstructing the objects from the serialized data.

Relevant classes

devDept.Serialization namespace:

  • FileHeader: This class contains essential information such as the file version, content type1, serialization mode2, file mode3, units, and more. It provides basic metadata for the file.
  • FileBody: This class stores all the Eyeshot objects, including entities, layers, blocks, and other master collections. It encapsulates the actual content of the file.
  • FileSerializer: This class is responsible for serializing and deserializing Eyeshot objects into the proprietary file format. It handles the conversion of objects and their properties to the surrogate representation, as well as the reconstruction of objects from the serialized data.

devDept.Eyeshot.Translators namespace:

  • ReadFile: This helper class provides functionality to open and read files. It assists in loading and parsing Eyeshot files, making it easier to access the content within.
  • WriteFile: This is a helper class used for saving files. It facilitates the process of writing and saving Eyeshot files, allowing for easy preservation of the content.

These classes are part of the Eyeshot framework and serve different purposes related to serialization, file handling, and data translation.

1 geometry: includes only geometrical data, i.e. for a circle Plane and radius (smaller file size, but slower loading process).
  tessellation: includes only tessellation data, i.e. for a circle the vertices (loading the file, the circle will be represented as a linear path)
  geometry&tessellation: includes both geometrical and tessellation data, i.e. for a circle Plane, radius, and vertices (bigger file size, but quick loading process).

2 Compressed or Uncompressed 
3 Standard or Assembly (multi-file) 

Code snippets

Open file

ReadFile readFile = new ReadFile(filename);
readFile.DoWork();
readFile.AddTo(design1);
design1.Invalidate();

Save file

WriteFile writeFile = new WriteFile(new WriteFileParams(design1), filename);
writeFile.DoWork();

Extendibility

Eyeshot provides easy extensibility for incorporating custom entities and data.
To achieve this, you need to follow these steps:

  • Define surrogates for your custom objects: Implement surrogates to represent the properties and state of your custom objects in a serializable format.

  • Create a derived class from FileSerializer: Inherit from the FileSerializer class and override the  FillModel() method. This allows you to handle the deserialization process for your custom objects and populate the model accordingly.

  • Use the Tag field of FileHeader: The Tag field in FileHeader can be used to manage the versioning of your custom objects. You can assign a version or identifier to the Tag field to ensure compatibility and handle any changes in future versions.

A recommended resource to get started is the FileFormatExtension source code sample. It provides a useful starting point and demonstrates how to extend the file format to include custom entities and data.

using devDept.Serialization
   
public class CustomDataSurrogate : Surrogate<CustomData> {     public CustomDataSurrogate(CustomData obj) : base(obj) // The base calls the CopyDataFromObject method. { }
public int Id; public string Description; public float Price;
protected override CustomData ConvertToObject() { CustomData cd = new CustomData(Id); CopyDataToObject(cd); return cd; }
protected override void CopyDataToObject(CustomData cd) { cd.Description = Description; if (Tag == "1.1") cd.Price = 100; // forces the price for old version 1.1. else cd.Price = Price; } protected override void CopyDataFromObject(CustomData cd) { Id = cd.Id; Description = cd.Description; Price = cd.Price; }
// those static methods must be created only for classes that derive from the Surrogate base class, no for custom entity surrogate, i.e. MyCircleSurrogate  #region Static Methods public static implicit operator CustomData(CustomDataSurrogate surrogate) { return surrogate == null ? null : surrogate.ConvertToObject(); } public static implicit operator CustomDataSurrogate(CustomData source) { return source == null ? null : source.ConvertToSurrogate(); } #endregion }
public class MyFileSerializer : FileSerializer
{
    // Tag to handle custom versions.
    public static string CustomTag = "1.2";
    
    // Constructor used in conjunction with the WriteFile class.   
    public MyFileSerializer()
    {
    }
    
    // Constructor used in conjunction with the ReadFile class.
    public MyFileSerializer(contentType contentType) : base(contentType)
    {
    }   

    protected override void FillModel()
    {
        if (ModelIsCompiled()) return;

base.FillModel(); // Adds the CustomData to the protobuf model and defines its surrogate. Model.Add(typeof(CustomData), false) .SetSurrogate(typeof(CustomDataSurrogate)); // Defines properties for CustomDataSurrogate MetaType mt = Model[typeof(CustomDataSurrogate)] .Add(1, "Id") .Add(2, "Description"); // Use the header tag to handle different definitions for your custom model. if (this.HeaderTag == "1.2") mt.Add(3, "Price"); mt.SetCallbacks(null, null, "BeforeDeserialize", null); // Fills Version and Tag during the deserialization. mt.UseConstructor = false; // Avoids to use the parameterless constructor during the serialization process. } protected override Type GetTypeForObject(string typeName) { // this override is available since build 12.0.518 Type t = Type.GetType(typeName, false, true); if (t != null) return t; return base.GetTypeForObject(typeName); } }

If you need to serialize a derived class of an existing Eyeshot class, you may find this article helpful.

UPDATES

  • Starting from Eyeshot 2020, the file format includes built-in support for circular references within Entity classes.
  • Starting from Eyeshot 2021, the multi-file format is natively supported. This allows you to split a large project into multiple files, making it more organized and manageable.

    Furthermore, Eyeshot now enables you to store custom data within the block object. This means that you can associate additional information or metadata with blocks in your project.

    Save the file with custom data
    design1.RootBlock.CustomData = customObject;
    var wfp = new WriteFileParams(design1);
    var wf = new WriteFile(wfp, fileName, new MyFileSerializer());
    wf.DoWork();
    Open file with custom data
    var rf = new ReadFile(fileName, new MyFileSerializer());
    rf.DoWork();
    var customObject = rf.Blocks[rf.Blocks.RootBlockName].CustomData;
  • Starting from version 2023.3.651, the Workspace.FileSerializerForCopyAndPaste property has been introduced. This property allows specifying the type of custom FileSerializer to be used in Eyeshot's CopyAndPaste functionality.

UNIT TESTING

We perform an extensive number of tests to verify backward reading compatibility for previous versions.

If you decide to extend the functionality of Eyeshot or make customizations, we highly recommend conducting similar tests to maintain backward compatibility.
By running unit tests on your customizations, you can ensure that they work seamlessly with different versions of Eyeshot.

We encourage you to share your unit test project with us. This will enable us to notify you of any changes that might impact your customizations.
By staying informed about updates and changes, you can adapt your customizations accordingly and ensure a smooth transition between different versions of Eyeshot.

 

Was this article helpful?
1 out of 1 found this helpful

Comments

2 comments
Date Votes
  • I currently use BinaryFormatter. This is very inefficient compared to Protobuffers. But BinaryFormatter is so easy to use since the class description is included in the file. I would like to switch to Protobuffers for so many reasons.

    In this post, it looks like it is required to make a surrogate class for each of my 'real' classes. This is painful because I have 100s of my classes. Is there some automated way to do this?

    https://github.com/mgravell/protobuf-net - In this project, they use attributes. Even this is time consuming. They also have a method of creating a .proto file from the C# source that contains the integer/property definitions. Then the C# code does not require any changes. Maybe this would not work for you since you are deciding what to serialize at runtime (ie: tessellation)

    Also, does this support reference map serialization? A great feature of BinaryFormatter is it contains a reference map. So if you serialize circular references, or the same object multiple times, it deserializes exactly the same. I'm sure protobuffers does not by default, but there maybe an options.

    I guess it is not possible to serialize Eyeshot objects with other protobuf libraries such as the protobuf-net library above? I would prefer to use your system directly.

    -Jas

    0
  • Hi James,

    protobuf-net supports the attributes definition but we use the surrogates to keep a clear separation between the Eyeshot objects definition and the one used for the serialization purpose. In this way, we can better handle versioning and overcome some limits about unsupported types in protobuf-net, like object, enum, multi-dimensional array, etc. Moreover, as you wrote, we can decide what to serialize at runtime according to the content type.

    Regarding the serialization of circular references, you can search on the web, this is one of the first result: https://stackoverflow.com/questions/7185632/understanding-protobuf-net-asreference-with-recursive-referencing

    0

Please sign in to leave a comment.

Articles in this section