RSS

Tag Archives: data

UniversalSerializer version 2.0, a universal and easy serializer for .NET

Source code can be downloaded here.

I made a new serializer, replacing the one of the version 1: 70 X faster, files are 100 X shorter, and it needs 110 X less RAM than the old version 1.

  1. What is it ?
    1. Summary
    2. Example of usage
      1. Example with a stream
      2. Example to XML
      3. Example for WPF
      4. Example for WinForms
    3. Why and how is it done ?
      1. Serialize no-default-constructor classes
      2. Serialize not-generic ICollection classes
      3. Reuse existing mechanisms and manage them in ITypeContainers
      4. Filters
        1. Type validator filter
        2. Private fields adder filter
        3. ForcedParametricConstructorTypes
      5. References
    4. Custom ITypeContainer and filters
      1. Making a ITypeContainer
      2. Making a set of filters
    5. The source code of UniversalSerializer
    6. Important points
    7. New version 2
      1. New serializer
      2. New licence
      3. Compatibility with files from version 1
      4. Adapt your sources to the new API
    8. Performances
      1. Processor
      2. File length
      3. RAM
      4. Whole resources
      5. Conclusion
      6. Test conditions
    9. Future
    10. Make the serializers’ job easier
    11. Thanks

What is it ?

UniversalSerializer is a Serializer/Deserializer for .NET/Silverlight/PCL/Windows Phone 8.
In other words, it saves and loads complex object values/instances.
The stream format can be binary, JSON or XML.

It contains DLLs for:

  • .NET 4 or later.
  • Silverlight 4 or later.
  • Portable Class Library 4.
  • Windows Phone 8.
  • Additional DLL for WPF types.
  • Additional DLL for WinForm types.

Summary

The objective of UniversalSerializer is to be able to serialize any type with no effort.

  • No need to add attributes nor interfaces to the types (classes & structures).

  • When a class instance is referenced several times, it is serialized only once.

  • Circular references are allowed.

  • Existing serialization or transcoding mechanisms are reused.
    Currently: [Serializable], ISerializable, [ValueSerializer] and [TypeConverter].

  • No-default-constructor types are allowed, if a parametric constructor can be exploited (it is automatic).

  • Not-generic ICollection classes can be serialized if an Add or an Insert method can be exploited (it is automatic).

Of course, all ordinary constructions as classes, structures, inheritance, public properties, public fields, enumerations, collections, dictionaries, etc.. are serialized by UniversalSerializer.

When a type is not serializable out-of-the-box, UniversalSerializer offers two mechanisms:

  • ITypeContainers
    We can include the problematic type (or set of types) in a custom class that will manage its serialization and deserialization.

  • A set of filters
    We can block some types, and force the serializer to store select source class’ private fields.

My best wish is that people add more Containers and Filters in the future, and maybe one day all types can be serializable.

UniversalSerializer can serialized to 3 formats: custom binary, JSON and XML.

The DLLs are safe (that is: not unsafe), they do not use pointers, not even IL code emission.

Example of usage

var data = new Hashtable();
using (var s = new UniversalSerializer(@"d:\temp\serialized.bin"))
{
s.Serialize(data);
var data2 = s.Deserialize<Hashtable>();
}

That is that simple!

Example with a stream

using (var ms = new MemoryStream())
{
var s = new UniversalSerializer(ms);
s.Serialize(data);
var data2 = s.Deserialize<Hashtable>();
}

Example to XML

using (FileStream fs = new FileStream("TestXmlFormatter.xml", FileMode.Create))
{
Parameters parameters = new Parameters() {
Stream = fs, SerializerFormatter = SerializerFormatters.XmlSerializationFormatter };
UniversalSerializer ser = new UniversalSerializer(parameters);

var data = new Hashtable();
ser.Serialize(data);
var deserialized = ser.Deserialize<Hashtable>();
}

Example for WPF

There is a specialized DLL for WPF, that manages more WPF types:

var data = new System.Windows.Window() { Title = "Hello!" };
using (var s = new UniversalSerializerWPF(@"d:\temp\serialized.bin"))
{
s.Serialize(data);
var data2 = s.Deserialize<System.Windows.Window>();
}

Example for WinForms

There is a specialized DLL for WinForms, that manages more WinForms types:

var data = new System.Windows.Forms.Form() { Text = "Hello!" };
using (var s = new UniversalSerializerWinForm(@"d:\temp\serialized.bin"))
{
s.Serialize(data);
var data2 = s.Deserialize<System.Windows.Forms.Form>();
}

Why and how is it done ?

I needed a universal serializer, able to serialize anything without modification.

But the existing serializers have drawbacks:

  • Some need special attributes or interfaces.
  • Others do not take fields into account.
  • References are a problem. Usually the instances are duplicated, and the deserialized objects point to different instances.
    Circular references are never managed.
  • None can manage class types with no default (no-parameters) constructor, as far as I know.
  • Particular .NET classes need more attention but are sealed therefore can not be inherited with serialization attributes.

So the solution needed a set of mechanisms and techniques.
We will see these mechanisms in the next chapters.

Serialize no-default-constructor classes

In .NET, default (no-parameters) class constructors are not compulsory. But they are needed by almost all serializers.
And they are frequent in the framework. Example: System.Windows.Controls.UIElementCollection.

The solution in UniversalSerializer is to search for parametric constructors, and to find a correspondence in the class between the constructor parameters and the fields, even if these fields are private.

In UIElementCollection, we have this constructor:

public UIElementCollection( UIElement visualParent, FrameworkElement logicalParent )

And these fields are available in the same class:

  • private readonly UIElement _visualParent;
  • private readonly FrameworkElement _logicalParent;

Types are equal and their name is very close. Enough to let UniversalSerializer try to create an instance with these values.
And it works !

Serialize not-generic ICollection classes

For some obscure reason, the ICollection interface does not provide Add nor Insert methods.
In other words, it defines a read-only collection, contrary to the generic ICollection<T>.
Normally, we could not deserialize a class that implement ICollection and not ICollection<T>.

Fortunately, in the real world nearly all of these classes have a Add or a Insert method, at least for internal use.
UniversalSerializer finds these methods and uses them to deserialize the collection class instance.

Reuse existing mechanisms and manage them in ITypeContainers

In some cases, it is more efficient or only possible to use existing transcoding mechanisms.

When I tried to serialize WPF controls, I discovered that:

  • [TypeConverter] lets transcode the object to some types which can be serialized. (string, byte array, etc.).
    Example: System.Windows.Input.Cursor
  • [ValueSerializer] lets transcode from and to a string.
    Example: System.Windows.Media.FontFamily
  • [Serializable] (and ISerializable) allows the use of BinaryFormatter.
    Example: System.Uri

If you consider FontFamily, you will understand that transcoding it to a string is much easier than trying to save its properties.
And safer, because setting a property can lead to unpredictable consequences on unknown or complex classes.

For these attributes, I created the mechanism of ITypeContainer. A container replaces the source instance by its transcoded value, usually a string or a byte array.
A container can be applied to a set of types, for example all the attribute-compatible types.

Examples and details can be found below.

Filters

Some types need special process, which is done by a set of filters.

Type validator filter

This filter allows you to prevent UniversalSerializer from serializing some problematic types.

For example, I met some classes using System.IntPtr .
Serializing this type only leads to problems since they are only used internally in the classes, even when they are stored in public properties/fields.

Private fields adder filter

This filter tells the serializer to add a particular private field to the serialization data.

For example, System.Windows.Controls.Panel needs _uiElementCollection to fill its Children property, since Children is read-only.
With the filter, the solution is easy. And any type that inherits Panel, such as StackPanel, will benefit this filter.

ForcedParametricConstructorTypes

It is not a filter but a list of types.
When a type is in this list, UniversalSerializer ignores its default (not parametric) constructor and searches for a parametric constructor.
Example: System.Windows.Forms.PropertyManager . It is much easier to use its parametric constructor than to write a ITypeContainer for this type.

References

Consider this code:

var data = new System.Windows.Controls.TextBox[2];
data[0] = new System.Windows.Controls.TextBox() { Text = "TextBox1" };
data[1] = data[0]; // Same reference
using (var s = new UniversalSerializerWPF(@"d:\temp\serialized.bin"))
{
s.Serialize(data);
var data2 = s.Deserialize<System.Windows.Controls.TextBox[]>();
data2[0].Text = "New text"; // Affects the two references.
bool sameReference = object.ReferenceEquals(data2[0], data2[1]);
}
  1. UniversalSerializer serializes only one instance of the TextBox.
  2. It deserializes only one instance of TextBox, and creates two references pointing to it.
    Proofs: sameReference is true, and Text is identical in both references.

Custom ITypeContainer and filters

Given that the main objective of UniversalSerializer is to be able to serialize any type, it is essential we share our experience.

So I tried to make the containers and filters creation as easy as possible, to help you play with them and share your solutions.

Making a ITypeContainer

The objective is to replace the problematic instance by an easy instance. In general, the container will contain very simple types (string, int, byte array, etc..).

Let’s take an example:

/// <summary>
/// . No default (no-param) constructor.
/// . The only constructor has a parameter with no corresponding field.
/// . The field ATextBox has no public 'set' and is different type from constructor's parameter.
/// </summary>
public class MyStrangeClassNeedsACustomContainer
{
    /// <summary>
    /// It is built from the constructor's parameter.
    /// Since its 'set' method is not public, it will not be serialized directly.
    /// </summary>
    public TextBox ATextBox { get; private set; }
    public MyStrangeClassNeedsACustomContainer(int NumberAsTitle)
    {
        this.ATextBox = new TextBox() { Text = NumberAsTitle.ToString() };
    }
}

As written in the summary, this class causes some difficulties to the serializer(s).

To overcome the problem, we create a container:

class ContainerForMyStrangeClass : UniversalSerializerLib2.ITypeContainer
{
#region Here you add data to be serialized in place of the class instance
public int AnInteger; // We store the smallest, sufficient and necessary data.
#endregion Here you add data to be serialized in place of the class instance

public UniversalSerializerLib2.ITypeContainer CreateNewContainer(object ContainedObject)
{
MyStrangeClassNeedsACustomerContainer sourceInstance = ContainedObject as MyStrangeClassNeedsACustomerContainer;
return new ContainerForMyStrangeClass() { AnInteger = int.Parse(sourceInstance.ATextBox.Text) };
}
public object Deserialize()
{
return new MyStrangeClassNeedsACustomerContainer(this.AnInteger);
}
public bool IsValidType(Type type)
{
return Tools.TypeIs(type, typeof(MyStrangeClassNeedsACustomerContainer));
}
public bool ApplyEvenIfThereIsAValidConstructor
{
get { return false; }
}
public bool ApplyToStructures
{
get { return false; }
}
}

A detail: all methods behave as static methods (but are not), except Deserialize().

Let’s see them more in details:

  • public int AnInteger

    It is not part of the ITypeContainer interface.
    Here we will store the information we will need later at deserialization.

  • ITypeContainer CreateNewContainer(object ContainedObject)

    Used during serialization.
    This is a kind of constructor for this container instance. The parameter will be the source class instance to serialize.

  • object Deserialize()

    Used during deserialization.
    The container instance will produce a new instance, a copy of the source instance, using our field AnInteger.

  • bool IsValidType(Type type)

    Used during serialization.
    Returns true is the type inherits from, or is, the source type.
    This is a filter.
    We can choose to accept inherited types or not, to accept several compatible types, etc..

  • bool ApplyEvenIfThereIsAValidConstructor

    Used during serialization.
    Returns true if this container applies to class types with a default (no-param) constructor.
    Can be useful to very general containers.

  • bool ApplyToStructures

    Used during serialization.
    Returns true if this container applies to structure types, and not only class types.
    Can be useful to very general containers.

Steps are:

  1. The serializer checks if the source type (MyStrangeClassNeedsACustomerContainer) is managed by a container.
    Our container class (ContainerForMyStrangeClass) answers yes, via IsValidType(), ApplyEvenIfThereIsANoParamConstructor and ApplyToStructures.
  2. The serializer builds an instance of our container, via CreateNewContainer().
    CreateNewContainer builds an instance and sets its field AnInteger.
  3. The serializer stores (serializes) this container instance in place of the source instance.
  4. The deserializer retrieves (deserializes) the container instance.
  5. The deserializer calls Deserialize() and obtains a copy of the source class instance.
    Deserialize() creates this copy using its field AnInteger.

Now we serialize it:

/* This example needs a custom ITypeContainer.
Normally, this class can not be serialized (see details in its source).
But thanks to this container, we can serialize the class as a small data (an integer).
*/

var data = new MyStrangeClassNeedsACustomContainer(123);

using (MemoryStream ms = new MemoryStream())
{
var Modifiers = new CustomModifiers();
Modifiers.Containers = new ITypeContainer[] {
new ContainerForMyStrangeClass()
};

var p = new Parameters() { Stream = ms, customModifiers = Modifiers };
UniversalSerializer ser = new UniversalSerializer(p);

ser.Serialize(data);
var data2 = ser.Deserialize<MyStrangeClassNeedsACustomContainer>();

bool ok = data2.ATextBox.Text == "123";
}

As you can see, the implementation is very easy.

The Tools static class offers some help:

  • Type Tools.TypeIs(Type ObjectType, Type SearchedType)

    It is equivalent to the C#’s ‘is’, but for Types.
    For example, TypeIs((typeof(List<int>), typeof(List<>)) returns true.

  • Type DerivedType(Type ObjectType, Type SearchedType)

    Returns the type corresponding to SearchedType that is inherited by OjectType.
    For example, DerivedType(typeof(MyList), typeof(List<>)) returns typeof(List<int>) when MyList is

    MyList: List<int> { }.

  • FieldInfo FieldInfoFromName(Type t, string name)

    Returns the FiedInfo of the named field of the type.
    We will use it in the next chapter.

Making a set of filters

Please note the filter mechanism is totally independent from the ITypeContainers.
They can be used together, or separately.

Let’s take an example:

public class ThisClassNeedsFilters
{
public ShouldNotBeSerialized Useless;
private int Integer;
public string Value { get { return this.Integer.ToString(); } }
public ThisClassNeedsFilters()
{
}
public ThisClassNeedsFilters(int a)
{
this.Integer = a;
this.Useless = new ShouldNotBeSerialized();
}
}
public class ShouldNotBeSerialized
{
}

This class (ThisClassNeedsFilters) have some problems:

  • It contains a ShouldNotBeSerialized.
    Let’s imagine the class ShouldNotBeSerialized has to be avoided for some reasons, I don’t know why, maybe it is poisoned !
  • The field Integer is not public and therefore is ignored by the serializer(s).
  • Even the constructor parameter name is different from any field or property.
    Anyway the serializer does not need this constructor, as it already has a default constructor.

To overcome these problems, we write a custom set of filters:

/// <summary>
/// Tells the serializer to add some certain private fields to store the type.
/// </summary>
static FieldInfo[] MyAdditionalPrivateFieldsAdder(Type t)
{
if (Tools.TypeIs(t, typeof(ThisClassNeedsFilters)))
return new FieldInfo[] { Tools.FieldInfoFromName(t, "Integer") };
return null;
}
/// <summary>
/// Returns 'false' if this type should not be serialized at all.
/// That will let the default value created by the constructor of its container class/structure.
/// </summary>
static bool MyTypeSerializationValidator(Type t)
{
return ! Tools.TypeIs(t, typeof(ShouldNotBeSerialized));
}

They are self-explanary:

  • FieldInfo[] MyAdditionalPrivateFieldsAdder(Type t)

    makes the serializer add a private field (Integer) to every source instance of this type (ThisClassNeedsFilters).

  • bool MyTypeSerializationValidator(Type t)

    prevents the serializer from storing any instance of this type (ShouldNotBeSerialized).
    Consequently, any instance of ThisClassNeedsFilters will not set the Useless field.

Now we serialize it:

/* This example needs custom filters.
Normally, this class can be serialized but with wrong fields.
Thanks to these filters, we can serialize the class appropriately.
*/

using (MemoryStream ms = new MemoryStream())
{
var Modifiers = new CustomModifiers();
Modifiers.FilterSets = new FilterSet[] {
new FilterSet() {
AdditionalPrivateFieldsAdder=MyAdditionalPrivateFieldsAdder,
TypeSerializationValidator=MyTypeSerializationValidator } };

var p = new Parameters() { Stream = ms, customModifiers = Modifiers };
var ser = new UniversalSerializer(p);

var data = new ThisClassNeedsFilters(123);
ser.Serialize(data);
var data2 = ser.Deserialize<ThisClassNeedsFilters>();

bool ok = data2.Value == "123" && data2.Useless == null;
}

The implementation is even easier than with ITypeContainer.

The source code of UniversalSerializer

All sources are C#.

The solutions have been created with Visual Studio 2012, but should be compatible with VS 2010 (not tested).
Compilation only needs .NET 4, Silvberlight 4 or PCL 4, not any third-party DLLs, the source code is complete.

There are 6 DLLs:

  • UniversalSerializer2.dll
    The main general DLL for .NET 4 or later, with very few dependencies.

  • UniversalSerializer2ForSilverlight.dll
    For Silverlight 4 or later.

  • UniversalSerializerPortableLib2.dll
    A Portable Class Library you can use on many frameworks (tested under .NET).

  • UniversalSerializer2ForWindowsPhone8.dll
    For Windows Phone 8 (not much tested, please help).

  • UniversalSerializerWPF2.dll
    A specialized DLL for WPF, that manages more WPF types.

  • UniversalSerializerWinForm2.dll
    A specialized DLL for WinForms, that manages more WinForms types.

In the source archive, you will find these VS solutions:

  • “UniversalSerializer Benchmark\UniversalSerializer Benchmark.sln”
    This application compares resource consumption of 12 serializers on .NET, including 4 versions of UniversalSerializer.

  • “UniversalSerializer Desktop Debug Libs\UniversalSerializer Desktop Debug Libs.sln”
    This solution compiles the 4 DLLs for the desktop, in debug mode.

  • “UniversalSerializer Desktop Release Libs\UniversalSerializerReleaseLibs.sln”
    This solution compiles the 4 DLLs for the desktop, in release mode.

  • “UniversalSerializer Lib\UniversalSerializer.csproj”
    This project compiles the main library.
    You can add this project to your own solution to be able to debug the DLL.

  • “UniversalSerializer Silverlight\UniversalSerializer Silverlight.sln”
    This solution contains a test application for Silverlight and the corresponding DLL project.

  • “UniversalSerializer Windows Phone 8 experimental\UniversalSerializer Windows Phone 8.sln”
    This solution contains a test application for Silverlight for Windows Phone 8 and the corresponding DLL project.
    Please note this DLL will not run on Windows Phone 7.1 (the property ParameterExpression.IsByRef does not exist on that platform and makes the DLL to not work properly). I did not try WP 7.5.

  • “UniversalSerializerPortableLib\UniversalSerializerPortableLib.sln”
    This solution contains a test application and the corresponding PCL (Portable Class Library) project.

  • “UniversalSerializerWinForm\UniversalSerializerWinForm.sln”
    This solution contains a test application for WinForm and the corresponding DLL projects (the general DLL and a specialized DLL).

  • “UniversalSerializerWPF\UniversalSerializerWPF Tester\UniversalSerializer Tester.sln”
    This solution contains a test application for WPF and the corresponding DLL projects (the general DLL and a specialized DLL).

Important points

  • Versioning.
    The serialization/deserialization process identifies types by their complete name (from type.AssemblyQualifiedName).
    This kind of name depends on the type’s assembly version, therefore please take care of framework and DLL versioning !

  • File format.
    The current stream format is subject to changes in the future.
    For that reason the DLL names have a version number.
    If you serialize to a file, I suggest you to add a version number to the file name.

New version 2

New serializer

I have written a new serializer from nothing.
The code source does not contain anything from FastBinaryJSON any more.

The differences are:

  • 70 X faster, files are 100 X shorter, it needs 110 X less RAM.
    As measured in the benchmark (200 k items, 5 loops, 3 bytes structure).
    Now this serializer is fast and efficient.
  • The serializer works with streams, not with byte arrays.
    The objective are to consume very few memory and to write to disk (or to any stream destination) as quick as possible.
    An important consequence is we can serialize to a compressed stream directly, there is no intermediary process or structures. I see that as essential in order to serialize very big data or collections.

  • Three file formats are supported: custom binary, JSON and XML.
    And you can create your own formats using the Formatter interface.

  • A new multiplex binary format.
    This format has been designed with .NET structures in mind, and to facilitate, accelerate and reduce the size of the serialized data.
    This format integrates the concepts of collections, dictionaries, properties, fields, instances & references.
    Type descriptors and object values are multiplexed in the same stream.
    Please note this format has nothing in common with the one of FastBinaryJSON, therefore is not compatible with UniversalSerializer version 1.

  • Some improvements over version 1:

    • Parametric constructors have now priority over containers for structures (ex: KeyValuePair<,>).
    • Nullable<T> is managed correctly.
    • Integers can optionally be compressed as 7-bits variable length. Please note that uncompressed integers can occasionally be compressed better by an external compressor (as WinRar).

New licence

As the code is completely genuine, contrary to the version 1, I have had the possibility to change the license.
I chose Ms-RL, a license that lets you use the source in any project. The only notable condition is if you modify the source you have to set this modification public (only the file you modified, not any other files).
I hope that will help to share containers and improvements that can be useful to many.

Compatibility with files from version 1

Since the formats are completely different, there is no compatibility at all.
If you want to read data you wrote in the old format, I suggest you to reference the old “UniversalSerializer1.dll”. DLLs version 1 and 2 can be linked together as they use different namespaces.

Adapt your sources to the new API

There are a few differences from API of version 1:

  • The namespace is now UniversalSerializer2.
  • The serializer now works with streams.

  • Now you have to make a new instance of UniversalSerializer first, then call Serialize and/or Deserialize().

  • ITypeContainer.ApplyEvenIfThereIsANoParamConstructor has been renamed as ITypeContainer.ApplyEvenIfThereIsAValidConstructor.

Performances

During development, I needed to compare this serializer with the existing serializers, in order to eliminate possible weaknesses.
I created a simple benchmark that counts the resource needs: processor, stream and RAM.
Although benchmarks can be discussed indefinitely, because they are far from perfection and they depend on many parameters, they keep some interest. We only have to not forget they are not very precise.

The data we will serialize and deserialize as a test is an array of a small structure constituted by 3 bytes.
The array contains 200,000 items of this structure, and we serialize and deserialize it 5 times.
See below a chapter giving more details about the test conditions.

Processor

UniversalSerializer is listed 4 times: binary format, JSON format, XML format and the old version 1.
The slowest of all serializers is the old version 1.0 of UniversalSerializer, which was based on FastBinaryJSON but with a big additional resource waste (shame on me !).

File length

The produced file length can be important to your project. That is why I did an effort in order to reduce the data waste.
In fact UniversalSerializer tries to eliminate all superfluous data in its structure. In this example, a perfect uncompressed file length would be 600,000 bytes long. UniversalSerializer is only 338 bytes upper, due to the type descriptors.

Protobuf-net is a bit particular because it seems to do a kind of compression. That is more evident with another data structure containing 3 Int32 (in place of 3 bytes): Protobuf-net produces the same file length: about 8,8 bytes per structure.
For information, by default UniversalSerializer compresses integers (except bytes) with a 7-bits compression scheme and produces a 897,750 bytes file when serializing the 3 Int32 structure array (that is 4,5 bytes/structure on average). This compression can be switched off.

The old UniversalSerializer version 1 goes on being ridiculous. I let it in the diagram, it is too funny !

RAM

Counting the RAM consumption is a tough and uncertain job.
Here I present the GC Memory consumption, using System.GC.GetTotalMemory(..). It should be noted that we obtain different numbers using System.Diagnostics.Process.GetCurrentProcess().WorkingSet64.
But after many tries, it seems the GC method is more stable and significant.

I did a particular effort in UniversalSerializer to reduce the RAM consumption. The result seems to reflect these efforts.
Once again, the old UniversalSerializer version 1 produces astronomical numbers, consuming about 700 Mio of RAM. What a joke ! That is 110 times the consumption of the new version 2. That makes me laugh.  😉

An important lesson I leaned here is: the bigger is the array we serialize, the bigger will be the resources consumed by a serializer.
This lesson seems to be valid for all serializers (yes, including UniversalSerializer).

Whole resources

Here we try to evaluate the whole resource consumption of each serializer.
Numbers are percentages based on resource consumption of UniversalSerializer (binary).
The Total number is the mean of the three resource percentages. For example, Protobuf-net: time=137%, file length=277%, ram=208%, mean=(137+277+208)/3=207 %.

The usual joke is the total resource consumption of the old UniversalSerializer version 1: 9,187 %. Bigger than an elephant ! It was too enormous for the diagram, I had to cut-it off.   😉

Conclusion

My personal conclusion is I reached my goals: create a resource-saving serializer that let us serialize many kinds of types.
The Xml and Json formatters consume too many resources, but I know I can improve them greatly by replacing the .NET classes currently in use.

Test conditions

Some details about the conditions of this test.

The benchmark project sources can be found in the main source archive of UniversalSerializer.
That lets you examine it and tell me if you think something is wrong in my methodology.

A capture of this application:

The structure we serialize:

public struct MyByteColor
{
public byte R;
public byte G;
public byte B;
}

Some points to be considered:

  • The tests are run once only (to avoid pre-caching).
  • There is no virtual memory file in Windows (to avoid disk accesses when a lot of memory is needed by a serializer).
  • The file stream is written to a RAM-disk (to avoid disk accesses).
  • GC memory is measured continuously by a timer (every 60 ms) in another thread, in order to obtain its peak value and not only its final value.

Future

I would like to improve some aspects:

  • Add more ITypeContainers and filters.
    You can help me. Tell me when types are not serialized correctly. And please share your solution, as containers & filters.
  • Make a DLL for Windows Runtime.

Make the serializers’ job easier

This experience taught me why serializers have difficulties with some types and how we can facilitate its job when we author a class.

  1. Write a default (no-param) constructor when possible.
    The serializer will reconstruct the instance from its fields and properties.
  2. If you can not write a default (no-param) constructor, write a public parametric constructor with corresponding private fields.
    The field should have the same type and the same name. With UniversalSerializer, the name can be slightly different: Param -> param or _param or _Param.
  3. Implement a ValueSerializerAttribute or a TypeConverterAttribute when an instance can be constructed from something as simple as a string.
    Especially when your class contains many public optimisation fields or properties.
    For example, FontFamily can be constructed from a simple string, no matter if it contains many other informations they all come from this simple string.
    I don’t know if many serializers take these attributes into accounts, but at least UniversalSerializer does.
  4. All optimisation (as caching) fields or properties should be private, when possible. See above.
  5. When you create an object collection, implement IList.
      Because ICollection does not allow to add items to the collection.
  6. When you create a generic collection, implement ICollection<>.
      Because IEnumerable<> does not allow to add items to the collection.

Thanks

I thank Mehdi Gholam for his fastBinaryJSON.
I learned interesting things reading his code.
Thank you for sharing it.

Thank the Mono team also, I used one function of this nice framework.

And I thank the Protobuf-net team, not only because I used one function of their work, but because their serializer forced me to improve mine a lot also. A useful competition.   🙂

 

Tags: , , , , , , , , , , , ,