4.7.0

What's Changed

• Data Column now exposes definition and repetition levels #288
• ParquetSerializer correctly interprets nulls at various levels of nested structures #282

Full Changelog: 4.6.2...4.7.0

4.6.2

What's Changed

• Class Serialisation Performance Maintenance by @aloneguid in #284

This release has massive performance improvements for ParquetSerializer comparing to ParquetConvert (marked as "Legacy" in tests):

Method	Mean	Error	StdDev	Ratio	RatioSD	Gen0	Gen1	Allocated	Alloc Ratio
Serialise_Legacy	483.0 us	78.04 us	4.28 us	1.00	0.00	73.2422	7.8125	301.23 KB	1.00
Deserialise_Legacy	881.5 us	224.63 us	12.31 us	1.83	0.03	39.0625	12.6953	161.13 KB	0.53
Serialise	417.0 us	140.59 us	7.71 us	0.86	0.02	66.8945	0.4883	273.25 KB	0.91
Deserialise	241.7 us	72.25 us	3.96 us	0.50	0.01	34.6680	0.2441	142 KB	0.47

Full Changelog: 4.6.1...4.6.2

4.6.1

What's Changed

• ParquetSerializer support for append operations by @aloneguid in #280

Full Changelog: 4.6.0...4.6.1

4.6.0

This release contains completely rewritten class serializer using .NET expression trees. All the class serialisation documentation has been updated to reflect this. Existing ParquetConvert serializer is left untouched and marked obsolete, hence there is no increase in the major version number.
If you are using ParquetConvert, consider switching to ParquetSerializer soon, because no new features or fixes will be added to ParquetConvert.

New class serializer supports all primitive types and nested types (structs, lists, maps and their combinations). It also fully conforms to Dremel specification, which was a massive headache to implement properly.

Other improvements:

• Documentation updated for low-level and serializer API on how to use nested data types.
• Corrected how definition and repetition levels are calculated, making it more conformant to Parquet specification.
• Schema path calculation logic improved and does not rely on string splitting/joining, which allows you to use any characters anywhere in column names (#278)

Full Changelog: 4.5.4...4.6.0

4.5.4

What's Changed

• bitpacking range check bug on destination buffer (#267)

Full Changelog: 4.5.3...4.5.4

4.5.3

What's Changed

• Class serialization includes inherited properties by default by @aloneguid in #271
• Ability to use JsonIgnore and JsonPropertyName in serialisation

Full Changelog: 4.5.2...4.5.3

4.5.2

What's Changed

• support null entries in structures by @aloneguid in #264
• SchemaReflector and PropertyHelpers.GetDeclaredPropertyFromClassType … by @clarkfinger in #269

New Contributors

• @clarkfinger made their first contribution in #269

Full Changelog: 4.5.1...4.5.2

4.5.1

What's Changed

• Spark fails to read files if "created_by" metadata field is not in the format is expects. by @aloneguid in #262

Full Changelog: 4.5.0...4.5.1

4.5.0

What's Changed

• Performance and encodings by @aloneguid in #249
• Massive performance improvement in RLE encoder/decoder (tests to follow)
• Support DELTA_BINARY_PACKED encoding.
• Support DELTA_LENGTH_BYTE_ARRAY encoding.
• Support DELTA_BYTE_ARRAY encoding.
• Reading back the schema with adjusted encoding of DateTime, is not re… by @dxdjgl in #248
• Testing the 'Unspecified Kind' behaviour (and Local behaviour) by @xclayl in #253
• Fix issue destroying existing file with append in ParquetConvert.Seri… by @oyhovd in #259

New Contributors

• @oyhovd made their first contribution in #259

Performance

In general, this library outperforms native code in raw write performance which was the aim of this release (see chart). Unfortunately, writing strings is still slightly slower, which will improve in the next versions.

Full Changelog: 4.4.7...4.5.0

4.4.7

What's Changed

• Changes to test code to run on my machine by @xclayl in #250
• Remove new() constraint for Serialize methods by @francoishill in #254

New Contributors

• @xclayl made their first contribution in #250
• @francoishill made their first contribution in #254

Full Changelog: 4.4.6...4.4.7