From 4f6535eeab106cb5976d909bdff2d66452a81e43 Mon Sep 17 00:00:00 2001 From: PeninaR Date: Tue, 21 Aug 2018 16:10:38 +0300 Subject: [PATCH] DOC-405 moved Reducing the Space Object Footprint into main docs --- ...wering-the-space-object-footprint.markdown | 350 ------------------ .../12.3/dev-java/custom-matching.markdown | 21 +- ...wering-the-space-object-footprint.markdown | 344 +++++++++++++++++ 3 files changed, 355 insertions(+), 360 deletions(-) delete mode 100644 site/content/sbp/lowering-the-space-object-footprint.markdown create mode 100644 site/content/xap/12.3/dev-java/lowering-the-space-object-footprint.markdown diff --git a/site/content/sbp/lowering-the-space-object-footprint.markdown b/site/content/sbp/lowering-the-space-object-footprint.markdown deleted file mode 100644 index 76fccc210..000000000 --- a/site/content/sbp/lowering-the-space-object-footprint.markdown +++ /dev/null @@ -1,350 +0,0 @@ ---- -type: post -title: Reducing the Space Object Footprint -categories: SBP -parent: data-access-patterns.html -weight: 900 ---- - - - -|Author|XAP Version|Last Updated | Reference | Download | -|------|-----------|-------------|-----------|----------| -|Shay Hassidim| 7.1 | July 2009 | | | - - - - -# Overview -By default, when using the GigaSpaces Java API, the Space stores Space object fields as is. No data compaction or compression is done while the object is transported across the network, or when stored within the Space. - -{{% note %}} - -- The Compressed Storage Type is a different approach than the compact serialization, pattern, and compresses non-primitive fields using the zip utilities. -- The Binary Storage Type stores non-primitive fields within the Space in its byte array form. It does not compress or reduce the footprint of the data as the compact pattern does. This approach makes it unnecessary to introduce the nested Space object data type to the Space JVM, or to de-serialize them on the Space side. The Binary Storage Type may improve performance when the Space object stores large collections. -- C++ and .Net API data objects may undergo some compaction when sent across the network. -{{% /note %}} - -With the Compact Serialization pattern, you can reduce the footprint of the data object in the Space memory when it is stored in the data grid, which allows storing more Space objects per memory unit. This pattern works very well when the Space object includes a large number of numerical values, because these objects are stored as a more optimal data type. - -The compact serialization pattern enables complete control over the format of the data object while transported over the network and when stored within the Space. This technique: - -- Compacts the object payload data when transported over the network and when stored in memory. -- Avoids the de-serialization involved when space object written to the space from a remote client (for non primitive fields such as user defined classes or collection field) -- Avoids the de-serialization replicated to a backup space(s) -- Avoids the serialization involved when reading an object back from the space into the client process (at the space side). - -# The Basic Flow -With the compact serialization pattern: - -- Before you write the object you serialize all non indexed fields (payload data) into one byte array field using the GigaSpaces serialization API (pack). -- You serialize/de-serialize all indexed fields as usual (have the `writeExternal` , `readExternal` implementation to write and read these into the stream). -- After reading the object from the space you should de-serialize the byte array data (unpack). - -{{% align center %}} -![bin_ser.jpg](/attachment_files/sbp/bin_ser.jpg) -{{% /align %}} - -When the object is written to the space: - -- The non-indexed fields are compacted and serialized into the same field (as a byte array). -- All the indexed fields + the byte array are serialized as usual via the `writeExternal` call. -- The object with arrives in the space, de-serialized, indexed fields stored as usual and the byte array field stored as is. - -When the object is read from the space: - -- The read template undergoes the same actions as when writing an object to the space -- The matching object is serialized and sent to the client. -- When the matching object arrives the client side is it de-serialized, and the byte array data is de-serialized and expand (in a lazy manner). - -# The Implementation -Using the compact serialization pattern can reduce the object footprint when stored within the space in drastic manner. As much as you will have more fields as part of the space object serialized using the GigaSpaces Serialization API, the memory footprint overhead will be smaller compared to the default serialization mode. - -The compact serialization pattern involves creation the following methods: - -1. `pack` method - Packs the object data into one field. Serialize the non-Indexed fields into the byte array. -2. `unpack` method - Unpacks the object data into one field. De-serialize the non-Indexed fields from the byte array. -3. `writeExternal` method - Serialize the object data. Required for the `Externalizable` implementation. Serialize the indexed fields and the byte array. -4. `readExternal` method - De-serialize the object data. Required for the `Externalizable` implementation. De-serialize the indexed fields and the byte array. -5. `checkNulls` method - Handles null data for the indexed and byte array fields. -6. `getnulls` method - Handles null data for non indexed fields. - -# BinaryOutputStream and BinaryInputStream -The `BinaryOutputStream` contains various method to serialize all java's primitive type, their Object wrappers and arrays forms in a compacted mode. `BinaryInputStream` is its counterpart for de serialization. Your `pack` and `unpack` methods will be using an instance of those classes. - -# Example -With the [attached example](/attachment_files/sbp/BinaryCompaction.zip) we have a space class with 37 fields. - -- 1 Integer data type field (indexed used for queries). -- 12 String fields -- 12 Long fields -- 12 Integer Fields. - -{{% info %}} -With this example - The footprint overhead of the default serialization compared to a compact format is **250%**. -{{% /info %}} - -To run this example copy the example package zip into \GigaSpaces Root\examples\, extract the zip file and follow the instructions at the readme file. - -## The Original Space class -Our example involves a space class that will be modified to follow the compact serialization pattern. - -The original class includes: - -- One Integer indexed field. -- 12 String type non indexed fields declared as space class fields -- 12 Long type non indexed fields declared as space class fields -- 12 Integer type non indexed fields declared as space class fields -- Getter and Setter methods for the above fields - -The original class looks like this: - - -```java -@SpaceClass -public class SimpleEntry { - - public SimpleEntry() { - } - private Integer queryField; - private Long _longFieldA1; - - .... - - @SpaceRouting - @SpaceIndex(type=SpaceIndexType.BASIC) - public Integer getQueryField() { - return queryField; - } - - // getter and setter methods - public void setQueryField(Integer field) { - queryField = field; - } - - public Long get_longFieldA1() { - return _longFieldA1; - } - - public void set_longFieldA1(Long fieldA1) { - _longFieldA1 = fieldA1; - } -``` - -## The BinaryFormatEntry class -The modified class that implements the compact serialization pattern includes: - -- Using the `@SpaceClass(includeProperties=IncludeProperties.EXPLICIT)` decoration - this allows you to control which fields will be Space class fields explicitly. -- One Integer indexed field. -- One byte array field declared as a space class field. -- 12 String type non indexed fields. These are not space class fields. -- 12 Long type non indexed fields. These are not space class fields. -- 12 Integer type non indexed fields. These are not space class fields. -- Getter and setter methods for the above fields. -- `pack` and `unpack` method and few helper methods. -- `Externalizable` implementation - `writeExternal` and `readExternal` methods - -The modified class looks like this: - - -```java -@SpaceClass(includeProperties=IncludeProperties.EXPLICIT) -public class BinaryFormatEntry implements Externalizable { - - public BinaryFormatEntry(){} - - private Integer queryField; - private byte[] _binary; - - private Long _longFieldA1; - .... - - @SpaceRouting - @SpaceIndex(type=SpaceIndexType.BASIC) - public Long getQueryField() - { - return queryField; - } - - public void setQueryField(Long _queryField) - { - queryField = _queryField; - } - - @SpaceProperty - public byte[] getBinary() { - return _binary; - } - - public void setBinary(byte[] _binary) { - this._binary = _binary; - } - - public Long get_longFieldA1() { - return _longFieldA1; - } - - public void set_longFieldA1(Long fieldA1) { - _longFieldA1 = fieldA1; - } - ... - - public void pack(){...} - public void unpack(){...} - public void writeExternal(ObjectOutput out){...} - public void readExternal(ObjectInput in) {...} - private long getnulls(){...} - private short checkNulls() {...} -} -``` - -## The pack method -The `pack` method serializes the object non indexed data. It is called explicitly before calling the space write operation. This method serialize the object data by placing the data into the byte array field. Null values fields indication stored within one field. The `BinaryOutputStream` utility class is used to write the compacted data into the byte array. - - -```java -public void pack() throws Exception -{ - BinaryOutputStream output = new BinaryOutputStream(); - long nulls = getNulls(); - output.writeLong(nulls); - if (_longFieldA1 != null) - output.writeLong(_longFieldA1); - - // ... etc. for all other compactable fields. - - _binary = output.toByteArray(); - output.close(); -} -``` - -## The unpack method -This method de-serialize the object data by extracting the data from the byte array field and populating the fields with their corresponding values. `null` values fields are non-populated. This method is called after calling the space read operation. The `BinaryOutputStream` utility class is used to read the compacted data and place it into the relevant field. - - -```java -public void unpack() throws Exception -{ - BinaryInputStream input = new BinaryInputStream(_binary); - long nulls = input.readLong(); - int i = 0; - - if ((nulls & 1L << i) == 0) - _longFieldA1 = input.readLong(); - i++; - - // ... etc. for all other compactable fields. - - input.close(); - _binary = null; -} -``` - -## The writeExternal method -The `writeExternal` method serializes the object data into the output stream. The object data involves a field indicates which fields have `null` value, the indexed fields and a byte array field that includes all non indexed fields data (created by the `pack` method). The `writeExternal` assumes the `pack` method has been called explicitly prior the space write method call that initiated the `writeExternal` call. - - -```java -public void writeExternal(ObjectOutput out) throws IOException { - short nulls = 0; - int i=0; - - nulls = checkNulls(); - - out.writeShort(nulls); - if (queryField != null) { - out.writeLong(queryField); - } - if (_binary != null) { - out.write(_binary); - } -} -``` - -## The readExternal method -The `readExternal` method essentially performs the opposite of the what the `writeExternal` method is doing. This methods populates the indexed fields data and the byte array field data. Later, the remaining fields will be populated once the `unpack` method will be called. - - -```java -public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { - short nulls; - int i=0; - nulls = in.readShort(); - - if( (nulls & 1L << i) == 0 ) - queryField = in.readLong(); - i++; - if( (nulls & 1L << i) == 0 ) - { - byte[] data = new byte[500]; - int len = in.read(data); - _binary = new byte[len]; - System.arraycopy(data, 0, _binary, 0, len); - } -} -``` - -## The checkNulls method -This method goes through the indexed fields and the byte array field and place into a `short` data type field an indication for the ones with null value using a bit map. - - -```java -private short checkNulls() { - short nulls = 0; - int i = 0; - - nulls = (short) ((queryField == null) ? nulls | 1 << i : nulls); - i++; - nulls = (short) ((_binary == null) ? nulls | 1 << i : nulls); - i++; - return nulls; -} -``` - -## The getnulls method -This method goes through all class non indexed fields (the ones that their data is stored within the byte array) and place into a `long` data type field indication for the ones with null value using a bit map. - - -```java -private long getnulls() -{ - long nulls = 0; - int i=0; - - nulls = ((_longFieldA1 == null) ? nulls | 1L << i : nulls ) ; - i++; - nulls = ((_longFieldB1 == null) ? nulls | 1L << i : nulls ) ; - i++; - ... - return nulls; -} -``` - -## The Factory method -The example using a factory method called `generateBinaryFormatEntry` to create the space object. Once it has been populated , its `pack` method is called. - - -```java -private BinaryFormatEntry generateBinaryFormatEntry(int id){ - BinaryFormatEntry bfe = new BinaryFormatEntry(id, value1 , value2 ?) - bfe.pack(); // the pack method is called implicitly as part of the factory method - return bfe; -} -``` - -## Writing and Reading the Object from the space -The following code snipped illustrates how the compact serialized object is written into the space and read from the space: - - -```java -GigaSpace _gigaspace; -BinaryFormatEntry testBFE = generateBinaryFormatEntry(500); -_gigaspace.write(testBFE, Lease.FOREVER); -BinaryFormatEntry templateBFE = new BinaryFormatEntry(); -templateBFE.setQueryField (new Long(500)); -BinaryFormatEntry resBFE = (BinaryFormatEntry)_gigaspace.read(templateBFE, 0); -resBFE.unpack(); // this deserialize the binary data into the object fields -``` - -# References -The [PackRat](http://www.openspaces.org/display/PRT/PackRat) project allows you to use the compact serialization pattern via simple annotations. diff --git a/site/content/xap/12.3/dev-java/custom-matching.markdown b/site/content/xap/12.3/dev-java/custom-matching.markdown index a98256516..0dc239ccf 100644 --- a/site/content/xap/12.3/dev-java/custom-matching.markdown +++ b/site/content/xap/12.3/dev-java/custom-matching.markdown @@ -10,24 +10,26 @@ weight: 330 Queries are usually indexed and executed using primitive fields (long, float, string, etc.). These fields can be at the root level of the Space object, or embedded within [nested objects](./query-sql.html#nested-object-query) in the Space object. You can construct a query using a template object or SQL, to specify the criteria you want to use when the matching phase is performed within the Space when looking for the relevant objects. -However, in some cases you may need a custom data type with custom business logic to find matching objects within the Space, instead of the usual [primitive data type](http://java.sun.com/docs/books/tutorial/java/nutsandbolts/datatypes.html) comparison. To allow the Space to invoke your business logic when the matching process is conducted, the [Comparable](http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Comparable.html) interface should be implemented for a class that stores the data you want to use with your custom business logic. +However, in some cases you may need a custom data type with custom business logic to find matching objects within the Space, instead of the [primitive data type](http://java.sun.com/docs/books/tutorial/java/nutsandbolts/datatypes.html) comparison. To allow the Space to invoke your business logic when the matching process is conducted, the [Comparable](http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Comparable.html) interface should be implemented for a class that stores the data you want to use with your custom business logic. {{% tip %}} The [Comparable](http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Comparable.html) implementation should not be done for the Space class itself, but for one of its fields. {{% /tip %}} -{{% info %}} To index the data to speed up the custom matching process, index the field so that its class implements the `Comparable` interface using the `EXTENDED` index type as part of the Space class. See the [Indexing](./indexing.html) section for additional information about how to enable the `EXTENDED` index. Indexing the custom type field should be used with caution, because it doesn't support a `Comparable.compareTo` implementation that performs relative-based matching, which is demonstrated in the example on this page. -{{%/info%}} +You can index the data to speed up the custom matching process; index the field so that its class implements the `Comparable` interface using the `EXTENDED` index type as part of the Space class. Indexing the custom type field should be used with caution, because it doesn't support a `Comparable.compareTo` implementation that performs relative-based matching, which is demonstrated in the example on this page. + +{{%note%}} +See the [Indexing](./indexing.html) section for additional information about how to enable the `EXTENDED` index. +{{%/note%}} # Vector Compare Example -The following example illustrates a business logic implementation used to query for vector data (an array of Integer values), using the [Euclidean distance](http://en.wikipedia.org/wiki/Euclidean_distance) formula: +The following example is a business logic implementation used to query for vector data (an array of Integer values), using the [Euclidean distance](http://en.wikipedia.org/wiki/Euclidean_distance) formula: {{% panel %}}![EuclideanDistance.jpg](/attachment_files/sbp/EuclideanDistance.jpg){{% /panel %}} The object that holds the array implements the [Comparable](http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Comparable.html) interface. The Space class has a getter method for this field indexing, using the `EXTENDED` index. The actual query involves indexed fields for several sample data points within the vector, together with the custom field: - ```java SQLQuery query= new SQLQuery(Vector.class , "(x0 > ? AND x0 < ? ) " + @@ -52,11 +54,11 @@ The following is an example of a target vector, and a matching vector found usin To scale the system, deploy the Space in a [partitioned cluster schema](/product_overview/terminology.html). This allows queries such as matching to be executed across all the partitions in parallel, speeding up the query execution time. {{% /tip %}} -See **The Application** tab for the full query usage. This allows the `Comparable.compareTo` implementation to be performed on a smaller candidate subset of objects. +The **Application** tab shows the full query usage. This allows the `Comparable.compareTo` implementation to be performed on a smaller candidate subset of objects. {{%tabs%}} -{{%tab " The Comparable implementation "%}} +{{%tab " Comparable Implementation "%}} ```java @@ -129,7 +131,7 @@ public class VectorData implements Serializable, Comparable { {{% /tab %}} -{{%tab " The Space Class "%}} +{{%tab " Space Class "%}} ```java @@ -190,7 +192,7 @@ public class Vector { {{% /tab %}} -{{%tab " The Application "%}} +{{%tab " Application "%}} ```java @@ -256,5 +258,4 @@ static int[] getRandomVector() ``` {{% /tab %}} - {{% /tabs %}} diff --git a/site/content/xap/12.3/dev-java/lowering-the-space-object-footprint.markdown b/site/content/xap/12.3/dev-java/lowering-the-space-object-footprint.markdown new file mode 100644 index 000000000..3ffa9041c --- /dev/null +++ b/site/content/xap/12.3/dev-java/lowering-the-space-object-footprint.markdown @@ -0,0 +1,344 @@ +--- +type: post123 +title: Reducing the Space Object Footprint +categories: XAP123, OSS +parent: modeling-your-data.html +weight: 570 + +--- + +# Overview + +By default, when using the GigaSpaces Java API, Space object fields are stored in the Space as is. No data compaction or compression is done while the object is transported across the network, or when stored within the Space. + +With the compact serialization pattern, you can reduce the footprint of the data object in the Space memory when it is stored in the data grid, which allows storing more Space objects per memory unit. This pattern works very well when the Space object includes a large number of numerical values, because these objects are stored as a more optimal data type. + +The compact serialization pattern enables complete control over the format of the data object while transported over the network and when stored within the Space. This technique: + +- Compresses the object payload data when transported over the network and when stored in memory. +- Avoids the need for de-serialization when a Space object is written to the Space from a remote client (for non-primitive fields such as user-defined classes or collection fields). +- Avoids the need for de-serialization when data is replicated to the backup Space. +- Avoids the need for serialization when reading an object back from the Space to the client process (on the Space side). + +{{% note %}} + +- The Compressed storage type compresses non-primitive fields using the zip utilities, which is a different approach than the compact serialization pattern. +- The Binary storage type stores non-primitive fields within the Space in their byte array form. It does not compress or reduce the footprint of the data like the compact serialization pattern does. This approach makes it unnecessary to introduce the nested Space object data type to the Space JVM, or to de-serialize it on the Space side. The Binary storage type may improve performance when the Space object stores large collections. +- C++ and .Net API data objects may undergo some compaction when sent across the network. +{{% /note %}} + +# Data Compaction Flow + +With the compact serialization pattern, the following happens to the data object: + +- Before writing the object, all non-indexed fields (payload data) are serialized (packed) into one byte array field using the GigaSpaces serialization API. +- All indexed fields are serialized/de-serialized as usual (the `writeExternal`, `readExternal` implementation is available to write and read these fields into the stream). +- After reading the object from the Space, the byte array data is de-serialized (unpacked). + +{{% align center %}} +![bin_ser.jpg](/attachment_files/sbp/bin_ser.jpg) +{{% /align %}} + +When an object is written to the Space: + +- The non-indexed fields are compacted and serialized into the same field (as a byte array). +- All the indexed fields plus the byte array are serialized as usual via the `writeExternal` call. +- The object arrives de-serialized in the Space, indexed fields stored as usual, and the byte array field is stored as is. + +When an object is read from the Space: + +- The read template undergoes the same actions as when writing an object to the Space +- The matching object is serialized and sent to the client. +- When the matching object arrives at the client side is it de-serialized, and the byte array data is de-serialized and expanded (in a lazy manner). + +# Implementation + +Using the compact serialization pattern can drastically reduce the object footprint when it is stored in the Space. While more of the Space object fields are serialized using the GigaSpaces Serialization API, the memory footprint overhead is smaller compared to the default serialization mode. + +The compact serialization pattern involves creating the following methods: + +1. `pack` method - Packs the object data into a single field, and serializes the non-indexed fields into the byte array. +2. `unpack` method - Unpacks the object data into a single field, and deserializes the non-indexed fields from the byte array. +3. `writeExternal` method - Serializes the object data., the index fields and the byte array. Required for the `Externalizable` interface. +4. `readExternal` method - Deserializes the object data, the indexed fields, and the byte array. Required for the `Externalizable` interface. +5. `checkNulls` method - Handles null data for the indexed and byte array fields. +6. `getnulls` method - Handles null data for non-indexed fields. + +## BinaryOutputStream and BinaryInputStream + +The `BinaryOutputStream` contains various methods to serialize all of the Java primitive types, along with their object wrappers and arrays forms in compacted mode. `BinaryInputStream` is its counterpart for deserialization. Your `pack` and `unpack` methods will use an instance of those classes. + +## Example + +The [provided code example](/attachment_files/sbp/BinaryCompaction.zip) includes a Space class with 37 fields: + +- One Integer data type field (indexed for queries). +- 12 String fields +- 12 Long fields +- 12 Integer fields + +In this example, the footprint overhead of the default serialization is **250%** larger than the footprint of the compact format. To run the example, copy the zip file to `\GigaSpaces Root\examples\`, extract the contents, and follow the instructions in the readme file. + +### Original Space Class + +Our example involves a Space class that is modified to follow the compact serialization pattern. + +The original class includes: + +- One Integer indexed field. +- 12 String type non-indexed fields declared as Space class fields. +- 12 Long type non-indexed fields declared as Space class fields. +- 12 Integer type non-indexed fields declared as Space class fields. +- Getter and Setter methods for the above fields. + +The original class looks like this: + +```java +@SpaceClass +public class SimpleEntry { + + public SimpleEntry() { + } + private Integer queryField; + private Long _longFieldA1; + + .... + + @SpaceRouting + @SpaceIndex(type=SpaceIndexType.BASIC) + public Integer getQueryField() { + return queryField; + } + + // getter and setter methods + public void setQueryField(Integer field) { + queryField = field; + } + + public Long get_longFieldA1() { + return _longFieldA1; + } + + public void set_longFieldA1(Long fieldA1) { + _longFieldA1 = fieldA1; + } +``` + +### BinaryFormatEntry Class + +The modified class that implements the compact serialization pattern uses the `@SpaceClass(includeProperties=IncludeProperties.EXPLICIT)` decoration, which allows you to control which fields will be Space class fields explicitly. This class also includes: + +- One Integer indexed field. +- One byte array field declared as a Space class field. +- 12 String type non-indexed fields that are not Space class fields. +- 12 Long type non-indexed fields that are not Space class fields. +- 12 Integer type non-indexed fields that are not Space class fields. +- Getter and setter methods for the above fields. +- `pack` and `unpack` methods, and several helper methods. +- `Externalizable` implementation with `writeExternal` and `readExternal` methods + +The modified class looks like this: + +```java +@SpaceClass(includeProperties=IncludeProperties.EXPLICIT) +public class BinaryFormatEntry implements Externalizable { + + public BinaryFormatEntry(){} + + private Integer queryField; + private byte[] _binary; + + private Long _longFieldA1; + .... + + @SpaceRouting + @SpaceIndex(type=SpaceIndexType.BASIC) + public Long getQueryField() + { + return queryField; + } + + public void setQueryField(Long _queryField) + { + queryField = _queryField; + } + + @SpaceProperty + public byte[] getBinary() { + return _binary; + } + + public void setBinary(byte[] _binary) { + this._binary = _binary; + } + + public Long get_longFieldA1() { + return _longFieldA1; + } + + public void set_longFieldA1(Long fieldA1) { + _longFieldA1 = fieldA1; + } + ... + + public void pack(){...} + public void unpack(){...} + public void writeExternal(ObjectOutput out){...} + public void readExternal(ObjectInput in) {...} + private long getnulls(){...} + private short checkNulls() {...} +} +``` + +### Pack Method + +The `pack` method serializes the object's non-indexed data. It is called explicitly before calling the Space write operation. The pack method serialize the object data by placing the data into the byte array field. Null value field indications are stored within a single field. The `BinaryOutputStream` utility class is used to write the compacted data to the byte array. + +```java +public void pack() throws Exception +{ + BinaryOutputStream output = new BinaryOutputStream(); + long nulls = getNulls(); + output.writeLong(nulls); + if (_longFieldA1 != null) + output.writeLong(_longFieldA1); + + // ... etc. for all other compactable fields. + + _binary = output.toByteArray(); + output.close(); +} +``` + +### Unpack Method + +The `unpack` method deserializes the object data by extracting the data from the byte array field and populating the fields with their corresponding values. `null` value fields are non-populated. This method is called after calling the Space read operation. The `BinaryOutputStream` utility class is used to read the compacted data and place it in the relevant field. + +```java +public void unpack() throws Exception +{ + BinaryInputStream input = new BinaryInputStream(_binary); + long nulls = input.readLong(); + int i = 0; + + if ((nulls & 1L << i) == 0) + _longFieldA1 = input.readLong(); + i++; + + // ... etc. for all other compactable fields. + + input.close(); + _binary = null; +} +``` + +### writeExternal Method + +The `writeExternal` method serializes the object data into the output stream. The object data includes a field that indicates which fields have `null` values, the indexed fields, and a byte array field that includes all non-indexed field data (created by the `pack` method). The `writeExternal` method assumes that the `pack` method has been called explicitly prior to the Space write method call that initiated the `writeExternal` call. + +```java +public void writeExternal(ObjectOutput out) throws IOException { + short nulls = 0; + int i=0; + + nulls = checkNulls(); + + out.writeShort(nulls); + if (queryField != null) { + out.writeLong(queryField); + } + if (_binary != null) { + out.write(_binary); + } +} +``` + +### readExternal Method + +The `readExternal` method performs the opposite action to the `writeExternal` method, populating the indexed field data and the byte array field data. Later, the remaining fields are populated when the `unpack` method is called. + + +```java +public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { + short nulls; + int i=0; + nulls = in.readShort(); + + if( (nulls & 1L << i) == 0 ) + queryField = in.readLong(); + i++; + if( (nulls & 1L << i) == 0 ) + { + byte[] data = new byte[500]; + int len = in.read(data); + _binary = new byte[len]; + System.arraycopy(data, 0, _binary, 0, len); + } +} +``` + +### checkNulls Method + +The `checkNulls` method reads the indexed fields and the byte array field, and places an indication for the ones with null values in a `short` data type field, using a bitmap. + +```java +private short checkNulls() { + short nulls = 0; + int i = 0; + + nulls = (short) ((queryField == null) ? nulls | 1 << i : nulls); + i++; + nulls = (short) ((_binary == null) ? nulls | 1 << i : nulls); + i++; + return nulls; +} +``` + +### getNulls Method + +The `getNulls` method reads all class non-indexed fields (those whose data is stored within the byte array) and place an indication for the ones with null valued in a `long` data type field, using a bitmap. + +```java +private long getnulls() +{ + long nulls = 0; + int i=0; + + nulls = ((_longFieldA1 == null) ? nulls | 1L << i : nulls ) ; + i++; + nulls = ((_longFieldB1 == null) ? nulls | 1L << i : nulls ) ; + i++; + ... + return nulls; +} +``` + +### Factory Method + +The example uses a factory method called `generateBinaryFormatEntry` to create the Space object. After the Space has been populated, the `pack` method is called. + +```java +private BinaryFormatEntry generateBinaryFormatEntry(int id){ + BinaryFormatEntry bfe = new BinaryFormatEntry(id, value1 , value2 ?) + bfe.pack(); // the pack method is called implicitly as part of the factory method + return bfe; +} +``` + +### Writing and Reading the Object from the Sspace + +The following code snippet shows how the compact serialized object is written to the Space, and read from the Space: + +```java +GigaSpace _gigaspace; +BinaryFormatEntry testBFE = generateBinaryFormatEntry(500); +_gigaspace.write(testBFE, Lease.FOREVER); +BinaryFormatEntry templateBFE = new BinaryFormatEntry(); +templateBFE.setQueryField (new Long(500)); +BinaryFormatEntry resBFE = (BinaryFormatEntry)_gigaspace.read(templateBFE, 0); +resBFE.unpack(); // this deserialize the binary data into the object fields +``` + +# References + +The [PackRat](http://www.openspaces.org/display/PRT/PackRat) project allows you to use the compact serialization pattern via simple annotations.