Apache Release 3.3.0

This release removes support for allocating off-heap memory in Java9 and Java10

Only LTS Java versions are now fully supported and since the Java 9 and Java 10 no
longer receive security patches, this release removes off-heap memory support
for these versions.

This applies to Datasketches-Memory when allocating Memory-Mapped files,
Direct or direct ByteBuffers in these runtime versions.
Support for allocating Memory in both heap and non-heap remains unaffected
for Java 8, Java 11, Java 12 and Java 13.

Apache Release 3.2.0

• This release extends the quantiles KLL sketch to include both doubles and float types as well as full updatable operation off-heap in contiguous memory.

• This release also adds two new methods to Theta Union based on the discussion on Druid issue # 12261:

  • getCurrentBytes() This behaves similarly to the Theta Sketch method by the same name.
  • getMaxUnionBytes() This behaves similarly to the static Sketches method by the same name, but is non-static and returns the maximum stored bytes for this union given its nominal entries configuration.

Apache Release 3.1.0

This is primarily a maintenance release.
Highlights:

• Fixed "Mikhail's Issue #368". This was a specific Theta set-operation corner case bug.
  • In addition, Developed a standard model for all Theta, Tuple set operation corner cases. Implemented across all corner cases for Java (and C++).
• Improvements in clarity of Javadocs, primarily in Theta, Tuple sketches.
• Fixed some parameter leakage cases related to LGTM warnings.
• Improved variable naming to enhance code clarity.
• Fixed warnings discovered by latest SpotBugs.
• Significant improvements and cleanup of ArrayOfDoubles Tuple Sketch code.

Apache Release 3.0.0

This release of DataSketches-Java 3.0.0 depends on DataSketches-Memory 2.0.0 and is compatible with JDK8 through JDK13. If you wish to compile this release from source please refer to the README of this release.

There are some minor API changes as part of this release

• The MurmurHash3v2 class has been relocated to be part of the DataSketches-Memory component.
• Some of the API changes in the new Memory component are reflected in this component where they are exposed as part of the public API here. For example, the API of the MemoryRequestServer of Memory has changed slightly.
• Because this is a major release a number of deprecated methods have been removed.

Apache Release 2.0.0

Introducing the Relative Error Quantiles Sketch

See website documentation.

Improved input null and empty handling policies for Theta / Tuple sketches

Exceptions are now being thrown where there could be accidental destruction of data. Where a null is harmless, it is ignored. However, receiving an empty sketch is considered valid, and the result follows the set operation logic.

Operation	Null Policy	Empty Policy
Union	Ignored	Ignored
Intersection	Exception	Empty result
AnotB, first Argument, A	Exception	Empty result
AnotB, subsequent Argument, B	Ignored	Ignored

Improved Set Operation handling

In Theta Sketch and Tuple Sketch (Generic), the APIs for Union, AnotB and Intersection now allow for both stateless and stateful operation.

• see ExamplesTest.java for full example code

• Pseudocode model for stateless operation.
  Two set operation arguments and the result are accomplished in one call:
  result = setOp(Sketch A, Sketch B);
  where setOp = union, intersect, or aNotB

• Pseudocode model for stateful operation. Any number of arguments can be presented sequentially:
  void setOp(Sketch 1);
void setOp(Sketch 2);
...
void setOp(Sketch N);
result = getResult();
  Where setOp = union or intersect
  Because the AnotB operation is asymmetric, the method names are a little different:
  void setA(Sketch 1); //for the first argument
  void notB(Sketch 2); //for all subsequent arguments
  ...
void notB(Sketch N);
result = getResult();

Tuple Sketch (Generic) set operations now allow combining with Theta Sketch

• Pseudocode model for stateless operation:
  result = setOp(TupleSketch A, ThetaSketch B);

• Pseudocode model for stateful operation:
  void setOp(TupleSketch 1);
void setOp(ThetaSketch 1);
void setOp(TupleSketch 2);
...
void setOp(ThetaSketch N);
result = getResult();

Package changes

• The tuple package was too large and confusing. The ArrayOfDoubles classes, generic base classes and example generic classes were all mingled together. These have now been split up into the following hierarchy:
  • tuple/
    • adouble/
    • aninteger/
    • arrayofdoubles
    • strings

Lots of internal refactoring, code cleanup, and documentation improvements

Release Process

• Replaced Travis-CI with GitHub Actions
• Added Maven toolchains.xml in anticipation of multiple JDK releases
• Cleaned up Release Zip files to exclude unneeded hidden files.

Apache Release 1.3.0-incubating

• This release contains a critical fix for a recently discovered bug identified by Druid Issue #9736.

Apache Release 1.2.0-incubating

• Significant Merge speed improvements to HLL
• Fixed issue #292

This Release was directly derived from 1.2.0-incubating-RC2. Please see notes there.

Shortly before this was released a bug (issue #296) was discovered. PR #297 fixes this issue but it wasn't in time for this release. This is now fixed in master and will be part of the next release.

This bug only affects unit test code on Windows platforms. It has no effect on run-time production code.

1.2.0-incubating-RC2

This RC2 fixes a recently discovered bug identified by Issue #292 . It was fixed with PR #293, which contained other documentation and code cleanup unrelated to the bug. The specific files changed relating to the bug:

• main/.../datasketches/tuple/Union.java
• test/.../datasketches/tuple/adouble/UpdatableSketchWithDoubleSummaryTest.java

1.2.0-incubating-RC1

Significant Merge speed improvements to HLL

Apache Release 1.1.0-incubating

Changes from previous release:

• This release fixes a bug in Theta Sketches caused by the merge of a SingleItemSketch which created an additional hash to be entered into the merge. This caused over-counting in the merge result.

• Minor updates to LICENSE, NOTICE, and README files.