Skip to content

Commit

Permalink
ORC-1563: Fix orc.bloom.filter.fpp default value and orc.compress
Browse files Browse the repository at this point in the history
… notes of Spark and Hive config docs

### What changes were proposed in this pull request?
1. Add `orc.compress` enumeration value description
    - LZO, LZ4 (ORC-77) since ORC 1.2
    - ZSTD (ORC-363) since ORC 1.6
2. Fix `orc.bloom.filter.fpp` default value (ORC-1338) since ORC 1.8.2

### Why are the changes needed?
The document is out of date.

### How was this patch tested?
local check

Closes #1709 from cxzl25/ORC-1563.

Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 0184a66)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
cxzl25 authored and dongjoon-hyun committed Dec 27, 2023
1 parent 6baecb1 commit 34175f8
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions site/_docs/hive-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ with the same options.

Key | Default | Notes
:----------------------- | :---------- | :------------------------
orc.compress | ZLIB | high level compression = {NONE, ZLIB, SNAPPY}
orc.compress | ZLIB | high level compression = {NONE, ZLIB, SNAPPY, LZO, LZ4, ZSTD}
orc.compress.size | 262,144 | compression chunk size
orc.stripe.size | 67,108,864 | memory buffer in bytes for writing
orc.row.index.stride | 10,000 | number of rows between index entries
orc.create.index | true | whether the ORC writer create indexes as part of the file or not
orc.bloom.filter.columns | "" | comma separated list of column names
orc.bloom.filter.fpp | 0.05 | bloom filter false positive rate
orc.bloom.filter.fpp | 0.01 | bloom filter false positive rate

For example, to create an ORC table without high level compression:

Expand Down
4 changes: 2 additions & 2 deletions site/_docs/spark-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ with the same options.

Key | Default | Notes
:----------------------- | :---------- | :------------------------
orc.compress | ZLIB | high level compression = {NONE, ZLIB, SNAPPY, ZSTD}
orc.compress | ZLIB | high level compression = {NONE, ZLIB, SNAPPY, LZO, LZ4, ZSTD}
orc.compress.size | 262,144 | compression chunk size
orc.stripe.size | 67,108,864 | memory buffer in bytes for writing
orc.row.index.stride | 10,000 | number of rows between index entries
orc.create.index | true | whether the ORC writer create indexes as part of the file or not
orc.bloom.filter.columns | "" | comma separated list of column names
orc.bloom.filter.fpp | 0.05 | bloom filter false positive rate
orc.bloom.filter.fpp | 0.01 | bloom filter false positive rate
orc.key.provider | "hadoop" | key provider
orc.encrypt | "" | list of keys and columns to encrypt with
orc.mask | "" | masks to apply to the encrypted columns
Expand Down

0 comments on commit 34175f8

Please sign in to comment.