Skip to content

Commit

Permalink
Move ParquetMetadataWriter to its own module, update documentation (#…
Browse files Browse the repository at this point in the history
…6202)

* Move `ThriftMetadataWriter` and `ParquetMetadataWriter` to a new module

* Improve documentation, make pub(crate)

* Apply suggestions from code review

Co-authored-by: Ed Seidl <etseidl@users.noreply.github.com>

* Add comment side effect of writing column and offset indexes

* Document how to write bloom filters

* Update parquet/src/file/metadata/writer.rs

Co-authored-by: Ed Seidl <etseidl@users.noreply.github.com>

---------

Co-authored-by: Ed Seidl <etseidl@users.noreply.github.com>
  • Loading branch information
alamb and etseidl authored Aug 13, 2024
1 parent 1238bb1 commit 468a564
Show file tree
Hide file tree
Showing 3 changed files with 629 additions and 513 deletions.
6 changes: 4 additions & 2 deletions parquet/src/file/metadata/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -50,10 +50,9 @@
//! [`decode_metadata`]: crate::file::footer::decode_metadata
//!
//! Writing:
//! * Write `ParquetMetaData` to bytes in memory: Not yet supported (see [#6002])
//! * Write `ParquetMetaData` to bytes in memory: [`ParquetMetaDataWriter`]
//! * Writes `ParquetMetaData` to an async target: Not yet supported
//!
//! [#6002]: https://github.com/apache/arrow-rs/issues/6002
//!
//! # Metadata Encodings and Structures
//!
Expand Down Expand Up @@ -95,6 +94,7 @@
//! * Same name, different struct
//! ```
mod memory;
mod writer;

use std::ops::Range;
use std::sync::Arc;
Expand All @@ -115,6 +115,8 @@ use crate::schema::types::{
ColumnDescPtr, ColumnDescriptor, ColumnPath, SchemaDescPtr, SchemaDescriptor,
Type as SchemaType,
};
pub use writer::ParquetMetaDataWriter;
pub(crate) use writer::ThriftMetadataWriter;

/// Page level statistics for each column chunk of each row group.
///
Expand Down
Loading

0 comments on commit 468a564

Please sign in to comment.