Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: add docs, part of #37 #6453

Merged
merged 2 commits into from
Oct 1, 2024
Merged

chore: add docs, part of #37 #6453

merged 2 commits into from
Oct 1, 2024

Conversation

ByteBaker
Copy link
Contributor

@ByteBaker ByteBaker commented Sep 25, 2024 •
edited
Loading

  • add pragma #![warn(missing_docs)] to the following

    • arrow-flight
    • arrow-ipc
    • arrow-integration-test
    • arrow-integration-testing
    • object_store

  • add documentation to remove any warnings thus generated

  • also document the caveat with using level 10 GZIP compression in Parquet. Closes What is the highest compression level in gzip? #6282.

@github-actions github-actions bot added arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate labels Sep 25, 2024
@ByteBaker
Copy link
Contributor Author

@alamb please have a look.

The only culprits that remain now are parquet and arrow-ipc, with over 200 & 850 warnings each, so they'll take some time. But we're in the right direction.

@ByteBaker
Copy link
Contributor Author

Okay turns out these warnings were not emitted when tested using clippy. I'll fix these soon and circle back on this.

@alamb alamb assigned alamb and unassigned alamb Sep 25, 2024
@alamb alamb marked this pull request as draft September 25, 2024 16:16
@ByteBaker ByteBaker force-pushed the require_docs branch 3 times, most recently from 743da3d to 3b5b3b0 Compare September 27, 2024 11:43
@ByteBaker
Copy link
Contributor Author

@alamb I'm not sure what to do about vendored code. Some guidance/help needed here.

@github-actions github-actions bot added the object-store Object Store Interface label Sep 27, 2024
@alamb
Copy link
Contributor

alamb commented Sep 28, 2024 •
edited
Loading

@alamb I'm not sure what to do about vendored code. Some guidance/help needed here.

Since this code is auto generated, I think we should disable any clippy lints that are failing for it rather than try to add documentation

@ByteBaker
chore: add docs, part of apache#37
618c775 
- add pragma `#![warn(missing_docs)]` to the following
  - `arrow-flight`
  - `arrow-ipc`
  - `arrow-integration-test`
  - `arrow-integration-testing`
  - `object_store`

- also document the caveat with using level 10 GZIP compression in
  parquet. See apache#6282.
@github-actions github-actions bot added the parquet Changes to the parquet crate label Sep 29, 2024
@ByteBaker
Copy link
Contributor Author

Weird that I missed this. I've been doing this for every autogenerated file so far. Somehow I thought the error was coming from running git diff post regen.sh, and I even added documentation. All that for nothing. 😢

@ByteBaker ByteBaker marked this pull request as ready for review September 29, 2024 07:22
@ByteBaker
Copy link
Contributor Author

@alamb the PR is updated and ready for review.

alamb
alamb reviewed Sep 30, 2024
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much @ByteBaker -- this PR is almost perfect. I think the only thing that needs to be fixed is the potential panic in log_metadata.

arrow-flight/src/bin/flight_sql_client.rs Outdated Show resolved Hide resolved
arrow-flight/src/lib.rs Show resolved Hide resolved
arrow-integration-testing/src/flight_server_scenarios/integration_test.rs Show resolved Hide resolved
parquet/src/compression.rs Show resolved Hide resolved
parquet/src/compression.rs Show resolved Hide resolved
@ByteBaker
Copy link
Contributor Author

The changes have been incorporated. Please review.

@ByteBaker
Copy link
Contributor Author

The warning is due to a spurious network issue. The same test has passed in previous run. I think we can merge this, or maybe run the failed step again just be extra sure.

alamb
alamb approved these changes Oct 1, 2024
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @ByteBaker ❤️

@alamb
Copy link
Contributor

alamb commented Oct 1, 2024

Epic -- thanks again @ByteBaker

@alamb alamb merged commit 7781bc2 into apache:master Oct 1, 2024
33 checks passed
@ByteBaker
Copy link
Contributor Author

ByteBaker commented Oct 1, 2024 •
edited
Loading

@alamb I'm just happy to make Arrow better! 😅

One more PR and it'll be complete, end to end.

@alamb
Copy link
Contributor

alamb commented Oct 1, 2024

We appreciate it very much

@Turbo87 Turbo87 mentioned this pull request Oct 3, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alamb alamb alamb approved these changes

Assignees
No one assigned
Labels
arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate object-store Object Store Interface parquet Changes to the parquet crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

What is the highest compression level in gzip?
2 participants
@ByteBaker @alamb