Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rustdoc: Scrape examples #9154

Merged
merged 3 commits into from
Aug 10, 2023
Merged

Conversation

killercup
Copy link
Contributor

Objective

Provide more usage examples in API docs so that people can see methods being used in context.

Solution

Enable experimental rustdoc feature "scrape examples". See https://doc.rust-lang.org/nightly/rustdoc/scraped-examples.html for official docs.

Example screenshots of examples :)

image image image

Limitations

  • Only methods seem to show examples so far
  • It may be confusing to have curated examples from doc comments followed by snippets from examples/

@killercup
Copy link
Contributor Author

If this is approved, consider including it in 0.11.1!

@killercup
Copy link
Contributor Author

re ci: check-bans fail seems unrelated

Copy link
Member

@cart cart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I played around with this locally and its a huge win! It does create the new risk of confusing / niche example contexts showing up as the first entry, as we aren't defining precedence order. We're no longer fully in control of the doc content for a given item. But (1) I have yet to find an instance that made things worse and (2) we can always remove that example from consideration if we encounter a nasty case.

The amount of added content here is massive. Love this!

@cart
Copy link
Member

cart commented Jul 14, 2023

I also agree that its worth including in 0.11.1

@cart cart added this to the 0.11.1 milestone Jul 14, 2023
@cart cart added the C-Docs An addition or correction to our documentation label Jul 14, 2023
Copy link
Member

@james7132 james7132 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact this also improves example discoverability is great. So long as this doesn't pull directly from other first-party crates in the workspace, and only from the examples, this looks to be a massive win for our docs.

@@ -265,6 +265,7 @@ crossbeam-channel = "0.5.0"
[[example]]
name = "hello_world"
path = "examples/hello_world.rs"
doc-scrape-examples = true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed? it's not mentioned in the page you linked

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, so it builds locally without this, but IIRC docs.rs needs it to be aware of which examples to read from

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this because cargo said it will need this when the examples use dev dependencies:

$ cargo +nightly doc -Zunstable-options -Zrustdoc-scrape-examples
warning: Rustdoc did not scrape the following examples because they require dev-dependencies: hello_world, bloom_2d, move_sprite, rotation, mesh2d, mesh2d_manual, mesh2d_vertex_color_texture, 2d_shapes, custom_gltf_vertex_attribute, 2d_gizmos, sprite, sprite_flipping, sprite_sheet, text2d, texture_atlas, transparency_2d, pixel_perfect, 3d_scene, 3d_shapes, generate_custom_mesh, anti_aliasing, 3d_gizmos, atmospheric_fog, fog, blend_modes, lighting, lines, ssao, spotlight, bloom_3d, load_gltf, tonemapping, orthographic, parenting, pbr, parallax_mapping, render_to_texture, shadow_biases, shadow_caster_receiver, skybox, spherical_area_lights, split_screen, texture, transparency_3d, two_passes, update_gltf_scene, vertex_colors, wireframe, no_prepass, animated_fox, morph_targets, animated_transform, cubic_curve, custom_skinned_mesh, gltf_skinned_mesh, custom_loop, drag_and_drop, empty, empty_defaults, headless, logs, plugin, plugin_group, return_after_run, thread_pool_resources, no_renderer, without_winit, asset_loading, custom_asset, custom_asset_io, hot_asset_reloading, async_compute, external_source_external_thread, audio, audio_control, decodable, spatial_audio_2d, spatial_audio_3d, pitch, log_diagnostics, custom_diagnostic, ecs_guide, apply_deferred, component_change_detection, custom_query_param, event, fixed_timestep, generic_system, hierarchy, iter_combinations, parallel_query, removal_detection, run_conditions, startup_system, state, system_piping, system_closure, system_param, timers, alien_cake_addict, breakout, contributors, game_menu, char_input_events, gamepad_input, gamepad_input_events, gamepad_rumble, keyboard_input, keyboard_modifiers, keyboard_input_events, mouse_input, mouse_input_events, mouse_grab, touch_input, touch_input_events, text_input, reflection, generic_reflection, reflection_types, trait_reflection, scene, custom_vertex_attribute, post_processing, shader_defs, shader_material, shader_prepass, shader_material_screenspace_texture, shader_material_glsl, shader_instancing, animate_shader, compute_shader_game_of_life, array_texture, texture_binding_array, bevymark, many_animated_sprites, many_buttons, many_cubes, many_gizmos, many_foxes, many_glyphs, many_lights, many_sprites, transform_hierarchy, text_pipeline, scene_viewer, gamepad_viewer, nondeterministic_system_order, 3d_rotation, scale, transform, translation, borders, button, display_and_visibility, window_fallthrough, font_atlas_debug, overflow, overflow_debug, relative_cursor_position, size_constraints, text, text_debug, flex_layout, text_wrap_debug, grid, transparency_ui, z_index, ui, ui_scaling, ui_texture_atlas, viewport_debug, clear_color, low_power, multiple_windows, scale_factor_override, screenshot, transparent_window, window_settings, resizing, minimising, window_resizing, fallback_image
    If you want Rustdoc to scrape these examples, then add `doc-scrape-examples = true`
    to the [[example]] target configuration of at least one example.

Cargo.toml Outdated
@@ -2108,3 +2289,7 @@ codegen-units = 1
inherits = "release"
lto = "fat"
panic = "abort"

[package.metadata.docs.rs]
all-features = true
Copy link
Member

@mockersf mockersf Jul 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one seems unrelated. I don't remember, does docs.rs shows the warning about feature automatically or does that needs to be added manually?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, not related. I saw this in another project and added it as a precaution in a separate commit. Should I skip it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed it just to avoid unexpected outcomes. We can add it in another PR if we consider the change and decide we like it.

@alice-i-cecile
Copy link
Member

I'm on board in principle with enabling this! Haven't reviewed the diff yet though.

@nicopap
Copy link
Contributor

nicopap commented Jul 18, 2023

I've played with this in bevy-ui-navigation. In my experience it's tricky to get the config right so that docs.rs properly scraps examples.

@nicopap
Copy link
Contributor

nicopap commented Jul 18, 2023

@mockersf Would this work with https://dev-docs.bevyengine.org/bevy/index.html?

@mockersf
Copy link
Member

not directly, the extra flags from https://doc.rust-lang.org/nightly/rustdoc/scraped-examples.html#how-to-use-this-feature need to be passed to

run: cargo doc --all-features --no-deps -p bevy

As those are unstable flags, if they need a nightly rust doc, we would also need to switch this job to nightly

Copy link
Contributor Author

@killercup killercup left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rebasing on main to get CI updates. Should be good to go, but I can remove the second commit if you want :)

Update: CI still fails. Is it because the changes to Cargo.toml invalidate the cache?

@@ -265,6 +265,7 @@ crossbeam-channel = "0.5.0"
[[example]]
name = "hello_world"
path = "examples/hello_world.rs"
doc-scrape-examples = true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this because cargo said it will need this when the examples use dev dependencies:

$ cargo +nightly doc -Zunstable-options -Zrustdoc-scrape-examples
warning: Rustdoc did not scrape the following examples because they require dev-dependencies: hello_world, bloom_2d, move_sprite, rotation, mesh2d, mesh2d_manual, mesh2d_vertex_color_texture, 2d_shapes, custom_gltf_vertex_attribute, 2d_gizmos, sprite, sprite_flipping, sprite_sheet, text2d, texture_atlas, transparency_2d, pixel_perfect, 3d_scene, 3d_shapes, generate_custom_mesh, anti_aliasing, 3d_gizmos, atmospheric_fog, fog, blend_modes, lighting, lines, ssao, spotlight, bloom_3d, load_gltf, tonemapping, orthographic, parenting, pbr, parallax_mapping, render_to_texture, shadow_biases, shadow_caster_receiver, skybox, spherical_area_lights, split_screen, texture, transparency_3d, two_passes, update_gltf_scene, vertex_colors, wireframe, no_prepass, animated_fox, morph_targets, animated_transform, cubic_curve, custom_skinned_mesh, gltf_skinned_mesh, custom_loop, drag_and_drop, empty, empty_defaults, headless, logs, plugin, plugin_group, return_after_run, thread_pool_resources, no_renderer, without_winit, asset_loading, custom_asset, custom_asset_io, hot_asset_reloading, async_compute, external_source_external_thread, audio, audio_control, decodable, spatial_audio_2d, spatial_audio_3d, pitch, log_diagnostics, custom_diagnostic, ecs_guide, apply_deferred, component_change_detection, custom_query_param, event, fixed_timestep, generic_system, hierarchy, iter_combinations, parallel_query, removal_detection, run_conditions, startup_system, state, system_piping, system_closure, system_param, timers, alien_cake_addict, breakout, contributors, game_menu, char_input_events, gamepad_input, gamepad_input_events, gamepad_rumble, keyboard_input, keyboard_modifiers, keyboard_input_events, mouse_input, mouse_input_events, mouse_grab, touch_input, touch_input_events, text_input, reflection, generic_reflection, reflection_types, trait_reflection, scene, custom_vertex_attribute, post_processing, shader_defs, shader_material, shader_prepass, shader_material_screenspace_texture, shader_material_glsl, shader_instancing, animate_shader, compute_shader_game_of_life, array_texture, texture_binding_array, bevymark, many_animated_sprites, many_buttons, many_cubes, many_gizmos, many_foxes, many_glyphs, many_lights, many_sprites, transform_hierarchy, text_pipeline, scene_viewer, gamepad_viewer, nondeterministic_system_order, 3d_rotation, scale, transform, translation, borders, button, display_and_visibility, window_fallthrough, font_atlas_debug, overflow, overflow_debug, relative_cursor_position, size_constraints, text, text_debug, flex_layout, text_wrap_debug, grid, transparency_ui, z_index, ui, ui_scaling, ui_texture_atlas, viewport_debug, clear_color, low_power, multiple_windows, scale_factor_override, screenshot, transparent_window, window_settings, resizing, minimising, window_resizing, fallback_image
    If you want Rustdoc to scrape these examples, then add `doc-scrape-examples = true`
    to the [[example]] target configuration of at least one example.

Cargo.toml Outdated
@@ -2108,3 +2289,7 @@ codegen-units = 1
inherits = "release"
lto = "fat"
panic = "abort"

[package.metadata.docs.rs]
all-features = true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, not related. I saw this in another project and added it as a precaution in a separate commit. Should I skip it?

@killercup killercup changed the title Rustdoc: Scrape example Rustdoc: Scrape examples Aug 7, 2023
@killercup
Copy link
Contributor Author

Just so you know, new examples will need these flags in Cargo.toml. Maybe a good idea to check this in CI.

@cart cart enabled auto-merge August 10, 2023 00:08
@cart cart added this pull request to the merge queue Aug 10, 2023
Merged via the queue into bevyengine:main with commit 2cc1068 Aug 10, 2023
23 of 24 checks passed
cart added a commit that referenced this pull request Aug 10, 2023
# Objective

Provide more usage examples in API docs so that people can see methods
being used in context.

## Solution

Enable experimental rustdoc feature "scrape examples". See
<https://doc.rust-lang.org/nightly/rustdoc/scraped-examples.html> for
official docs.

## Example screenshots of examples :)

<img width="1013" alt="image"
src="https://github.com/bevyengine/bevy/assets/20063/7abc8baa-3149-476f-b2f2-ce7693758bee">

<img width="1033" alt="image"
src="https://github.com/bevyengine/bevy/assets/20063/163e7e22-c55e-46ab-8f3d-75fb97c3ad7a">

<img width="1009" alt="image"
src="https://github.com/bevyengine/bevy/assets/20063/a50b1147-e252-43f3-9adb-81960b8aa6c3">


## Limitations

- Only methods seem to show examples so far
- It may be confusing to have curated examples from doc comments
followed by snippets from `examples/`

---------

Co-authored-by: Carter Anderson <mcanders1@gmail.com>
@mockersf
Copy link
Member

I think this doesn't work. It was part of 0.11.1 so docs on docs.rs should be built with it but I can't find any scraped example

@dennisorlando
Copy link

I think this doesn't work. It was part of 0.11.1 so docs on docs.rs should be built with it but I can't find any scraped example

looks the same to me

@killercup
Copy link
Contributor Author

@mockersf FYI if we can make the CI for dev-docs run on nightly, it will pick up rust-lang/cargo#13074 and include examples :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-Docs An addition or correction to our documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants