Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolving framework symbols #1564

Closed
tmds opened this issue Apr 17, 2020 · 12 comments
Closed

Resolving framework symbols #1564

tmds opened this issue Apr 17, 2020 · 12 comments

Comments

@tmds
Copy link
Member

tmds commented Apr 17, 2020

As described in https://github.com/dotnet/runtime/blob/master/docs/project/linux-performance-tracing.md, to do performance tracing and resolve framework symbols, you should

fetch EXACTLY the right version of crossgen for the runtime you happen to be using

For source-build, the simplest may be to include crossgen?
This could be included in the runtime package, or maybe as part of a debug symbols package.

In our source-build .NET Core 3.1 we include pdb files for Microsoft.NETCore.App dlls.
I'm not sure what tools can use these pdb files? and how they relate to needing, or not needing crossgen?
Also, we don't seem to have pdb files for Microsoft.AspNetCore.App, so we may be missing debug info for ASP.NET framework?

cc @brianrob @noahfalk @omajid @dagood @crummel @dseefeld

@dagood
Copy link
Member

dagood commented Apr 17, 2020

(Context I grabbed from the doc to get myself up to speed:) This is about perf tracing the shared framework (runtime) we crossgen during source-build. Not apps that are crossgen'd as part of runtime-dependent or self-contained publish.

include crossgen?

The clear way forward here is #1215. There's no obvious place to stick it based on the Microsoft build's current behavior. (Not that it can't be unique to source-build... but would be nice not to have to invent something.)

Also, we don't seem to have pdb files for Microsoft.AspNetCore.App

Filed #1565 for this.


I think it would be useful to get this sentence clearer from that doc:

Framework symbols need to be manually generated at the time the trace is collected.

Why "need"? Is there something about perf tracing where the data doesn't exist, e.g. it's app-specific in some way? Or is it just because the symbols aren't generally accessible, so we can provide PDBs instead if we've managed to keep track of them?

(I think I'm basically stating the same question here. 😛)

@noahfalk
Copy link
Member

I think it would be useful to get this sentence clearer from that doc:

Framework symbols need to be manually generated at the time the trace is collected.

Why "need"? Is there something about perf tracing where the data doesn't exist, ...

When the runtime is first deployed onto a machine the symbolic data that maps RVAs to method names for R2R code is present inside the R2R images, however the profiling tools don't understand how to directly parse the R2R format to extract the data. The tools do understand a textual format called perfmap so the crossgen tool is used to read the RVA->method name map from the R2R format and transcode it to the textual perfmap format. If you skip this step then the IPs for code loaded from R2R images will not be resolvable in the trace.

@tmds
Copy link
Member Author

tmds commented Apr 20, 2020

Also, we don't seem to have pdb files for Microsoft.AspNetCore.App
Filed #1565 for this.

Thanks @dagood !

When the runtime is first deployed onto a machine the symbolic data that maps RVAs to method names for R2R code is present inside the R2R images, however the profiling tools don't understand how to directly parse the R2R format to extract the data.

@noahfalk thank you for chiming in. I wonder what is the proper way forward here:

@noahfalk
Copy link
Member

The first option there, including crossgen with source-build, is the one I'd suggest exploring first. Its possible further research would discover a problem, but two things that make it appealing to me:

  • Regardless of profiling, developers may want to use crossgen to precompile their own app. If crossgen is already being provided for one purpose it is usually low-cost to acquire/use it for a 2nd purpose as well.
  • When customers are deploying their own applications using R2R binaries they compiled then they will want perfmap data for those binaries when profiling. If crossgen is available as a tool on the machine then they can get perfmap files regardless of how their app was deployed. If crossgen is not available then additional onus falls on the customer to pre-create the perfmap at build time and figure out how it would be deployed when it is needed.

Also to cover all the bases I guess I should ask, is it reasonable that perf could understand enough of the R2R binary format that we didn't need to transcode it? For customers I suspect it would be ideal if no pre-processing was needed, but I don't know if this is something the maintainers of perf would consider?

@tmds
Copy link
Member Author

tmds commented Apr 27, 2020

Also to cover all the bases I guess I should ask, is it reasonable that perf could understand enough of the R2R binary format that we didn't need to transcode it? For customers I suspect it would be ideal if no pre-processing was needed, but I don't know if this is something the maintainers of perf would consider?

I'm not a perf contributor, based on input I got, this is a valid option.

@noahfalk
Copy link
Member

Thanks @tdms!

@davidwr @jkotas @MichalStrehovsky - What would you guys think about having an external tool (perf) understand enough of the R2R format that it could decode symbolic information directly? The upside is that it makes .NET feel more platform native, the downside is that it increases our compat burden. I don't know how much churn you were anticipating the format to have in the future?
cc @sdmaclea

@jkotas
Copy link
Member

jkotas commented Apr 27, 2020

I do not think that the format is complete and stable enough to do that.

@jkotas
Copy link
Member

jkotas commented Apr 27, 2020

I think that the right options for this are:

  • Option 1: Generate the R2R images with platform native envelope (e.g. .so) and then acquire the platform native symbols for these as any other platform native symbols.
  • Option 2: Generate the symbols at runtime using perf jitdump or something similar.

@brianrob
Copy link
Member

I agree with @jkotas that the options he specifies are good choices. FWIW, I intend to update perfcollect to implement option # 2.

@noahfalk
Copy link
Member

Thanks all!

Generate the R2R images with platform native envelope (e.g. .so) and then acquire the platform native symbols for these as any other platform native symbols.

Is this a task that has pre-existing priority around it, or we'd need to drive it based on the demands of the perf scenario? I recalled hearing desire to use platform native formats over the years, but I hadn't gotten the impression it was something where active progress was occuring (or would be likely to occur in the near future).

FWIW, I intend to update perfcollect to implement option # 2.

My primary concern (if I am remembering correctly) is that it does a flushed file write on every R2R method load which probably distorts startup a fair amount. Am I more worried about it than I should be or any thoughts on how we ameliorate it?

@sdmaclea
Copy link
Contributor

sdmaclea commented Apr 27, 2020

flushed file write on every R2R method load

The flushing was removed to improve perf. dotnet/runtime#229

@jkotas
Copy link
Member

jkotas commented Apr 27, 2020

Generate the R2R images with platform native envelope (e.g. .so) and then acquire the platform native symbols for these as any other platform native symbols.

Is this a task that has pre-existing priority around it, or we'd need to drive it based on the demands of the perf scenario?

This is not scheduled at the moment, but I can imagine it getting prioritized at some point for one or more of: performance, diagnostics, security, be Linux native.

@dotnet dotnet locked and limited conversation to collaborators Nov 16, 2023
@MichaelSimons MichaelSimons converted this issue into discussion #3740 Nov 16, 2023
@github-project-automation github-project-automation bot moved this from Backlog to Done in .NET Source Build Nov 16, 2023

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

8 participants