-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Profiler only generates xplane.pb (Tensorflow 2.11 & 2.12) #613
Comments
Hi HLneoh, just want to check that if the tools (eg. overview page, tensorflow stats..etc) still works for you? As if they still works fine for you, this should be something expected. Because our recent change removed dependency on those materialized JSON tool files, but the data will be generated from xplane.pb on the fly directly. |
Hi @HLneoh, I have an environment that is similar to your env2, but it seems that due to issues with protobuf, the profile page is not displayed at all. It would be greatly appreciated if you could share your protobuf version with me, tks! |
This might have to do with location of profile data being different starting in 2.12 (can't test 2.11 because of cuda dependency). I was able to get this to work by copying the data manually with tensorflow v2.13. |
System information
Ubuntu 22.04
Virtual env #1
Virtual env #2
Description
Profiler only generates xplane.pb. Missing plugin files for Overview Page, Input Pipeline Analyzer, TensorFlow Stats, Trace Viewer, GPU Kernel Stats, Memory Profile Tool and Pod Viewer.
Results
Warning obtained as below when viewing the overview page.
No step marker observed and hence the step time is unknown. This may happen if (1) training steps are not instrumented (e.g., if you are not using Keras) or (2) the profiling duration is shorter than the step time. For (1), you need to add step instrumentation; for (2), you may try to profile longer.
The text was updated successfully, but these errors were encountered: