You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
In some routine profiling check on performance on typical usage scenarios , (significant) slowdown is detected after some recent code change. After analysis, _is_annotation_tid() in data_store is identified as the method currently consuming too much time and causing slow down. After detailed analysis and further debugging, it looks like with the recent underlying changes making the look up of the dictionaries (populated by methods affected by changes) different than before , and newly added exception handling code also affects performance.
To Reproduce
Steps to reproduce the behavior:
using standard profiling test code such as in Profiling new data pack speed #805 (using a standard pipeline for NLTK based POS tagger and NER processing)
Perform profiling test in PyCharm
See performance difference, and the Cprofile analysis report and Call Graph it generated
Expected behavior
_is_annotation_tid() is identified to be "hot"(consumes significant time) by cprofile. Also further debugging shows excessive exceptions (more than previous version) was thrown from it.
Environment (please complete the following information):
OS: All
Version : current code base (0.3, snapshot of Feb.06 or Feb.14)
Python and Package verions: 3.8
Additional context
(Currently investigating) recent underlying code change related to tid, entry and related code for populating related dictionary
The text was updated successfully, but these errors were encountered:
J007X
changed the title
_is_annotation_tid() in data_store throwing (significantly) more exceptions (than before) and causing slowing down in typical usage scenarios
_is_annotation_tid() in data_store exceptions throwing causing (significant) slowing down in typical usage scenarios (such as NLP)
Mar 13, 2023
Adjust the title and description slightly to reflect latest investigation results -- after adding some tracing/debugging code it seems the slow down is caused by exception throwing code newly added (in which the Call Graph identified related method as "green" as they are system method). The dictionary access was different from before however the changes are small and not causing this significant changes in performance.
Describe the bug
In some routine profiling check on performance on typical usage scenarios , (significant) slowdown is detected after some recent code change. After analysis, _is_annotation_tid() in data_store is identified as the method currently consuming too much time and causing slow down. After detailed analysis and further debugging, it looks like with the recent underlying changes making the look up of the dictionaries (populated by methods affected by changes) different than before , and newly added exception handling code also affects performance.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
_is_annotation_tid() is identified to be "hot"(consumes significant time) by cprofile. Also further debugging shows excessive exceptions (more than previous version) was thrown from it.
Environment (please complete the following information):
Additional context
(Currently investigating) recent underlying code change related to tid, entry and related code for populating related dictionary
The text was updated successfully, but these errors were encountered: