You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using default_beam_search of BeamRNNTInfer with partial_hypothesis.
We put the the last token from the previous step's hypothesis in this part. The last token is used to get the probability of predicted token of the current step from the decoder. For instance, If the last token is s, then the result would be {'s?': -2.3274096590466797, 's.': -0.9459273740649223}. Then we remove the last token from the hypothesis of current step and the final result of default_beam_search would be {'?': -2.3274096590466797, '.': -0.9459273740649223} by this logic.
However, in my case, my model somehow results the blank token, for instance, {'s?': -2.3274096590466797, 's': -0.9459273740649223} then, the logic do not filter s, which is the last token of previous hypothesis. It makes the function returns hypothesis like this {'?': -2.3274096590466797, 's': -0.9459273740649223}, which leads the last token 's'` appear once again. (Actually, the original problem here is the model predicts the blank token. Would be thankful if you let me know why this is happening!)
So, my suggestion is how about modifying like this?
# Remove the original input label if partial hypothesis was provided
if partial_hypotheses is not None:
for hyp in kept_hyps:
if hyp.y_sequence[0] == partial_hypotheses.y_sequence[-1]:
hyp.y_sequence = hyp.y_sequence[1:]
if len(hyp.y_sequence) == 0:
hyp.y_sequence = [self.blank]
return self.sort_nbest(kept_hyps)
I think the right place is the Issues tab, but I cannot reveal the codes for reproduction for now. I just want to make a quick report without preparing the another codes to reproduce this situation.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi, I have found a possible decoding error in current beam search decoding logic: nemo/collections/asr/parts/submodules/rnnt_beam_decoding.py#L663.
I am using
default_beam_search
ofBeamRNNTInfer
withpartial_hypothesis
.We put the the last token from the previous step's hypothesis in this part. The last token is used to get the probability of predicted token of the current step from the decoder. For instance, If the last token is
s
, then the result would be{'s?': -2.3274096590466797, 's.': -0.9459273740649223}
. Then we remove the last token from the hypothesis of current step and the final result ofdefault_beam_search
would be{'?': -2.3274096590466797, '.': -0.9459273740649223}
by this logic.However, in my case, my model somehow results the blank token, for instance,
{'s?': -2.3274096590466797, 's': -0.9459273740649223}
then, the logic do not filters
, which is the last token of previous hypothesis. It makes the function returns hypothesis like this {'?': -2.3274096590466797, 's': -0.9459273740649223}, which leads the last token
's'` appear once again. (Actually, the original problem here is the model predicts the blank token. Would be thankful if you let me know why this is happening!)So, my suggestion is how about modifying like this?
I think the right place is the Issues tab, but I cannot reveal the codes for reproduction for now. I just want to make a quick report without preparing the another codes to reproduce this situation.
Beta Was this translation helpful? Give feedback.
All reactions