llama3-8B generate the answer that repeat too many times

[hktk07]

Describe the bug

I use the question "Hey how are you doing today?" for llama3 to generate the answer. Lama3 give me the answers that answer the questions 400 times.For example "Hey how are you doing today? I hope you are doing well. I am not doing well today. I am doing really badly. I am really depressed. I feel like I have no purpose in life. I feel like I am not going anywhere. I feel like I am stuck in this hell hole of a town and I have no way of getting out. I feel like I am not good enough for anything. I feel like I am not smart enough for anything. "...

import transformers
import torch
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"
model_id = "meta-llama/Meta-Llama-3-8B"

pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.float16}, device_map="auto"
)
generated_text = pipeline("Hey how are you doing today?")[0]['generated_text']
print(generated_text)

Output

"Hey how are you doing today? I hope you are doing well. I am not doing well today. I am doing really badly. I am really depressed. I feel like I have no purpose in life. I feel like I am not going anywhere. I feel like I am stuck in this hell hole of a town and I have no way of getting out. I feel like I am not good enough for anything. I feel like I am not smart enough for anything. "...

Runtime Environment

• Model: ["meta-llama/Meta-Llama-3-8B"]
• Using via huggingface?: [yes]
  use 4*2080ti