[Question]: how to completely convert SequenceTagger model to ONNX format

[trinhtuanvubk]

Question

I want to convert SequenceTagger model to ONNX format. As I see it, the model consists of two parts: embedding and 'after embedding', I use the code below to convert these two parts to ONNX format

For embedding: output shape: [ batch, lengths, 768]

model = SequenceTagger.load(model_path)
assert isinstance(model.embeddings, TransformerWordEmbeddings)
sentences = [Sentence("This is a sentence.")]
model.embeddings = model.embeddings.export_onnx("flert-embeddings.onnx", sentences, providers=["CUDAExecutionProvider", "CPUExecutionProvider"])

For 'after embedding': input shape: [ batch, lenghts, 768]

model = SequenceTagger.load(model_path)
example_sentence = Sentence("This is a sentence.")
longer_sentence = Sentence("This is a way longer sentence to ensure varying lengths work with LSTM.")
reordered_sentences = sorted([example_sentence, longer_sentence], key=len, reverse=True)
tensors = model._prepare_tensors(reordered_sentences)
torch.onnx.export(
      model,
      tensors,
      "sequencetagger.onnx",
      input_names=["sentence_tensor"],
      output_names=["scores"],
      opset_version=9,
      verbose=True,
  )

So I have 2 questions:

1. In training phase, Did the embedding update weights or freeze?
2. How can I convert model to only 1 ONNX?

[trinhtuanvubk]

@helpmefindaname Hi, Im sorry for tagging you here. I have gone through many your issues of this problem, hope you can help me. Thank you

[helpmefindaname]

Hi @trinhtuanvubk

1. Onnx-Embeddings are not trainable. Just train with the transformer embeddings and specify if you want the weights to be updated or not with the fine_tune parameter.
2. There are Onnx tools for that. In terms of flair, this is not directly supported.

[trinhtuanvubk]

@helpmefindaname Thank you for you answer.

• I relized that I updated the embedding weights during finetuning phase. And when converting, I had to convert the embedding and the model separately and then combine them.

[trinhtuanvubk]

I have updated code to convert sequence tagger model to onnx on https://github.com/trinhtuanvubk/NER-Flair-ONNX/tree/main
hope its useful if someone need