I have a simple Seq2Seq model trained according to "Attention is all you need" and implemented using PyTorch. The model works fine. I decided to export it to ONNX. I exported the encoder and decoder separately. When using the ONNX model, the encoder works fine. However, the decoder only works for the same length of input sequence for which it was exported. For all other lengths, it ends with an error:
The input tensor cannot be reshaped to the requested shape. Input shape:{2,1,300}, requested shape:{1,20,15}
The embedding size is 300.
I don't think this is a problem with dynamic axes, as I set them correctly after the first failure. I tried to solve the problem by using a constant input length for the decoder and applying a mask, but this resulted in nonsensical output. Thank you in advance for any tips.
ONNX export of Seq2Seq model - issue with decoder input length
29 views Asked by Dodiak At
0
There are 0 answers
Related Questions in PYTORCH
- Influence of Unused FFN on Model Accuracy in PyTorch
- Conda CMAKE CXX Compiler error while compiling Pytorch
- Which library can replace causal_conv1d in machine learning programming?
- yolo v5 export to torchscript: how to generate constants.pkl
- Pytorch distribute process across nodes and gpu
- My ICNN doesn't seem to work for any n_hidden
- a problem for save and load a pytorch model
- The meaning of an out_channel in nn.Conv2d pytorch
- config QConfig in pytorch QAT
- Can't load the saved model in PyTorch
- How can I convert a flax.linen.Module to a torch.nn.Module?
- Snuffle in PyTorch Dataloader
- Cuda out of Memory but I have no free space
- Can not load scripted model using torch::jit::load
- Should I train my model with a set of pictures as one input data or I need to crop to small one using Pytorch
Related Questions in ONNX
- Stable Diffusion pipe always outputs 512*512 images regardless of the input resolution
- onnx runtime web run onnx, when enable gpu, cannot use dynamic input shape
- How to call onnx in onnx runtime web with dynamic input shape(ignoring input shape check)
- Device_map not wokring for ORTModelForSeq2SeqLM - Potential bug?
- Is dynamic axes configuration incorrect or converting to Torch Script required while converting the following Pytorch model to ONNX format?
- How to convert a python custom model class that wraps a scikit-learn pipeline containing a classifier to an onnx model?
- How to converting GIT (ImageToText / image captioner ) model to ONNX format
- When call onnx model, how to convert image file to correct model input
- Merging 6 ONNX Models into One for Unity Barracuda
- How can i fix a "TypeError: 'BatchEncoding' object is not an iterator" error
- finding the input size for detectron2 model to convert to onnx
- python - How can I retrain an ONNX model?
- Inference speed problem even if using a high-end Hardware
- ONNX export of Seq2Seq model - issue with decoder input length
- Pytorch model converted to Onnx Inference issue
Related Questions in SEQ2SEQ
- Should I use beam search on validation phase?
- How to finetune the LLM to output the text with SSML tags?
- I am deploying a seq2seq model for a text2sql generation, i want to be sure that i am on the right path
- Seq2Seq Model input shape
- How to optimise Hyperparameters for Whisper finetuning?
- Transformers // Predicting next transaction based on sequence of previous transactions // Sequence2One task
- ONNX export of Seq2Seq model - issue with decoder input length
- TensorFlow Model with multiple inputs and a single output (Text Based)
- only use Bartmodel BartEncoder to replace seq2seq encoder(I'm an NLP kid)
- BERT fine tuned transformer for chat bot not meeting expected performance
- Wrong Shape Output from Tensorflow Model with Custom Layers
- What are differences between T5 and Bart?
- Pytorch nn.LSTM: RuntimeError: For unbatched 2-D input, hx and cx should also be 2-D but got (3-D, 3-D) tensors
- tensorflow multivariable seq 2 seq model return only lagged forcast
- Training a transformer to copy sequence to identical sequence?
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)