I have a simple Seq2Seq model trained according to "Attention is all you need" and implemented using PyTorch. The model works fine. I decided to export it to ONNX. I exported the encoder and decoder separately. When using the ONNX model, the encoder works fine. However, the decoder only works for the same length of input sequence for which it was exported. For all other lengths, it ends with an error:
The input tensor cannot be reshaped to the requested shape. Input shape:{2,1,300}, requested shape:{1,20,15}
The embedding size is 300.
I don't think this is a problem with dynamic axes, as I set them correctly after the first failure. I tried to solve the problem by using a constant input length for the decoder and applying a mask, but this resulted in nonsensical output. Thank you in advance for any tips.
ONNX export of Seq2Seq model - issue with decoder input length
29 Views Asked by Dodiak At
0
There are 0 best solutions below
Related Questions in PYTORCH
- Influence of Unused FFN on Model Accuracy in PyTorch
- Conda CMAKE CXX Compiler error while compiling Pytorch
- Which library can replace causal_conv1d in machine learning programming?
- yolo v5 export to torchscript: how to generate constants.pkl
- Pytorch distribute process across nodes and gpu
- My ICNN doesn't seem to work for any n_hidden
- a problem for save and load a pytorch model
- The meaning of an out_channel in nn.Conv2d pytorch
- config QConfig in pytorch QAT
- Can't load the saved model in PyTorch
- How can I convert a flax.linen.Module to a torch.nn.Module?
- Snuffle in PyTorch Dataloader
- Cuda out of Memory but I have no free space
- Can not load scripted model using torch::jit::load
- Should I train my model with a set of pictures as one input data or I need to crop to small one using Pytorch
Related Questions in ONNX
- Stable Diffusion pipe always outputs 512*512 images regardless of the input resolution
- onnx runtime web run onnx, when enable gpu, cannot use dynamic input shape
- How to call onnx in onnx runtime web with dynamic input shape(ignoring input shape check)
- Device_map not wokring for ORTModelForSeq2SeqLM - Potential bug?
- Is dynamic axes configuration incorrect or converting to Torch Script required while converting the following Pytorch model to ONNX format?
- How to convert a python custom model class that wraps a scikit-learn pipeline containing a classifier to an onnx model?
- How to converting GIT (ImageToText / image captioner ) model to ONNX format
- When call onnx model, how to convert image file to correct model input
- Merging 6 ONNX Models into One for Unity Barracuda
- How can i fix a "TypeError: 'BatchEncoding' object is not an iterator" error
- finding the input size for detectron2 model to convert to onnx
- python - How can I retrain an ONNX model?
- Inference speed problem even if using a high-end Hardware
- ONNX export of Seq2Seq model - issue with decoder input length
- Pytorch model converted to Onnx Inference issue
Related Questions in SEQ2SEQ
- Does using FP16 help accelerate generation? (HuggingFace BART)
- how does nn.embedding for developing an encoder-decoder model works?
- why seq2seq model return negative loss if I used a pre-trained embedding model
- Is there a way for a closed domain chatbot to build using seq2seq, generative modeling or other methods like RNNs?
- Temporal Fusion Transformer model training encountered Gradient Vanishing
- Training a transformer to copy sequence to identical sequence?
- tensorflow multivariable seq 2 seq model return only lagged forcast
- Error: Invalid argument: ConcatOp : Dimensions of inputs should match
- Transforming keras model output during training and use multiple losses
- LSTM seq2seq input and output with different number of time steps
- predict sequence of tuples using Transformer model
- How does the finetune on transformer (t5) work?
- ValueError: Shapes (None, 16) and (None, 16, 16) are incompatible (LSTMs)
- How to translate my own sentence using Attention mechanism?
- how to create a seq2seq NLP model based on a transformer with BERT as the encoder?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?