I've been experimenting with sentiment analysis using Recurrent Neural Networks (RNNs) following a tutorial I found here. While my model is functional, I've hit a roadblock with its accuracy, consistently achieving between 85-90%. I've attempted various optimizations, but I'm uncertain about what constitutes a reasonable baseline to conclude my efforts, especially considering the potential efficiency of a Transformer-based LSTM model.
My main query is not centered around enhancing accuracy further; rather, I seek guidance on determining when to consider a sentiment analysis model as sufficiently effective for practical use, particularly when employing RNNs for this task. As I'm relatively new to machine learning, I lack clarity on when to determine that my objective has been accomplished.
Hmm, this is certainly not an easy question to answer. Generally, at a very basic level, your model should outperform a random baseline.
There's definitely nothing wrong with using your model with 85-90% accuracy and should be fine for 'most' applications (I'm using the term 'most' loosely here). But again, if you're doing this research purposes, then higher performance is appreciated (but this is not the only factor that determines significance of findings).
In addition to the above point, the use of a trained model (and the milestone determining if you've achieved your goal) is pertinent / highly subjective to the dataset and the severity of the downstream task.
That being said, it might be worth finetuning a pretrained model on your dataset (if you have one), if you haven't done so.