<h2>Stocks Tweets Project</h2><p>This project is devoted to the study of the relationship between publications in social networks and the corresponding stock prices.&nbsp;</p><p>All code is open-source, here is the link to GitHub: <a target="_blank" rel="noopener noreferrer" href="https://github.com/fin-algo-lake-ai/stocks-tweets-project">https://github.com/fin-algo-lake-ai/stocks-tweets-project</a></p><h3>Dataset</h3><p>At the first stage of the project, the original dataset from the article [Jaggi M. at al., 2021] was taken for work. It contains 6.4 million tweets for 25 stocks.&nbsp;</p><p>After examining the dataset a number of issues had been found and fixed:&nbsp;</p><ul><li>added stop words preprocessing (initially stop words were just removed)</li><li>corrected filling missing data algorithm to more stable one</li><li>data labels were recalculated in a different way (with isolation in time of the message release date and the price change date)<br>&nbsp;</li></ul><h3>Experiment setup</h3><p>After loading the dataset, it is divided into train and test sets in a ratio of 0.85 to 0.15. To check the stability and variability of each model, the split is carried out several times with a different seed value. Each partition is tested independently of the others.</p><p>Of all the dataset fields, only one is used - the preprocessed message text, the rest of the features (date, time, author, etc.) are intentionally not used in the pipeline of this work to avoid distortion in the search for the correlation of the entities under study.</p><p>To estimate model quality, we used the Accuracy metric. As the dataset is perfectly balanced, this metric is very intuitive, as the value 0.5 corresponds to random noise.</p><p>After training and saving all the models, an ensemble of models is created to evaluate the cross-correlation of the models and obtain the final maximum achievable accuracy value.<br>&nbsp;</p><h3>Results</h3><p>Below is a final table and graph with results of individual models and their ensembles.</p><figure class="table" style="float:left;"><table><tbody><tr><td style="background-color:#9fc5e8;border:1pt solid #000000;padding:5pt;"><p style="text-align:center;"><strong>Model Type</strong></p></td><td style="background-color:#9fc5e8;border:1pt solid #000000;padding:5pt;"><p style="text-align:center;"><strong>Accuracy</strong></p></td><td style="background-color:#9fc5e8;border:1pt solid #000000;padding:5pt;"><p style="text-align:center;"><strong>Model</strong></p><p style="text-align:center;"><strong>File Size</strong></p></td><td style="background-color:#9fc5e8;border:1pt solid #000000;padding:5pt;"><p style="text-align:center;"><strong>Comments</strong></p></td></tr><tr><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;"><p>Baseline</p><p>(Random Guess)</p></td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">0.500 +- 0.003</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">-</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">It’s not an actual model (DummyClassifier with strategy="uniform")</td></tr><tr><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">Transformer (Roberta)</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">0.545 +- 0.002</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">350 MB</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">Roberta and DistilRoberta had the same accuracy in our experiments</td></tr><tr><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">GBDT (Catboost)</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">0.560 +- 0.003</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">5 MB</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;"><p>Model details: CatBoostClassifier</p><p>n_estimators=300, max_depth = 8</p></td></tr><tr><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">Naive Bayes</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">0.568 +- 0.002</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">58 MB + 49 MB</td><td style="border:1pt solid #000000;padding:5pt;vertical-align:top;">Model details: MultinomialNB on TF-IDF features for word n-grams with length from 1 to&nbsp; 3, alpha = 0.1</td></tr></tbody></table></figure><p><img src="https://lh4.googleusercontent.com/-JpcdTjg_XplXbavyguxB2ZMhK4kylOX7HrA9o1mXiwmQ5XB9O7ECZChm9XB7GdVh15UWKtQUkwCDubqSQbSC-SQ78q6mGoPH2--QjUBKYMcjQhnqLp1uXxf1voqmALWDt4c7hCd-eG0CLK-Abq9BuOw2NZVAo8urLxVR-ntDrbbg4FnVlQgFTZ-4DjcMA"></p><p style="text-align:center;">Boxplot graphs for pairwise and triple ensembles.<br><i>(BL - baseline model (random uniform), CB - CatBoost model</i><br><i>NB - Naive Bayes model, TR - Transformer model)</i></p><p>&nbsp;</p><h3>Conclusion and future work</h3><p>The results turned out to be promising - despite the complexity of the subject area (predicting future price behavior), we managed to get statistically significant accuracy values (0.582) with a random baseline of 0.500 + -0.003. A separate interesting conclusion was that the size and algorithmic complexity of machine learning models do not always determine the result of their application in each specific case, requiring researchers to conduct more versatile experiments at the modeling stage.</p><p>Possible next steps:</p><ul><li>Calculate business-oriented metrics (closer to the real potential usage of the idea)</li><li>Labels: Take not just the next calendar day, but the next N days (up to a week). Also it's possible to experiment with the 0.5% threshold that is used for class label calculation.</li><li>Models: Complete results with RNN approaches (LSTM / GRU) - the main runs have already been carried out, they need to be added to the final ensemble.</li></ul><p>&nbsp;</p><h3>References</h3><ol><li>Yilmaz, E. S., Ozpolat, A., &amp; Destek, M. A. (2022). Do Twitter sentiments really effective on energy stocks? Evidence from the intercompany dependency. Environmental Science and Pollution Research, 29(52), 78757-78767.<br><a target="_blank" rel="noopener noreferrer" href="https://www.academia.edu/download/89681397/s11356-022-21269-9.pdf">https://www.academia.edu/download/89681397/s11356-022-21269-9.pdf</a></li><li>Jaggi, M., Mandal, P., Narang, S., Naseem, U., &amp; Khushi, M. (2021). Text Mining of Stocktwits Data for Predicting Stock Prices. arXiv preprint arXiv:2103.16388.<br><a target="_blank" rel="noopener noreferrer" href="https://arxiv.org/abs/2103.16388">https://arxiv.org/abs/2103.16388</a></li><li>Rahul Pandey R. (2021) Common Pitfalls to Avoid in Forecasting Models for Stock Price Prediction<br><a target="_blank" rel="noopener noreferrer" href="https://medium.com/geekculture/common-pitfalls-to-avoid-in-forecasting-models-for-stock-price-prediction-3a7c3ff8b80">https://medium.com/geekculture/common-pitfalls-to-avoid-in-forecasting-models-for-stock-price-prediction-3a7c3ff8b80</a></li><li>Adusumilli, R. (2020). NLP in the Stock Market. Leveraging sentiment analysis on 10-k fillings as an edge<br><a target="_blank" rel="noopener noreferrer" href="https://towardsdatascience.com/nlp-in-the-stock-market-8760d062eb92">https://towardsdatascience.com/nlp-in-the-stock-market-8760d062eb92</a></li><li>Sun, Y., Liu, X., Chen, G., Hao, Y., &amp; Zhang, Z. J. (2020). How mood affects the stock market: Empirical evidence from microblogs. Information &amp; Management, 57(5), 103181.<br><a target="_blank" rel="noopener noreferrer" href="https://www.sciencedirect.com/science/article/pii/S0378720618307183">https://www.sciencedirect.com/science/article/pii/S0378720618307183</a></li><li>Yuz, T. (2018). A Sentiment Analysis Approach to Predicting Stock Returns<br><a target="_blank" rel="noopener noreferrer" href="https://medium.com/@tomyuz/a-sentiment-analysis-approach-to-predicting-stock-returns-d5ca8b75a42">https://medium.com/@tomyuz/a-sentiment-analysis-approach-to-predicting-stock-returns-d5ca8b75a42</a></li></ol><h3><br>Contacts and feedback</h3><p>Please, feel free to contact us at this Telegram group: <a target="_blank" rel="noopener noreferrer" href="https://t.me/tweets_for_stocks_feed">https://t.me/tweets_for_stocks_feed</a></p><p>&nbsp;</p>

Finding correlation between tweets and future stock prices using ML/NLP approaches
[this is a graduation project of the course "Natural Language Processing course, stream 3, autumn 2022"]

Tweets for Stocksactive, Founded 3 years ago

Tweets for Stocks
active,
Founded 3 years ago