<h2>Deteсt trolls and bots in YouTube comments</h2><h3>Research project</h3><p>For detection noise information (bot and troll messages) we need to make following:</p><ul><li>Because we don't have labeled data we have to analyze data in order to be able to find method for noise message and usefully &nbsp;information separating. A small study was done to solve this problem. (See <a target="_blank" rel="noopener noreferrer" href="https://github.com/mvlebedev/ods_nlp_huawei_fall_2022/blob/main/Report/report_eng.pdf">report</a> and <a target="_blank" rel="noopener noreferrer" href="https://github.com/mvlebedev/ods_nlp_huawei_fall_2022">repository</a> with project).</li><li>In the second part of project we need to develop machine learning model for troll and bots detection.</li></ul><h3>Team and contribution</h3><ul><li>At the moment, the project has only one main developer (email mixam.maxim@gmail.com), you are free to contact and ask any questions. We may discuss further development.</li></ul><h3><strong>Related Work</strong></h3><p>A short list of related works and their descriptions can be found <a target="_blank" rel="noopener noreferrer" href="https://github.com/mvlebedev/ods_nlp_huawei_fall_2022/blob/main/Report/report_eng.pdf">here</a>.</p><h3>Experiment design</h3><figure class="image"><img src="https://storage.yandexcloud.net/ds-ods/files/content/2023/02/05/abb3b99e/diagram.png"></figure><h3>Dataset</h3><p>The data will be comments from YouTube channel "вДудь". On these videos, the blogger interviews famous actors, politicians, etc. The program for uploading comments to a file is in the <a target="_blank" rel="noopener noreferrer" href="https://github.com/mvlebedev/ods_nlp_huawei_fall_2022">repository.</a></p><h3>Results</h3><p>Three approaches were used to identify troll and bot comments (See pictures below too). The first is topic modeling with <strong>LDA</strong>. The second method of parsing messages is based on the use of embeddings obtained from <strong>RuBert</strong>. Further, for this vector representation, the <strong>GMM</strong> and <strong>K-means</strong> clustering methods were used. Based on the hand review of the data for the corresponding clusters, a conclusion was made about the class number to which the noise messages correspond. The third approach is {t-distributed} stochastic neighbor nesting (<strong>t-SNE</strong>) applied to <strong>RuBert embeddings</strong>. Noise comments were considered points lying on the periphery of a circle of a given radius.As a result, the analysis of texts that were marked as noise (with bots and trolls), we can conclude that thematic modeling showed the worst. There is a lot of information in his texts that does not look like the work of trolls and bots. Methods based on <strong>RuBert embeddings</strong> showed very good results. Approach based on the identification of noise points at the <strong>t-SNE</strong> boundary performance showed good results. This method does not require additional analysis of classes like all other methods. The only thing he selected is less than all the points (objects) for the noise class. This is due to the choice of a large circle radius.&nbsp;</p><h4><strong>Latent Dirichlet Allocation</strong></h4><p>On the following picture is demonstrated distribution of words in topics. We may guess that topic 6 has spam information.</p><figure class="image image_resized" style="width:96.49%;"><img src="https://storage.yandexcloud.net/ds-ods/files/content/2023/02/28/cd8dc79f/themes.png"></figure><h4><strong>t-SNE и GMM (Noise comments: class 3)</strong></h4><figure class="image"><img src="https://storage.yandexcloud.net/ds-ods/files/content/2023/02/28/e2b65574/tsne_gmm.png"></figure><h4><strong>t-SNE и K-means (Noise comments: class 3)</strong></h4><figure class="image"><img src="https://storage.yandexcloud.net/ds-ods/files/content/2023/02/28/20a98479/tsne_kmeans.png"></figure><p>&nbsp;</p><h4><strong>t-SNE and selection of points on the periphery of a circle}</strong></h4><figure class="image"><img src="https://storage.yandexcloud.net/ds-ods/files/content/2023/02/28/2257b77d/tsne_outside_circle.png"></figure>

Max Lebedev

The project devoted to the development of an approach for
definitions of bots and trolls in Russian-language comments on the YouTube service.
It was created as part of a project for the «Natural Language Processing course» by HUAWEI

Deteсt trolls and bots in YouTube commentsactive, Founded 3 years ago

Deteсt trolls and bots in YouTube comments
active,
Founded 3 years ago