English

The first text model created is based on tweets collected from July 2016 until February 2020. From this collection we randomly selected approximately 5,000,000, excluding retweets and tweets having less than two words, to create the text model. The text model is created using the following paramters.

from b4msa.textmodel import TextModel
tm = TextModel(usr_option="delete",
               num_option="delete",
               url_option="delete",
               emo_option="none",
               token_min_filter=0.001,
               token_max_filter=0.999)

The aforementioned model is a bag of word model, where the number of tokens is 17,827 (i.e., \(m_b: \text{text} \rightarrow \mathbb R^{17827}\)).

This model (without using the text model trained with the training set) can be used as follow:

>>> from EvoMSA.base import EvoMSA
>>> evo = EvoMSA(TR=False, B4MSA=True, lang='en')

The next table shows the different models we have produced for the English language.

Source	Model
Sentiment strength detection forthe social web [SS]	SS-Youtube_En.evomsa
A corpus forresearch on deliberation and debate [SCv1]	SCv1_En.evomsa
Creating and characterizing a diverse corpus of sarcasm in dialogue [SCv2-GEN]	SCv2-GEN_En.evomsa
Multilingual Twitter sentiment classification: The role of human annotators [Human-Annotated]	Human annotated model
SemEval-2017 Task 4: Sentiment analysis in Twitter [Task-4]	semeval2017_En.evomsa
SemEval-2018 Task 1: Affect in tweets [Task-1]	semeval2018_anger_En.evomsa semeval2018_fear_En.evomsa semeval2018_joy_En.evomsa semeval2018_sadness_En.evomsa semeval2018_valence_En.evomsa

The following code shows the usage of two of the models with suffix .evomsa. The models used are semeval2017_En.evomsa, SCv1_En.evomsa, as well as \(m_b\) text-model, and without the text model obtained with the training set.

>>> from EvoMSA.utils import download
>>> sem2017 = download('semeval2017_En.evomsa')
>>> scv1 = download('SCv1_En.evomsa')
>>> evo = EvoMSA(TR=False, B4MSA=True, lang='ar',
                 models=[[sem2017, "sklearn.svm.LinearSVC"],
                         [scv1, "sklearn.svm.LinearSVC"]])

where sklearn.svm.LinearSVC can be any classifier following the structure of sklearn. evo is a instance of EvoMSA and it is missing to train it with a training set, i.e., call evo.fit.

It is important to mention that all the pre-trained models can be used alone. For example, the following lines show how to use SCv1_En.evomsa.

>>> from EvoMSA.utils import download
>>> from microtc.utils import load_model
>>> tm = load_model(download('SCv1_En.evomsa'))

At this point the model is in tm; this text model can be used with the function tm.predict.

>>> tm.predict(["Have a nice day", "I hate this movie"])