Arabic

The first text model created is based on tweets collected from January 2017 until February 2020. From this collection we randomly selected approximately 5,000,000, excluding retweets and tweets having less than two words, to create the text model. The text model is created using the following paramters.

from b4msa.textmodel import TextModel
tm = TextModel(usr_option="delete",
               num_option="delete",
               url_option="delete",
               emo_option="none",
               token_min_filter=0.001,
               token_max_filter=0.999)

The aforementioned model is a bag of word model, where the number of tokens is 24,906 (i.e., \(m_b: \text{text} \rightarrow \mathbb R^{24906}\)).

This model (without using the text model trained with the training set) can be used as follow :

>>> from EvoMSA.base import EvoMSA
>>> evo = EvoMSA(TR=False, B4MSA=True, lang='ar')

The next table shows the different models we have produced for the Arabic language. The first row contains the Human annotated model. The remaining rows include the different models for two competitions run at SemEval.

Source	Model
Arabic Sentiment Analysis and Cross-lingual Sentiment Resources	Human annotated model
SemEval-2017 Task 4: Sentiment analysis in Twitter [Task-4]	semeval2017_Ar.evomsa
SemEval-2018 Task 1: Affect in tweets [Task-1]	semeval2018_anger_Ar.evomsa semeval2018_fear_Ar.evomsa semeval2018_joy_Ar.evomsa semeval2018_sadness_Ar.evomsa semeval2018_valence_Ar.evomsa

The following code shows the usage of two of the models with suffix .evomsa. The models used are semeval2017_Ar.evomsa, semeval2018_valence_Ar.evomsa, as well as \(m_b\) text-model, and without the text model obtained with the training set.

>>> from EvoMSA.utils import download
>>> sem2017 = download('semeval2017_Ar.evomsa')
>>> valence = download('semeval2018_valence_Ar.evomsa')
>>> evo = EvoMSA(TR=False, B4MSA=True, lang='ar',
                 models=[[sem2017, "sklearn.svm.LinearSVC"],
                         [valence, "sklearn.svm.LinearSVC"]])

where sklearn.svm.LinearSVC can be any classifier following the structure of sklearn. evo is a instance of EvoMSA and it is missing to train it with a training set, i.e., call evo.fit.