Multi Linguage Sentiment Classification - Part 1

This project is the first part of a serie of two blogs. It aims to leverage Hugging Face transformers in order to perform multi linguage sentiment analysis.

Motivation

There are more and more people across the glob, increasingly sharing their opinion on social media platforms, review sites in different languages. To be able to efficiently analyze what is being expressed about them sentiment-wise, industries and organizations need to find the right technologies that is not only focused on English language.

This is what is the overal goal of this project, aimining to develop a tool able to understand sentiment expressed in different languages to finally those organisations to make the right decisions.

The Dataset

The dataset used for this project is license-free available on Sentiment NLPRoc.

Acknowledgements

1. Cross-Lingual Propagation for Deep Sentiment Analysis
2. A Helping Hand: Transfer Learning for Deep Sentiment Analysis

Scope of the project

The previous cited papers covers 9 languages. For simplicity sake, we will focus on the following 5 languages as described in the table below.

Prerequisites

• Python 3.6+
• Transformers 3.1.0
• All the requirements are specified in the requirements.txt file

Usage

Set Up of the project from the root directory of the project

• Create virtual environment

python3 -m venv your_virtual_environment

• Start virtual environment

source your_virtual_environment/bin/activate

Run the experimentation

This script will do the following tasks:

1. create the data folders: data/raw and data/processed
2. Unzip the original data into the data/raw folder
3. Run the processing to create the final data set
4. Run the evaluation to track the pretrained models performance using MLFlow tracking

• Run the automation script

chmod +x prepare_environment.sh
./prepare_environment.sh

After that you should be able to see the experimentation tracking from the Experimentation tab in the top left corner