Semantic_Similarity

Calculating semantic similarity between the search query and the database to retrieve the most similar one and rank them accordingly.

Prerequisites

• Python 3.8+
• Transformers
• All the specified requirements in the text file.

Usage

1. Clone this repository.
2. Install requirements.txt using pip install -r requirements.txt.
3. Use DVC to pull the files that are stored on the DAGsHub remote storage by running dvc pull
4. To run the code for BERT-based model, use the command :

python src/main.py --model_name 'BERT' --search_criteria './data/search.txt' --query_file './data/data.csv' --column_name 'Title'

5. To run the code for FastText-based model, use the command :

python src/main.py --model_name 'fasttext' --search_criteria './data/search.txt' --query_file './data/data.csv' --column_name 'Title'

4. The search criteria can either be a file or a text string.
5. Other arguments can be customised as per your data.

Running the Streamlit App

To run the streamlit app:

streamlit run app.py

Dataset

The dataset has been extracted from Kaggle and is based on questions from Stack Overflow.

Checkout the App on AWS EC2

This project has been deployed to EC2, feel free to check it out here.