source_code_classification/README.md

Source Code Classification

nl2ml_notebook_parser.py - script for parsing Kaggle notebooks and process them to JSON/CSV/Pandas.

bert_distances.ipynb - notebook with expiremints concerning sense of distance between BERT embeddings where input tokens were tokenized source code chunks.

bert_classifier.ipynb - notebook with preprocessing and training pipeline.

regex.ipynb - notebook with creating labels for code chunks with regex

logreg_classifier.ipynb.ipynb - notebook with building logreg on the regex labels with tf-idf