llm-playground/daily_picks.md

Daily Picks

This is for tracking daily papers, daily news, my daily discoveries/thoughts/work in the area.

Inspired by GenAI_LLM_timeline and Daily Papers but personalized and focused.

• Milestone-ish models/datasets/apps are categorized as 🚀News, even if they come with papers.
• 📚Papers are for better understanding the mechanisms and not just a new model trained differently, good blogs are also counted as papers.
• ⚡Discoveries are what changed my perspective or practice.
• News are dated by the time they happened. Discoveries and papers are dated by the time I noticeed their importance[^1].
• Style: only key words in the table, extra info should be available via the link or the food note.

Date	📚Papers	🚀News	⚡Discoveries	🧠Thoughts/work
8.7	DL for mathematicians & Will TP+DL change Math?		Ask Mathlib4
7.9	DT-Solver
6.7	INSTRUCTEVAL
6.7	INSTRUCTEVAL
6.6	InstructZero
6.5	Video-LLaMA
6.5	RLHF-APA
6.5	Orca
6.5	Tr+SD
6.2	RefinedWeb
6.2	StyleDrop
6.1	Hiera: A Hierarchical ViT
6.1	Hidden Language in SD
6.1	Birth of a Transformer
6.1	ReviewerGPT
5.31	Grammar Prompting for DSL
5.28	Geometric Algebra Transformers
5.26		Falcon 7B/40B & RefinedWeb
5.26		Gorilla	TF Agents
5.24	Recursively
5.23	VanillaNet
5.23	Sophia
5.23	QLoRA	guanaco-65B
5.22	RWKV
5.22		GPT4All 13B Snoozy
5.21			The Little Book
5.20	Thought Forest
5.20		248 H100 SXM5s	Cooperation & Hyena
5.20	CodeCompose
5.18	Meaning
5.18	LIMA
5.18	Embodied Experiences
5.17	DoReMi
5.17	Safe-RLHF
5.17	ToT
5.16	StructGPT
5.15			{{Guidance}}
5.13		Prompt Leak
5.13		CodeT5+
5.12			spacy-llm
5.12	TinyStories
5.12	MEGABYTE
5.10		IMAGEBIND
5.10			Named Tensor Notation
5.6		MMS
5.6			MEMIT & REMEDI
5.5		RedPajama-INCITE 7B
5.5		OpenAlpaca
5.5	ALiBi & Lion	MPT-7B	Composer & StreamingDataset && LLM Foundry
5.5	SELF-ALIGN	IBM Dromedary 65B
5.4	APO
5.4	Multi Query Attention & Fill-in-the-Middle objective	StarCoder-15B	bigcode/Megatron-LM
5.3	Sourcegraph Cody
5.3	FasterTransformer	replit-code-v1-3b
5.3		OpenLLaMA 7B
5.3		Chatbot Arena
5.3	Distilling Step-by-Step
5.2	Unlimiformer
5.2	Loss Landscapes
5.1	Self-Notes
4.29		Lamini 12B
4.28		StableVicuna 13B[^6]
4.28	Causal Reasoning & LLM
4.28	Iterative Bootstrapping
4.27	Formal Transformers
4.26	Transformers
4.26			HELM & benchmarks
4.26			Silent Bugs
4.26			Kernl
4.21			137 emergent abilities
4.21			Training logbook & metric
4.21			axolotl & genv
4.20	Verifiability
4.19			GPTCache
4.19	FlashAttention	StableLM	GPT-NeoX & Megatron
4.19			meerkat[^5]
4.19			CAMEL & chatarena
4.18	FT v.s. LoRA	BELLE
4.18		LLaVA
4.17			Alpaca-CoT
4.17		RedPajama-Data
4.17			alpaca_lora_4bit
4.17			Transformer Family
4.16	LLMs + Symbolic Solvers
4.16	suggest_premises
4.15		MiniGPT-4
4.15		web-llm
4.14			Buzzard's talk
4.14			ProofNet
4.14	Multimodal C4
4.13	CodeWhisperer
4.13	GPT-4 Annotating
4.12			LLMPruner
4.12	Galactic ChitChat
4.12		Dolly v2
4.12		DeepSpeed Chat
4.11	Toxicity
4.11	Privacy Attacks
4.11	Self-Debug
4.11	Auto-Sci
4.12			RunPod.io
4.10	pal
4.9			Patrick's talk :octocat:
4.9	ACT
4.9			dagster & mage-ai
4.8			data-centric-AI
4.8	Training Recipe
4.7		lightning & lit-llama
4.7		Vicuna
4.5		SAM
4.5		StackLLaMA & trl
4.4			text-generation-webui
4.4	LLM-Adapters
4.3			ChatML
4.3		Koala
4.2	Code Self-Improvement
4.2			ChuanhuChatGPT
4.1		LMFlow
3.31	Choose Your Weapon
3.30	Humans in Humans Out
3.30		galpaca-30b
3.30		BloombergGPT
3.30		Auto-GPT
3.29			guardrails & lmql & kor
3.29		GPT4All
3.29		LLaMA-Adapter
3.29			llama_index
3.28		OpenFlamingo
3.28		Cerebras-GPT
3.27		LeCun's talk
3.26	Low-Rank Simplicity Bias
3.25	APE
3.24		Dolly
3.23			dalai[^4]
3.23		ChatGPT Plugins
3.23			Cursor.so[^3]
3.22		Sparks of AGI
3.20		ChatGPT outage
3.16		Alpaca LoRA
3.15		GPT-4 TR
3.14		GPT-4
3.13		Alpaca
3.2			miniF2F
3.1		ChatGPT API
3.1			galai
2.26			ColossalAI[^2]
2.24		LLaMA
2.10		ChatGPT Plus
2.7		New Bing
2023
2022.11.30		ChatGPT
2021.08.10		Codex
2020.05.28		GPT-3

TODO

Decide whether include them and determine dates:

Date	Papers	News	Discoveries	Thoughts/work
4.18	SPQA
4.9			spaCy
3.31			simple-llm-finetuner
3.19		Web AI
1.6		NeevaAI

Related curated lists

Papers & Notes

• Mooler0410/LLMsPracticalGuide - A curated list of practical guide resources of LLMs.
• thunlp/PromptPapers - Must-read papers on prompt-based tuning for pre-trained language models.
• foocker/deeplearningtheory
• dair-ai/ML-Course-Notes - 🎓 Sharing machine learning course / lecture notes.
• ml4code - A Survey of Machine Learning for Big Code and Naturalness
• Everything-LLMs-And-Robotics - The world's largest GitHub Repository for LLMs + Robotics

Models

• Longyichen/Alpaca-family-library - Summarize all low-cost replication methods for Chatgpt.
• imaurer/awesome-decentralized-llm - Collection of LLM resources that can be used to build products you can "own" or to perform reproducible research.
• nichtdax/awesome-totally-open-chatgpt - A list of totally open alternatives to ChatGPT
• FreedomIntelligence/LLMZoo - ⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡
• arjunbansal/awesome-oss-llm-ift-rlhf - Collection of open source implementations of LLMs with IFT and RLHF that are striving to get to ChatGPT level of performance
• stanford-crfm/ecosystem-graphs - an ongoing effort to track the foundation model ecosystem

Training

• zhilizju/Awesome-instruction-tuning - A curated list of awesome instruction tuning datasets, models, papers and repositories.
• yaodongC/awesome-instruction-dataset - A collection of open-source dataset to train instruction-following LLMs (ChatGPT,LLaMA,Alpaca)
• PhoebusSi/Alpaca-CoT - We unified the interfaces of instruction-tuning data (e.g., CoT data), multiple LLMs and parameter-efficient methods (e.g., lora, p-tuning) together for easy use
• visenger/awesome-mlops - A curated list of references for MLOps

Reasoning

• lupantech/dl4math - Resources of deep learning for mathematical reasoning (DL4MATH).
• tensorush/Awesome-Maths-Learning - :sunglasses: :scroll: Collection of the most awesome Maths learning resources in the form of notes, videos and cheatsheets.

Prompting

• dair-ai/Prompt-Engineering-Guide - 🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

Apps

• reorx/awesome-chatgpt-api - Curated list of apps and tools that not only use the new ChatGPT API, but also allow users to configure their own API keys, enabling free and on-demand usage of their own quota.

[^1]: Models and datasets are already tracked seperately as simple machine-digestable files as models.txt and datasets.txt, and some on my likes. Repos are tracked by my stars, mostly in topic chatgpt, chatgpt-api, ai, artificial-intelligence, data-science and data-analysis, also in my star list lean-llm focusing on the building blocks of applying LLMs to the ITP/ATP area.

[^2]: The first open source RLHF pipeline

[^3]: Helped me experience prompt-based coding infinitely

[^4]: Helped me testing LLaMA and Alpaca locally

[^5]: Meerkat is a Python library for interactively exploring unstructured data with foundation models that understand them, you can also seamlessly switch between augmented data frames and reactive GUIs for easy verification and feedback.

[^6]: The AI World’s First Open Source RLHF LLM Chatbot