LLM-PaperReview

Review papers of NLP, mainly LLM. NLP / LLM 관련 논문 리뷰 레포입니다.

• 스케일부터 메소드까지. ChatGPT의 상업적 성공과 여러 발군의 오픈소스들을 업고 LLM은 나날이 빠른 속도로 성장하고 있습니다.
• 쏟아지는 LLM 관련 논문들을 깊고 넓게 공부하고 토론하고자 합니다. :)
• 논문 선정은, Pond에 모아둔 논문들 중 하나를 발표자가 발표일 1주 전까지 선정하여 공지하는 것으로 이루어집니다.

Semesters

• 2023 8/6 - 2023 10/19 Every Thursday(Finished).
• 2023 11/23 - 2024 2/1 Every Thursday(Finished).

일정 및 선정 논문

2기

	Paper	a.k.a	Affiliation	published date	Speaker	Youtube
11.23	REPLUG: Retrieval-Augmented Black-Box Language Models	REPLUG	Washington Univ.	May. 2023	김한성	LINK
11.30	Prefix-Tuning: Optimizing Continuous Prompts for Generation	Prefix-Tuning	Standford Univ.	Jan. 2021	임서연	LINK
12.7	QLoRA: Efficient Finetuning of Quantized LLMs	QLoRA	Washington Univ.	May. 2023	이상민	LINK
12.21	Direct Preference Optimization: Your Language Model is Secretly a Reward Model	DPO	Stanford Univ.	May. 2023	천재원	LINK
12.28	Efficient Streaming Language Models with Attention Sinks	StreamingLLM	MIT	Dec. 2023	김가영	LINK
1.4	Efficient Memory Management for Large Language Model Serving with PagedAttention	vLLM	UC Berkeley	Sep. 2023	이주형	LINK
1.18	Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks	RAG	FAIR	April. 2021	신중현	LINK
1.25	Mixtral of Experts	Mixtral	Mistral.AI	Jan. 2024	신혁준	LINK
2.1	Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions	-	Oxford Univ.	Oct. 2023	김현수	LINK

1기

	Paper	a.k.a	Affiliation	published date	Speaker	Youtube
8.17	RoFormer: Enhanced Transformer with Rotary Position Embedding	RoPE	Zhuiyi Technology	August. 2022	천재원	LINK
8.24	TRAIN SHORT, TEST LONG: ATTENTION WITH LINEAR BIASES ENABLES INPUT LENGTH EXTRAPOLATION	ALiBi	Facebook	April. 2022	이주형	LINK
8.31	Finetuned Language Models Are Zero-Shot Learners	FLAN	Google	Sep. 2021	천소영	LINK
9.7	WizardLM: Empowering Large Language Models to Follow Complex Instructions	WizardLM	Microsoft	Jun. 2023	박경택	LINK
9.14	G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment	G-Eval	Microsoft	May. 2023	신혁준	LINK
9.21	SimCSE: Simple Contrastive Learning of Sentence Embeddings	SimCSE	Princeton Univ.	May. 2022	김세형	LINK
10.5	LLaMA: Open and Efficient Foundation Language Models	LLaMA	Meta	Feb. 2023	김가영	LINK
10.12	LoRA: Low-Rank Adaptation of Large Language Models	LoRA	Microsoft	Oct. 2021	신중현	LINK
10.19	Training language models to follow instructions with human feedback	InstructGPT	OpenAI	March. 2022	홍영훈	LINK