자연어처리 (Natural language processing)
- 강의실: xxx
- Textbooks
- Dan Jurafsky and James H. Martin, Speech and Language Processing (3rd ed. draft), 2024 [link]
- Textbooks (additional)
- Jacob Eisenstein, Natural lanuage processing, 2018 [pdf]
- Christopher M. Bishop and Hugh Bishop, Deep learning: Foundations and concepts, Springer, 2023, [book]
- I. Goodfellow, Y. Bengio, A. Courville, Deep learning, MIT Press, 2016 [book]
- L. Tunstall, L. von Werra, T. Wolf, Natural Language Processing with Transformers, O'Reilly Media, 2022 [book]
- Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998 [link] [2nd_edition]
- References
- Lecture on deep learning for NLP at Stanford U. [link]
- Lecture on natural language computing at UoT [link]
- Lecture on natural language processing at UW [link]
- Lecture on recent advances on foundation models at UWaterloo [link]
- Pytorch [link]
- TensorFlow [link]
- Tensorflow Lecture at Stanford U. [link]
- AI lecture at Brown Univ.
- AI lecture at Berkeley Univ. [link]
- AIML lecture at Princeton Univ. [link]
- Deep RL lecture at Berkeley Univ. [link]
- Deep RL Tutorial at ICML '17 [link]
- 1. Introduction [pdf]
- 2. N-gram language models [pdf]
- Chapter 3, SLP2024 book [pdf]
- Katz backoff [link]
- Good-Turing smoothing [pdf]
- Brants et al. 2007, Large Language Models in Machine Translation, Google [pdf]
- 3. Markov Models [pdf]
- Chapter Appendix. A: Hidden Markov Models [pdf]
- 4. Sequence Labeling for Parts of Speech and Named Entities [pdf]
- 5. Probabilistic Context Free Grammars (PCFGs) & Parsing [pdf]
- Chapter 17: Context-Free Grammars and Constituency Parsing [pdf]
- Chapter Appendix. C: Statistical Constituency Parsing [pdf]
- Chapter Appendix. D: Context Free Grammars [pdf]
- 6. Vector Semantics and Embeddings [pdf]
- 7. Word Embeddings [pdf]
- 8. Neural Networks and Neural Language Models [pdf]
- Lecture on Neural Networks (at UW) [pdf]
- 9. RNNs and LSTMs [pdf][pdf]
- 10. Sequence to Sequence Models and Machine Translation [pdf]
- Lecture on Machine Translation (at UW) [pdf]
- 11. Transformers [pdf]
- 12. Pretrained language models [pdf]
- 13. Large Language Models & Alignment [pdf]
- 14. Large Language Models: Benchmarks & Evaluation [pdf]
- 15. Efficient Neural Network Training [pdf]
- DeepSpeed [pdf]
- DeepSpeed-MoE [pdf]
- 16. Large Language Models: Reasoning & Agents [pdf]
- 17. Large Language Models & In-Context Learning & Prompting
- 18. Retrieval-augmented Language Models [pdf]
- ACL 2023 Tutorial: Retrieval-based Language Models and Applications: [link]
- 19. Large Language Models: Knowledge Editing [pdf]
- 20. Multimodal Language Models [link]
- 21. LLM: life after DPO & challenges [pdf]
- 22. NLP, linguistics, and philosophy [pdf]
- Assignment 1: 제출기한 - 5월 13일
- Assignment 2: 제출기한 - 6월 9일
- Assignment 3: 제출기한 - 6월 23일