AI, Quant 투자 공부

TinyML and Efficient Deep Learning Computing MIT 6.5940 Fall 2023 TinyML and Efficient Deep Learning Computing Grading The class requirements include five labs, and one final project. This is a PhD level course, and by the end of this class you should have a good understanding of efficient deep learning techniques, and be able to deploy large language models (LLMs) hanlab.mit.edu 안정적인 LLM 서비스를 위..

Transformer Math 101 Transformer Math 101 We present basic math related to computation and memory usage for transformers blog.eleuther.ai MemGPT: Towards LLMs As Operating Systems Understanding the Performance of Transformer Inference Understanding the Performance of Transformer Inference Abstract The state of the art results in natural language processing tasks have been obtained by scaling up ..

The Current State of Continual Learning in AI The current state of continual learning in AI Why is ChatGPT only trained up until 2021? towardsdatascience.com A Comprehensive Survey of Continual Learning: Theory, Method and Application

An Intuition for Attention An Intuition for Attention | Jay Mody Deriving the equation for scaled dot product attention. jaykmody.com De-coded: Transformers explained in plain English De-coded: Transformers explained in plain English No code, maths, or mention of Keys, Queries and Values towardsdatascience.com

Efficient Streaming Language Models with Attention Sinks Efficient Streaming Language Models with Attention Sinks Deploying Large Language Models (LLMs) in streaming applications such as multi-round dialogue, where long interactions are expected, is urgently needed but poses two major challenges. Firstly, during the decoding stage, caching previous tokens' Key and Val arxiv.org Ring Attention wi..

Add Your Own Data to an LLM Using Retrieval-Augmented Generation (RAG) Add Your Own Data to an LLM Using Retrieval-Augmented Generation (RAG) Learn how to add your own proprietary data to a pre-trained LLM using a prompt-based technique called Retrieval-Augmented Generation. towardsdatascience.com RAG vs Finetuning — Which Is the Best Tool to Boost Your LLM Application? RAG vs Finetuning — Which..

Making Deep Learning Go Brrrr From First Principles

모델 parallelism Tensor Parallelism Column Parallelism Input 텐서: 복사 Weight 텐서: 수직방향으로 분할 내적 후 Concatenation 수행 Row Parallelism Input 텐서: 수직방향 Weight 텐서: 수평방향으로 분할 내적 후 add 수행 All-Gather와 Scatter 통신 생략하는 방법 Column-Row 순서대로 2개의 Liner 레이어를 연결함 : All-Gather와 Scatter 통신 생략 가능함 Attention 레이어를 Tensor Parallelism QKV projection: Column 방향으로 병렬화함 Output projection: Row 방향으로 병렬화함 Column-Row 순으로 연결 시 All-Gat..

7 Ways To Speed Up Inference of Your Hosted LLMs 7 Ways to Speed Up Inference of Your Hosted LLMs TLDR; techniques to speed up inference of LLMs to increase token generation speed and reduce memory consumption betterprogramming.pub Fixing Hallucinations in LLMs

From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting NExT-GPT: Any-to-Any Multimodal LLM When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Optimizing your LLM in production Do Machine Learning Models Memorize or Generalize? Do Machine Learning Models Memorize or Gen..