Harmonizing Multi-GPUs: Efficient Scaling of LLM Inference
TitanML docs | TitanML docs
The machine learning optimization company
docs.titanml.co
In the Fast Lane! Speculative Decoding - 10x Larger Model, No Extra Cost
TitanML docs | TitanML docs
The machine learning optimization company
docs.titanml.co
Large Language Models - the hardware connection
Large Language Models — the hardware connection | APNIC Blog
Guest Post: The role of networking when scaling LLM architecture's gigantic models.
blog.apnic.net
Tensor Parallelism - NADDOD Blog
Tensor parallelism alleviates memory issues in large-scale training. RoCE enables efficient communication for GPU tensor parallelism, accelerating computations.
www.naddod.com
Fast and Expressive LLM inference with Radixattention and SGLang
Fast and Expressive LLM Inference with RadixAttention and SGLang | LMSYS Org
<p>Large Language Models (LLMs) are increasingly utilized for complex tasks that require multiple chained generation calls, advanced prompting techniques, co...
lmsys.org
'Daily-Trend-Review' 카테고리의 다른 글
| 24/02/06: Why GPT-3.5 is (mostly) cheaper than Llama2 (0) | 2024.02.06 |
|---|---|
| 24/02/04: fine-tune your lown llama 2 model in a colab note book (0) | 2024.02.04 |
| 2024/01/26: Leading with open Models, frameworks, and systems (0) | 2024.01.26 |
| 2024/01/20: 스터디 내용 정리 (0) | 2024.01.20 |
| 2024/01/20: LLM Agents, DPO (0) | 2024.01.20 |