Daily-Trend-Review

2023/12/23: how to make LLMs go fast

hellcat 2023. 12. 23. 07:36

https://vgel.me/posts/faster-inference/

Table of Contents

 

How to make LLMs go fast

How to make LLMs go fast December 18, 2023 In my last post, we made a transformer by hand. There, we used the classic autoregressive sampler, along the lines of: This approach to inference is elegant and cuts to the heart of how LLMs work—they're autoreg

vgel.me