AI, Quant 투자 공부

Microsoft CopilotMicrosoft Copilot은 AI 기반의 도우미로, 사용자들이 더 빠르고 효율적으로 작업을 수행할 수 있도록 돕는 다양한 기능을 제공합니다. Copilot은 Microsoft 365, Dynamics 365, Power Platform 등 여러 Microsoft 제품에 통합되어 있으며, 다양한 작업 환경에서 사용자의 생산성을 높이는 데 기여합니다. 다음은 Copilot의 주요 기능과 활용 사례에 대한 설명입니다.### 주요 기능1. **텍스트 생성**   - Copilot은 자연어 처리(NLP) 기술을 활용하여 사용자가 입력한 간단한 지시에 따라 이메일, 보고서, 제안서 등 다양한 문서를 작성할 수 있습니다. 사용자는 몇 가지 키워드나 문장을 제공하기만 하면 Copil..

Exploring LLM pricing Is Flash Attention Stable? Efficient and Economic Large Language Model Inference with Attention Offloading Efficient and Economic Large Language Model Inference with Attention OffloadingTransformer-based large language models (LLMs) exhibit impressive performance in generative tasks but introduce significant challenges in real-world serving due to inefficient use of the exp..

Data Engineering for Scaling Language Models to 128K Context Are All Large Language Models Really in 1.58 Bits? Are all LLMs really 1.58 bits? Inference at 4x the speed or more?Dive deep in to changes to the Transformer architecture to learn about how researchers have discovered a huge speedup in LLM inference.learning-exhaust.hashnode.devFrugalGPT: How to Use Large Language Models While Reducin..

https://learning-exhaust.hashnode.dev/are-all-large-language-models-really-in-158-bits?ref=twitter-share Are all LLMs really 1.58 bits? Inference at 4x the speed or more? Dive deep in to changes to the Transformer architecture to learn about how researchers have discovered a huge speedup in LLM inference. learning-exhaust.hashnode.dev https://www.latent.space/ Latent Space | swyx & Alessio | Sub..

https://twitter.com/virattt/status/1778828787951546382 X의 virat님(@virattt) Friday is LLM battle day. I added DBRX to the financial metrics challenge. Overall, very impressed with DBRX. Main takeaways: • correctly calculated metrics • ranked top 4 fastest models • competitive pricing DBRX was +50% cheaper and +100% faster th twitter.com https://colab.research.google.com/gist/virattt/f290fa4ec878f..

Transformer math 101 Transformer Math 101 We present basic math related to computation and memory usage for transformers blog.eleuther.ai LLM inference - HW/SW Optimizations LLM Inference - HW/SW Optimizations | Notion Linkedin의 원저자(Sharada Yeluri)의 허락을 받아 원문을 번역 및 검수중입니다. tulip-phalange-a1e.notion.site Optimizing your LLM in production Optimizing your LLM in production Optimizing your LLM in pr..

LLM inference serives 5 disecting model performance 네 가지 종류의 성능 병목 현상 compute-bound memory bandwidth bound communications bound overhead bound : software-induced limitations 대부문의 처리 시간이 작업 일정을 계획하고 이를 HW에 제출하는데 소요 런타임에 필요한 모든 정보(텐서 데이터 유형, 장치, 호출할 커널)를 명시적으로 지정할 필요가 없는 유연한 언어(python)이나 프레임워크(pytorch)를 사용할 때 오버헤드 바인딩이 발생할 가능성이 높아짐 병목 현상 유형에 따라 지연 시간을 단축하도록 최적화 compute-bound 높은 peak FLOPS의 칩으로 업그레..

Open Source AI is AI we can Trust - with Soumith Chintala of Meta AI Open Source AI is AI we can Trust — with Soumith Chintala of Meta AI Listen now | The PyTorch creator riffs on geohot's Tinygrad, Chris Lattner's Mojo, Apple's MLX, the PyTorch Mafia, the upcoming Llama 3 and MTIA ASIC, AI robotics, and what it takes for open source AI to win! www.latent.space LLM inference Provider LeaderBoard..

Transfomer Alternatives 더 긴 context가 sub-quadratic을 위한 동기가 아님 transformer는 context size에 강한 제약을 가지고 있지 않지만 context size가 클수록 극단적으로 비쌈 Hyena와 같은 sub-quadratic architecture는 추론과 학습면에서 더 값 쌈 하이브리드 아키텍처의 emergence 하이브리드 아키텍처란 transformer-baed와 state-space으로 레이어를 분리함 당면 과제 중 하나는 많은 하드웨어 커널이 transformer operation에 최적화되었기 때문에 이를 떠나면 완벽하게 많은 것을 잃음 더 높은 속도 = 더 높은 GPU throughput subquadratic architecture ..

OLMo: Accelerating the Science of Language Models OLMo: Accelerating the Science of Language Models Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details o arxiv.org Understanding, Using, and Finet..