Key Questions Before Starting an LLM Startup
Before diving into an LLM-based startup, you should think through these five questions carefully. Failing to do so is a recipe for trouble down the road. ...
NLP
Phi-2: The Surprising Power of Small Language Models
Microsoft released Phi-2, a 2.7 billion parameter language model that demonstrates outstanding reasoning and language understanding capabilities, achieving state-of-the-art performance among base language models with fewer than 13 billion parameters. On complex benchmarks, Phi-2 matches or outperforms models roughly 25 times its size, thanks to innovations in model scaling and training data curation. ...
Textbooks Are All You Need: Key Takeaways
Microsoft recently proposed an intriguing approach: training models on synthetic textbooks instead of the massive datasets typically used. Paper: https://arxiv.org/abs/2306.11644 ...
A ChatGPT-Written Hospital Appointment Bot
Anyone who’s tried booking an appointment at Peking University School of Stomatology knows how difficult it is. So let’s have ChatGPT write an appointment bot. Unfortunately, the booking logic it produced is hilariously superficial — basically a no-op: ...
Introduction to Retrieval-Augmented Generation (RAG)
Retrieval-Augmented Generation (RAG) is a natural language processing approach that combines pretrained parametric and non-parametric memory to improve performance on knowledge-intensive NLP tasks. This post covers the RAG framework and its potential applications. ...