Microsoft has introduced its latest iteration of the lightweight AI model called Phi-3 Mini. This model, the first of three smaller versions to be released by the company, boasts 3. 8 billion parameters and has undergone training on a relatively smaller dataset compared to larger language models like GPT-4. Users can now access Phi-3 Mini on platforms such as Azure, Hugging Face, and Ollama. Microsoft has plans to release two more versions, Phi-3 Small (7B parameters) and Phi-3 Medium (14B parameters), with parameters representing the model's capacity to understand complex instructions. In December, Microsoft released Phi-2, which performed on par with larger models like Llama 2. According to Microsoft, Phi-3 showcases even better performance than its predecessor and is capable of providing responses similar to models ten times its size. Eric Boyd, corporate vice president of Microsoft Azure AI Platform, emphasizes that Phi-3 Mini provides comparable capabilities to larger models like GPT-3. 5 but in a smaller form factor. Compared to larger models, smaller AI models are often more cost-effective to operate and deliver improved performance on personal devices like smartphones and laptops. Earlier this year, it was reported that Microsoft was assembling a team dedicated to developing lighter-weight AI models.

In addition to Phi, Microsoft has created Orca-Math, a model focused on solving mathematical problems. Other tech giants also offer their own small AI models, many of which target simpler tasks like document summarization or coding assistance. For example, Google's Gemma 2B and 7B are ideal for simple chatbots and language-related work, while Anthropic's Claude 3 Haiku excels at reading and summarizing dense research papers with graphs. Meta's recently released Llama 3 8B can be utilized for chatbots and coding assistance. Boyd elaborates on the development process, stating that developers trained Phi-3 using a "curriculum" inspired by how children learn from bedtime stories and books with simpler language and sentence structures that address broader topics. By leveraging an extensive list of over 3, 000 words, an LLM was used to create "children's books" to teach Phi. Furthermore, Boyd explains that Phi-3 builds upon the knowledge gained from previous iterations. While Phi-1 focused on coding and Phi-2 started to learn reasoning, Phi-3 excels in both coding and reasoning. Although the Phi-3 family has some understanding of general knowledge, it cannot match the breadth of a GPT-4 or another large language model trained on the entirety of the internet.