ggml-org/llama.cpp

Support for Phi-3 models

Closed

#6,849 opened on Apr 23, 2024

View on GitHubContributor guide
 (84 comments) (61 reactions) (0 assignees)C++ (21,764 forks)batch import
good first issuemodel

Repository metrics

Stars
 (124,134 stars)
PR merge metrics
 (Avg merge 6d 8h) (389 merged PRs in 30d)

Description

Microsoft recently released Phi-3 models in 3 variants (mini, small & medium). Can we add support for this new family of models.

Contributor guide

Research direction
Review the existing model implementations in llama.cpp (e.g., examples/llama.cpp) to understand the pattern for adding new architectures. Study the Phi 3 model architecture from Microsoft's release. Check the comments in this issue for any partial implementation or hints from maintainers. Look at the conversion scripts in examples/convert*.py for model conversion.
Tech stack
cpp
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Over 1 week
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Phi 3 model architectureC++ programmingllama.cpp codebase
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
25