ggml-org/llama.cpp

common: download from URL, improve parallel download progress status

Closed

#6,537 opened on Apr 8, 2024

View on GitHubContributor guide
 (7 comments) (0 reactions) (1 assignee)C++ (21,764 forks)batch import
enhancementgood first issuehelp wantedsplit

Repository metrics

Stars
 (124,134 stars)
PR merge metrics
 (Avg merge 6d 8h) (389 merged PRs in 30d)

Description

Context

When downloading a sharded model, files are downloaded in parallel, it was added in:

  • #6192

The progressions of each download conflict: image

Need to properly implement CURLOPT_NOPROGRESS for parallel download.

Example in #6515:

main --hf-repo ggml-org/models \
  --hf-file grok-1/grok-1-q4_0-00001-of-00009.gguf \
  --model   models/grok-1-q4_0-00001-of-00009.gguf \
  -ngl 64
   --prompt "I believe the meaning of life is"

Contributor guide

Research direction
Examine the existing parallel download implementation added in PR #6192, specifically how multiple CURL handles are managed and how progress is reported. Then research CURLOPT NOPROGRESS documentation to understand how to disable the default progress meter and implement a custom progress callback that aggregates progress from all handles without conflicting output. The relevant source code is likely in `common/download.cpp` or similar. Check comments and linked PR #6515 for an example usage. Implement the solution and test with a sharded model download.
Tech stack
cpp
Domain
cli
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Clibcurlparallel downloads
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40