facebookresearch/maskrcnn-benchmark

Inference time increases linearly with batch size

Open

#294 opened on Dec 20, 2018

View on GitHubContributor guide
 (8 comments) (2 reactions) (0 assignees)Python (2,574 forks)batch import
help wanted

Repository metrics

Stars
 (9,161 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

❓ Questions and Help

I tested the batch inference，this is result batch size time 1 0.22 2 0.42 4 0.84 6 1.2 8 crash OOM Why big batchsize doesn't save time?

Contributor guide

Research direction
Investigate why batch inference does not speed up per image and leads to OOM. Check if the model or data loading has memory inefficiencies, or if the batching implementation is suboptimal. Profile memory and time for different batch sizes.
Tech stack
pythonpytorch
Domain
machine learningaiperformance
Issue type
Research
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PythonPyTorch
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65