facebookresearch/fairscale

[chore] Cache wikitext2 dataset used for benchmarks/pipe.py to prevent throttling errors

Open

#743 opened on Jul 19, 2021

View on GitHubContributor guide
 (3 comments) (0 reactions) (1 assignee)Python (297 forks)auto 404
good first issue

Repository metrics

Stars
 (3,411 stars)
PR merge metrics
 (PR metrics pending)

Description

Chore

We should cache the dataset used for running fairscale/benchmarks/pipe.py so that we don't end up downloading this at every run. This has been done for OSS and we should do something similar for pipe.py. This is important because during periods of high PR checkins/development we could run into throttling errors if there are too many CircleCI jobs. That is something that possibly happened for OSS which is when we added the caching logic.

Contributor guide

Research direction
Examine the caching logic from the OSS PR (https://github.com/facebookresearch/fairscale/pull/465) and apply a similar caching mechanism to the wikitext2 dataset in benchmarks/pipe.py. Ensure the dataset is downloaded and cached in a persistent location to avoid re downloading on each CircleCI run.
Tech stack
python
Domain
devopsbuild system
Issue type
Chore
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
2
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Python
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60