pytorch/text

Vocab vectors using complete pretrained-embedding?

Open

#446 opened on Oct 12, 2018

View on GitHub
 (6 comments) (0 reactions) (0 assignees)Python (822 forks)batch import
enhancementhelp wanted

Repository metrics

Stars
 (3,396 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

I am new to pytorch and nlp. I have a question when I tried to build a model.

Since my training dataset is not so big, the size of its vocab is relatively small (around 5000). However, I want to deal with any other user input which could be out of this vocabulary.

The problem is, in the model I trained, the embedding layer's weight is based on the vectors of the field, not the whole word2vec pretrained embeddings. So I cannot modified it after the training is done.

I wondered is there any better approach to do it? Thanks in advance!

Contributor guide

Research direction
Investigate how PyTorch's torchtext handles vocabulary and embedding layers. Look into existing discussion on this issue (e.g., comments on GitHub issue #446) and evaluate if a utility function for loading full pretrained embeddings is feasible. Consider adding a method to allow users to specify a pretrained embedding source that maps tokens not in the training vocab to their vectors from a larger set.
Tech stack
python
Domain
machine learning
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
PyTorch basicsword embeddings
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
50