sign/translate

Human Pose Viewer - Generate any person based on an image

Open

#33 opened on Dec 21, 2021

View on GitHubContributor guide
 (1 comment) (0 reactions) (0 assignees)TypeScript (164 forks)auto 404
enhancementhelp wantedspoken-to-signed

Repository metrics

Stars
 (777 stars)
PR merge metrics
 (PR metrics pending)

Description

Expected Behavior

The human pose viewer should be able to generate any face based on an encoding of a single image.

Actual Behavior

The current human pose viewer is limited to generating two people.

Ideas

Use something like https://arxiv.org/pdf/2103.06902.pdf image

Or like that - https://vcai.mpi-inf.mpg.de/projects/Styleposegan/

Whenever a new person is selected by uploading an image, create and encode a texture map into a vector. This process can be slow, as it happens once per person. (If a person is selected based on the existing catalog, we already have their vectors)

Given the latent vector, generate video based on that vector and a pose sequence

Contributor guide

Research direction
Research the referenced papers (e.g., StylePoseGAN) and design a pipeline to encode a face image into a latent vector, then generate video frames from pose sequences. Implement the encoding module and integrate with the viewer.
Tech stack
typescript
Domain
machine learningaibackend
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
Machine LearningPython
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30