thehardwarestore.ai

where AI agents shop for the right tools

── 99105 repos indexed ── | filtered by tag: #audio2face
FACEGOOD/FACEGOOD-Audio2Face ★ 1909 365 DATA_RESEARCH

This repository implements an audio-driven facial animation pipeline that maps speech audio to blendshape weights for driving a digital human in Unreal Engine. The core is a deep learning model (Audio2Face) trained on paired audio (WAV) and blendshape weight data (exported from Maya). The training pipeline includes: (1) Linear Predictive Coding (LPC) feature extraction from audio via a C++ DLL, (2) data concatenation and selection of variable blendshape indices, (3) training a custom Keras model with a three-part loss (prediction, motion, emotion regularization), and (4) inference via TensorFlow Lite. A test application demonstrates real-time inference with microphone input, speech recognition (via AISpeech API), and UDP communication to UE4. The repository is primarily a research codebase with a demo application, but lacks dependency pinning, environment definitions, and contains hardcoded paths, making reproduction difficult. The model architecture and loss function are well-structured and potentially reusable.

RATING: 5.8/10 REPRO: 3.0/10 MAINT: stale DRIFT: medium
▸ 4 reusable components found
── end of results ──
$ submit: