Training a StyleTTS2 Model¶
This page picks up from Step 5: Choose a Model in the main guide. If you haven't completed Steps 1–4 in the main guide yet, start there first.
Step 1: Fetch Pretrained Models¶
StyleTTS2 uses several pretrained components — an F0 extractor, an ASR aligner, a PLBERT language model, and a WavLM model — that are downloaded from HuggingFace before training begins. If you are on a cluster where your GPU nodes don't have internet access, you'll want to run this step on a login node first so the files are cached and available when training starts.
everyvoice fetch-pretrained text-to-wav config/everyvoice-text-to-wav.yaml
If your GPU nodes do have internet access, you can skip this step and the files will be downloaded automatically when training starts.
Step 2: Run the Preprocessor¶
Your model needs to do a number of preprocessing steps in order to prepare for training. To preprocess everything you need, run the following:
everyvoice preprocess text-to-wav config/everyvoice-text-to-wav.yaml
Important
StyleTTS2 training requires a GPU with CUDA. Make sure you are running the following steps on a machine with a compatible GPU.
Step 2a: Inspect your symbol set¶
If loading your configuration failed with a pydantic.ValidationError mentioning you have some symbols that "are not present in the pretrained StyleTTS2 text-encoder symbol table", it's due to symbols in your data that are incompatible with StyleTTS2 — but it can be fixed! This can happen on any command that loads your text-to-wav config, including everyvoice preprocess above. If you didn't get this error, please proceed to Step 3.
Unlike EveryVoice's other models, StyleTTS2 uses a pretrained text encoder. That encoder has a fixed embedding table for a specific set of 178 symbols (letters, IPA phones, and punctuation) that it saw during training, and it cannot produce a meaningful embedding for a symbol it has never seen. Because of this, every symbol declared in your text configuration's symbols must also be a member of that pretrained set.
If your configuration declares a symbol outside that set, EveryVoice will refuse to load the configuration rather than silently proceeding to train a broken model, e.g.:
Value error, The following symbols declared in your TextConfig are not present in
the pretrained StyleTTS2 text-encoder symbol table: {'Ê’'}. Change them in your text,
remove them from your TextConfig, or run 'everyvoice check pretrained-symbols
--config <this config>' to get suggested 'to_replace' substitutions for the closest
pretrained symbols.
This check runs on every config load, so it only tells you which symbols are the problem. Finding a replacement for each one is a comparatively slow search, so it's kept out of this fast path and left to a command you run on demand:
everyvoice check pretrained-symbols --config config/everyvoice-text-to-wav.yaml
For each problem symbol, this prints the closest pretrained symbol it found (using panphon's articulatory-feature distance for IPA phones, edit distance for multigraphs, and a last-resort Unicode distance measure otherwise), along with a distance score, formatted as a ready-to-paste to_replace block, e.g.:
The following symbols declared in your text config are not present in the
pretrained text-encoder symbol table: ['Ê’']
Suggested substitutions — copy into your text config's 'to_replace':
to_replace:
'ʒ': 'ʃ' # distance=0.25
To fix your model, copy the suggested block into your text configuration, e.g.:
text:
to_replace:
"ʒ": "ʃ"
Note
A suggested substitution is a starting point, always sanity-check that the suggested symbol is phonetically reasonable for your language before adopting it, while also making sure that you are not mapping multiple symbols to the same symbol. Some symbols may have no close match at all, so the mappings might look fairly bad. My experience is that the model is still able to recover from a few of these bad mappings.
Step 3: Train your Model¶
StyleTTS2 training is split into two stages that must be run in order.
Stage 1
everyvoice train text-to-wav config/everyvoice-text-to-wav.yaml --mode first
Stage 2 Once Stage 1 has finished, run:
everyvoice train text-to-wav config/everyvoice-text-to-wav.yaml --mode second
Tip
While your model is training, you can use TensorBoard to view the logs, which will show information about the progress of training. To use TensorBoard, make sure that your conda environment is activated and run tensorboard --logdir path/to/logs_and_checkpoints. Then your logs will be viewable at http://localhost:6006.
By default, training uses PyTorch Lightning's "auto" strategy. If you are on a machine where you know the hardware, you can specify it:
everyvoice train text-to-wav config/everyvoice-text-to-wav.yaml --mode first -d 4 -a gpu
Which would use the GPU accelerator (-a gpu) and specify 4 devices/chips (-d 4).
Step 4: Synthesize Speech in Your Language!¶
StyleTTS2 generates speech by sampling a speaking style from a short reference audio clip. The reference audio should be a clean recording of a few seconds, but it doesn't need to match the text you want to synthesize. You can use any recording from your training data as a reference.
everyvoice synthesize text-to-wav logs_and_checkpoints/E2E-Experiment/base/stage-2-last.ckpt \
--reference path/to/reference.wav \
--text "your text here"
You can pass --text multiple times to synthesize several utterances at once:
everyvoice synthesize text-to-wav logs_and_checkpoints/E2E-Experiment/base/stage-2-last.ckpt \
--reference path/to/reference.wav \
--text "First sentence." \
--text "Second sentence."
Optional: Multilingual Training¶
StyleTTS2 can be trained on a mix of languages at once using a learned language embedding, based on the method described in Wang, Pine & Geng (2026). This is useful for combining a low-resource language with a higher-resource one (e.g. English) in a single model, or for doing cross-lingual speaker transfer for the purposes of anonymization (read Wang, Pine & Geng (2026) for more information).
To enable it, set the following in your config:
model:
multilingual: true
language_embedding_dim: 64 # optional, defaults to 64
Every row in your training and validation filelists must have a language value; the set of distinct languages found there becomes the model's language table automatically — there's nothing else to configure.
Both characters and ipa_phones (model.target_text_representation_level) work for multilingual training.
At synthesis time, pass --language to select which language embedding to condition on:
everyvoice synthesize text-to-wav logs_and_checkpoints/E2E-Experiment/base/stage-2-last.ckpt \
--reference path/to/reference.wav \
--text "your text here" \
--language eng
The value must match a language seen during training. The same flag is available in everyvoice demo, where it also shows a language dropdown in the UI for multilingual checkpoints.
Optional: Evaluation¶
If you want to evaluate the model you just built, you can make use of the everyvoice evaluate command. In order to use it, you have to first generate some audio (see Step 5) and then you can evaluate either a single file with everyvoice evaluate -f your_file.wav or a directory of audio files with everyvoice evaluate -d path_to_wavs/. This will report predictions for three metrics: Wideband Perceptual Estimation of Speech Quality (PESQ), Short-Time Objective Intelligibility (STOI), and Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) using the model described in this paper. You can also provide a non-matching reference to predict a Mean Opinion Score (MOS) for your generated audio: everyvoice evaluate -d path_to_wavs/ -r path_to_reference.wav. The reference should be a path to non-generated, good quality audio but it doesn't need to match the exact utterance that was generated.
Please refer to everyvoice evaluate --help for more information.
Note
Automatic evaluation can be helpful, but please take the reported numbers with a grain of salt. They are not always reliable, and do not always correlate well with human judgements.