kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimization

Introduction

Let's start with a real use case. Say we want to convert the following Spanish bass to the following Spanish soprano. They have high phoneme overlaps due to being parts of a choral ensemble, creating a nice testing environment for zero-shot SVC.

Here are the conversion results. None of our baselines and ablations have been trained on Spanish audio.

• knn-svc w/o {AS, OPT}: Sounds much duller than others, along with constant ringing.
• NeuCoSVC: Sometimes with glitches (e.g. 00:07). Also, notice that the soprano sounds thinner, likely influenced by its Mandarin corpus (OpenSinger) training.
• knn-svc vs knn-svc w/o {OPT}: Notice that trembling (e.g. around 00:12, 00:26, 00:33, 00:51) significantly decreases. This is the main symptom the smoothness optimization intends to treat.

Ablation and Model Comparisons

In this section, we provide samples for the ablation and model comparisons section (Table 1). "ls" refers to the LibriSpeech dataset. "os" refers to the OpenSinger dataset.

Duration Study

In this section, we provide examples for the duration study section (not in the paper due to length limit). The durations indicate those of the references.

Bonus

In this section, we play with knn-svc in various real use cases.