Phonetic information is one of the most essential components of a speech signal, playing an important role for many speech processing tasks. However, it is difficult to integrate phonetic information into speaker verification systems since it occurs primarily at the frame level while speaker characteristics typically reside at the segment level. In deep neural network-based speaker verification, existing methods only apply phonetic information to the frame-wise trained speaker embeddings. To improve this weakness, this paper proposes phonetic adaptation and hybrid multi-task learning and further combines these into c-vector and simplified c-vector architectures. Experiments on National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) 2010 show that the four proposed speaker embeddings achieve better performance than the baseline. The c-vector system performs the best, providing over 30% and 15% relative improvements in equal error rate (EER) for the core-extended and 10 s–10 s conditions, respectively. On the NIST SRE 2016, 2018, and VoxCeleb datasets, the proposed c-vector approach improves the performance even when there is a language mismatch within the training sets or between the training and evaluation sets. Extensive experimental results demonstrate the effectiveness and robustness of the proposed methods.
Digital Object Identifier (DOI)
This work was supported by the National Natural Science Foundation of China under grant no. 61403224 and no. U1836219.
The Switchboard, Fisher English and all the NIST SRE datasets are available from Linguistic Data Consortium (LDC, https://www.ldc.upenn.edu/).
The VoxCeleb 1&2 are available from http://www.robots.ox.ac.uk/~vgg/data/voxceleb/.
The Librispeech corpus and the noise and reverberation datasets, MUSAN and RIR, can be downloaded from http://www.openslr.org/resources.php.
The source code has been published in https://github.com/mycrazycracy/speaker-embedding-with-phonetic-information.
Liu, Yi; He, Liang; and Johnson, Michael T., "Introducing Phonetic Information to Speaker Embedding for Speaker Verification" (2019). Electrical and Computer Engineering Faculty Publications. 37.