Tan Lee
YOU?
Author Swipe
View article: Cross-cultural dimensions organizing prosodic attitudes reception
Cross-cultural dimensions organizing prosodic attitudes reception Open
We present a meta-analysis of results from experimental studies on attitude reception in seven languages (Brazilian Portuguese, Japanese, French, German, Cantonese, American English, Hindi). The studies involved free-labeling of perceived …
View article: PodEval: A Multimodal Evaluation Framework for Podcast Audio Generation
PodEval: A Multimodal Evaluation Framework for Podcast Audio Generation Open
Recently, an increasing number of multimodal (text and audio) benchmarks have emerged, primarily focusing on evaluating models' understanding capability. However, exploration into assessing generative capabilities remains limited, especial…
View article: An automated extraction of spectral-temporal and spatial-temporal features of EEG for emotion detection
An automated extraction of spectral-temporal and spatial-temporal features of EEG for emotion detection Open
Emotion is an integral part of human cognitive processes and behaviors. Automatic detection and classification of human emotion has been a goal of applied research. This study presents an approach to detecting emotion from multivariate ele…
View article: Deviant functional connectivity patterns in the EEG related to developmental dyslexia and their potential use for screening
Deviant functional connectivity patterns in the EEG related to developmental dyslexia and their potential use for screening Open
Developmental dyslexia (DD) is a common learning disorder with potential neural origins. While EEG-based brain activation measures combined with machine learning have shown promise for DD screening, these approaches often lack validation o…
View article: PodAgent: A Comprehensive Framework for Podcast Generation
PodAgent: A Comprehensive Framework for Podcast Generation Open
Existing Existing automatic audio generation methods struggle to generate podcast-like audio programs effectively. The key challenges lie in in-depth content generation, appropriate and expressive voice production. This paper proposed PodA…
View article: A Large-Scale Probing Analysis of Speaker-Specific Attributes in Self-Supervised Speech Representations
A Large-Scale Probing Analysis of Speaker-Specific Attributes in Self-Supervised Speech Representations Open
Speech self-supervised learning (SSL) models are known to learn hierarchical representations, yet how they encode different speaker-specific attributes remains under-explored. This study investigates the layer-wise disentanglement of speak…
View article: An Investigation of Reprogramming for Cross-Language Adaptation in Speaker Verification Systems
An Investigation of Reprogramming for Cross-Language Adaptation in Speaker Verification Systems Open
Language mismatch is among the most common and challenging domain mismatches in deploying speaker verification (SV) systems. Adversarial reprogramming has shown promising results in cross-language adaptation for SV. The reprogramming is im…
View article: CUEMPATHY: A Counseling Speech Dataset for Psychotherapy Research
CUEMPATHY: A Counseling Speech Dataset for Psychotherapy Research Open
Psychotherapy or counseling is typically conducted through spoken conversation between a therapist and a client. Analyzing the speech characteristics of psychotherapeutic interactions can help understand the factors associated with effecti…
View article: Personalized Voice Synthesis through Human-in-the-Loop Coordinate Descent
Personalized Voice Synthesis through Human-in-the-Loop Coordinate Descent Open
This paper describes a human-in-the-loop approach to personalized voice synthesis in the absence of reference speech data from the target speaker. It is intended to help vocally disabled individuals restore their lost voices without requir…
View article: ToneUnit: A Speech Discretization Approach for Tonal Language Speech Synthesis
ToneUnit: A Speech Discretization Approach for Tonal Language Speech Synthesis Open
Representing speech as discretized units has numerous benefits in supporting downstream spoken language processing tasks. However, the approach has been less explored in speech synthesis of tonal languages like Mandarin Chinese. Our prelim…
View article: A Parameter-efficient Language Extension Framework for Multilingual ASR
A Parameter-efficient Language Extension Framework for Multilingual ASR Open
Covering all languages with a multilingual speech recognition model (MASR) is very difficult. Performing language extension on top of an existing MASR is a desirable choice. In this study, the MASR continual learning problem is probabilist…
View article: Creating Personalized Synthetic Voices from Articulation Impaired Speech Using Augmented Reconstruction Loss
Creating Personalized Synthetic Voices from Articulation Impaired Speech Using Augmented Reconstruction Loss Open
This research is about the creation of personalized synthetic voices for head and neck cancer survivors. It is focused particularly on tongue cancer patients whose speech might exhibit severe articulation impairment. Our goal is to restore…
View article: LUPET: Incorporating Hierarchical Information Path into Multilingual ASR
LUPET: Incorporating Hierarchical Information Path into Multilingual ASR Open
Toward high-performance multilingual automatic speech recognition (ASR), various types of linguistic information and model design have demonstrated their effectiveness independently. They include language identity (LID), phoneme informatio…
View article: Modeling Intrapersonal and Interpersonal Influences for Automatic Estimation of Therapist Empathy in Counseling Conversation
Modeling Intrapersonal and Interpersonal Influences for Automatic Estimation of Therapist Empathy in Counseling Conversation Open
Counseling is usually conducted through spoken conversation between a therapist and a client. The empathy level of therapist is a key indicator of outcomes. Presuming that therapist's empathy expression is shaped by their past behavior and…
View article: A Study on Prosodic Entrainment in Relation to Therapist Empathy in Counseling Conversation
A Study on Prosodic Entrainment in Relation to Therapist Empathy in Counseling Conversation Open
Counseling is carried out as spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered an important index of the quality of counseling and often assessed by an observer or the client. …
View article: Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning
Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning Open
The development of deep neural networks (DNN) has significantly enhanced the performance of speaker verification (SV) systems in recent years. However, a critical issue that persists when applying DNN-based SV systems in practical applicat…
View article: CoMFLP: Correlation Measure based Fast Search on ASR Layer Pruning
CoMFLP: Correlation Measure based Fast Search on ASR Layer Pruning Open
Transformer-based speech recognition (ASR) model with deep layers exhibited significant performance improvement. However, the model is inefficient for deployment on resource-constrained devices. Layer pruning (LP) is a commonly used compre…
View article: Sparsely Shared LoRA on Whisper for Child Speech Recognition
Sparsely Shared LoRA on Whisper for Child Speech Recognition Open
Whisper is a powerful automatic speech recognition (ASR) model. Nevertheless, its zero-shot performance on low-resource speech requires further improvement. Child speech, as a representative type of low-resource speech, is leveraged for ad…
View article: ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading
ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading Open
While state-of-the-art Text-to-Speech systems can generate natural speech of very high quality at sentence level, they still meet great challenges in speech generation for paragraph / long-form reading. Such deficiencies are due to i) igno…
View article: Creating Personalized Synthetic Voices from Post-Glossectomy Speech with Guided Diffusion Models
Creating Personalized Synthetic Voices from Post-Glossectomy Speech with Guided Diffusion Models Open
This paper is about developing personalized speech synthesis systems with recordings of mildly impaired speech. In particular, we consider consonant and vowel alterations resulted from partial glossectomy, the surgical removal of part of t…
View article: Learning Representation of Therapist Empathy in Counseling Conversation Using Siamese Hierarchical Attention Network
Learning Representation of Therapist Empathy in Counseling Conversation Using Siamese Hierarchical Attention Network Open
Counseling is an activity of conversational speaking between a therapist and a client. Therapist empathy is an essential indicator of counseling quality and assessed subjectively by considering the entire conversation. This paper proposes …
View article: Diffusion-Based Mel-Spectrogram Enhancement for Personalized Speech Synthesis with Found Data
Diffusion-Based Mel-Spectrogram Enhancement for Personalized Speech Synthesis with Found Data Open
Creating synthetic voices with found data is challenging, as real-world recordings often contain various types of audio degradation. One way to address this problem is to pre-enhance the speech with an enhancement model and then use the en…
View article: Leveraging phone-level linguistic-acoustic similarity for utterance-level pronunciation scoring
Leveraging phone-level linguistic-acoustic similarity for utterance-level pronunciation scoring Open
Recent studies on pronunciation scoring have explored the effect of introducing phone embeddings as reference pronunciation, but mostly in an implicit manner, i.e., addition or concatenation of reference phone embedding and actual pronunci…
View article: An ASR-free Fluency Scoring Approach with Self-Supervised Learning
An ASR-free Fluency Scoring Approach with Self-Supervised Learning Open
A typical fluency scoring system generally relies on an automatic speech recognition (ASR) system to obtain time stamps in input speech for either the subsequent calculation of fluency-related features or directly modeling speech fluency w…
View article: Formality in psychotherapy: How are therapists’ and clients’ use of discourse particles related to therapist empathy?
Formality in psychotherapy: How are therapists’ and clients’ use of discourse particles related to therapist empathy? Open
Introduction Previous studies explored the preferences for therapists’ attire and office setting based on initial impressions as a reference for the formality in psychotherapy. This study examines the formality of psychotherapy by investig…
View article: Covariance Regularization for Probabilistic Linear Discriminant Analysis
Covariance Regularization for Probabilistic Linear Discriminant Analysis Open
Probabilistic linear discriminant analysis (PLDA) is commonly used in speaker verification systems to score the similarity of speaker embeddings. Recent studies improved the performance of PLDA in domain-matched conditions by diagonalizing…
View article: Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition
Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition Open
Very deep models for speaker recognition (SR) have demonstrated remarkable performance improvement in recent research. However, it is impractical to deploy these models for on-device applications with constrained computational resources. O…
View article: Model Compression for DNN-based Speaker Verification Using Weight Quantization
Model Compression for DNN-based Speaker Verification Using Weight Quantization Open
DNN-based speaker verification (SV) models demonstrate significant performance at relatively high computation costs. Model compression can be applied to reduce the model size for lower resource consumption. The present study exploits weigh…