Senior AI Engineer (Voice Systems)

Senior AI Exprt - Voice Bot Systems

· Work Location: o Hanoi: TechnoPark Tower, Vinhomes Ocean Park, Hanoi

o Hochiminh: Vincom Dong Khoi, HCM

Job overview

We are looking for an AI Expert to join our team as the technical anchor for next-generation real-time voice bot systems. While our Software Engineers focus on integration and infrastructure, this role owns the intelligence layer - designing, training, evaluating, and optimizing the AI models that make voice interactions actually work: from acoustic understanding to natural speech synthesis, speaker identification, and language comprehension in noisy, real-world conditions.

The ideal candidate has built voice AI systems in production - not just integrated APIs, but understood what's happening inside them and made deliberate choices about model architecture, training data, evaluation methodology, and latency trade-offs. You will work directly with Software Engineers to move models from research into production and with product stakeholders to define what "good" means for a voice bot.

Key Responsibilities:

Voice AI Research & Development

Design and develop end-to-end voice pipelines: ASR → NLU → dialogue management → TTS, with full understanding of the trade-offs at each stage Research, evaluate, and fine-tune models for Vietnamese-language voice tasks: ASR accuracy on domain-specific vocabulary (vehicle names, locations, plate numbers), TTS naturalness, speaker diarization in multi-party calls Build and maintain evaluation frameworks for voice AI quality: Word Error Rate (WER), Real-Time Factor (RTF), MOS scoring, speaker turn accuracy, end-to-end latency benchmarks Model Fine-tuning & Training

Fine-tune ASR models (Whisper, Gipformer, wav2vec 2.0) on domain-specific and accent-specific Vietnamese data Fine-tune TTS models (F5-TTS, VoxCPM2, VITS) to produce natural, brand-consistent voice output for CSKH scenarios Build and curate training datasets: audio collection pipelines, annotation tooling, data quality validation Apply speaker diarization and voice activity detection (VAD) to segment and attribute multi-speaker call audio LLM & NLU Integration

Design prompt strategies and context management for LLM-driven dialogue in voice-specific constraints (no markdown, short sentences, natural prosody cues) Develop intent classification, entity extraction, and slot-filling components tuned for spoken Vietnamese, including noisy transcripts from imperfect ASR Evaluate and mitigate hallucination in voice LLM pipelines, including ASR-grounded prompting and transcript correction techniques Production & Optimization

Optimize inference latency for real-time constraints: quantization, batching, streaming inference, model distillation Design and run A/B experiments on model variants in production; define statistical evaluation criteria for voice quality metrics Collaborate with Software Engineers on model serving architecture (GPU inference servers, ONNX runtime, TorchServe, Triton) Build monitoring pipelines for model performance drift, ASR degradation, and TTS quality regression in production

Qualifications:

Education

Bachelor's or Master's degree in Computer Science, Electrical Engineering, Applied Mathematics, Linguistics, or a directly related field. PhD is a plus for research-heavy responsibilities. Hands-on voice AI experience (required)

3–5+ years of hands-on experience building and deploying voice AI systems - not just integrating cloud APIs, but working with the models themselves Demonstrated experience with at least two of: ASR fine-tuning, TTS model training, speaker diarization, VAD, voice conversion, or acoustic modeling Prior production experience with real-time voice pipelines where latency, reliability, and audio quality were hard constraints Deep learning & NLP foundations

Solid understanding of sequence modeling (Transformer, CTC, RNN), attention mechanisms, and how they apply to speech and language tasks Familiarity with acoustic features (MFCCs, mel spectrograms, filterbanks) and audio signal processing fundamentals Experience with speech-specific deep learning frameworks: ESPnet, SpeechBrain, NeMo, Hugging Face transformers + datasets for audio Engineering skills

Strong Python; comfortable with PyTorch or TensorFlow for model development and fine-tuning Ability to write production-quality code, not just research scripts: modular, testable, observable Experience with ML experiment tracking (MLflow, W&B), model versioning, and reproducibility practices Familiarity with Docker and GPU infrastructure for model serving Evaluation mindset

Knows that a model is only as good as its evaluation - can design rigorous offline and online evaluation protocols for voice AI Experience with human evaluation studies (MOS, MUSHRA, preference tests) and automated proxies ________________________________________

Nice to Have

Experience with multilingual or low-resource language ASR/TTS, especially Vietnamese Familiarity with speaker diarization systems (pyannote, NeMo MSDD) applied to call center audio Knowledge of telephony audio constraints: G.711 µ-law/a-law codecs, 8kHz narrowband limitations, packet loss concealment Experience with streaming ASR (chunk-based inference, endpointing) for real-time applications Contributions to open-source speech/NLP projects or published research in voice AI

What This Role Is Not

This is not an API integration role. We need someone who understands what happens inside the model, not just how to call it. This is not a pure research role. Models need to ship into production voice systems with real latency and reliability requirements. This is not a data annotation role. You will design the data strategy and evaluation criteria - others will execute annotation at scale. What We Offer

Ownership of the AI layer in a production voice system serving real customers Direct collaboration with Software Engineers, DevOps, and product teams - short feedback loops between research and production Access to real call audio data for training and evaluation (with appropriate privacy controls) Competitive compensation with equity