Sr. QA Specialist (Conversational AI Testing)

Senior QA EngineerWe are seeking an experienced Senior QA Engineer to lead quality assurance efforts for our conversational AI applications. This role requires deep expertise in testing AI-driven systems, prompt engineering validation, and test automation. You will be responsible for designing comprehensive test strategies, identifying edge cases in AI behavior, and ensuring our conversational AI products meet the highest standards of reliability, accuracy, and security.Key ResponsibilitiesAI Application Testing: Design and execute test plans specifically for conversational AI systems, including chatbots, virtual assistants, and language model-based applicationsPrompt Engineering Validation: Collaborate with AI/ML teams to test and validate prompt effectiveness, consistency, and safety across various user scenariosTest Automation Development: Build and maintain automated test suites for conversational AI workflows, including API testing, response validation, and regression testingQuality Strategy: Develop QA strategies that address unique challenges of AI testing, including non-deterministic outputs, hallucination detection, and bias identificationSecurity & Compliance: Ensure testing aligns with organizational security standards, including HIPAA compliance (Standard 4500), information protection requirements (Standard 5110), and access control protocolsDocumentation & Reporting: Create detailed test cases, defect reports, and quality metrics; communicate findings to stakeholdersCross-functional Collaboration: Work with product, engineering, and AI/ML teams to understand requirements and validate solutionsRequired Qualifications5+ years of QA experience, with at least 2+ years testing AI/ML-based applications or conversational systemsPrompt Engineering Expertise: Demonstrated experience designing, testing, and validating prompts for language models; understanding of prompt injection, jailbreaking, and safety considerationsTest Automation Proficiency: Strong hands-on experience with test automation frameworks and tools (e.g., Selenium, Cypress, Postman, REST Assured, or equivalent); ability to write clean, maintainable automation codeTechnical Skills: Proficiency in at least one programming language (Python, Java, JavaScript, or similar); familiarity with API testing and CI/CD pipelinesAI/Conversational AI Knowledge: Understanding of how conversational AI systems work, including LLMs, NLP concepts, and common failure modesAnalytical Mindset: Strong problem-solving skills with ability to identify edge cases and non-obvious failure scenarios in AI systemsCommunication: Excellent written and verbal communication skills; ability to explain technical QA concepts to non-technical stakeholdersPreferred QualificationsExperience testing multiple AI platforms (OpenAI, Google Vertex AI, Azure OpenAI, Anthropic Claude, etc.)Familiarity with AI safety testing frameworks and bias detection methodologiesExperience with performance and load testing for AI applicationsKnowledge of HIPAA, SOC 2, or other healthcare/regulated industry compliance requirementsCertification in QA, test automation, or related fieldExperience with agile/scrum methodologiesBackground in healthcare, financial services, or other regulated industriesTechnical CompetenciesTest case design and executionTest automation and scriptingAPI and integration testingPerformance and load testingDefect tracking and reportingSQL and database query basicsVersion control systems (Git)Prompt engineering and LLM interactionSecurity and compliance testingRequired Experience5-8 years of professional experience in an IT related field preferably in a development or testing environmentRequired EducationBachelor's Degree in an IT related field or relevant work experienceEEO: "Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of – Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans."