Agentic AI Engineer in Test
Santander Bank, N.A. is a global leader in financial services, evolving into a technology-driven organization. The Agentic AI Engineer will design, build, and deploy autonomous AI agents powered by large language models, collaborating with various teams to create intelligent systems for enterprise environments.
Responsibilities
- Design and implement agentic systems with planning, reasoning, memory, tool use, and multi-step workflows
- Build LLM-powered pipelines using frameworks such as LangChain, AutoGen, Llama Index, or custom runtimes
- Implement MCP (Model Context Protocol) or similar protocols for secure and governed tool/API access
- Integrate agents with APIs, microservices, workflows, distributed systems, and enterprise platforms
- Develop retrieval pipelines (RAG), embeddings, memory stores, and prompt strategies, frameworks so as LLM/RAG test automation solutions
- Implement observability, telemetry, and monitoring for agent decisions and tool interactions
- Build CI/CD, automation, and MLOps practices for agent deployment
Skills
- Bachelor's Degree or equivalent work experience: Computer Science, Software Engineering or equivalent field
- 0+ Years Experience with test strategies, test plans, and test cases
- Experience in writing SQLs to query the backend during the test execution
- Expertise in testing web based applications and web services based systems with minimal to no user interface
- Experience building test automation frameworks for AI/ML or distributed systems
- Understanding of LLM evaluation techniques (BLEU, factuality checks, grounding scores, custom metrics)
- Ability to debug agent reasoning errors, hallucinations, planning failures, and tool misuse
- Knowledge of testing RAG systems (retrieval correctness, latency, vector search quality)
- Experience with structured testing for tool calls, MCP tools, API integrations, and orchestration chains
- Strong skills in observability, log analysis, and test data management
- Ability to apply responsible AI and safety testing practices
- Experience integrating AI with internal tools, data sources, and enterprise services
- Cloud engineering experience (AWS/Azure/GCP) and containerization (Docker, Kubernetes)
- Understanding of AI governance, responsible AI, and compliance requirements
- Experience with JIRA/ALM/Xray or similar tools
- Strong analytical, communication, documentation, and problem-solving skills
- Ability to manage workload, collaborate in Agile teams, effectively, and work independently
- Established work history or equivalent demonstrated through a combination of work experience, training, military service, or education
- Experience in Microsoft Office products
Benefits
- Fair and competitive rewards package
- Benefits are designed to support you, your family and your well-being, now and into the future
Company Overview
Company H1B Sponsorship