About the role

What we’re building\
Respan is building the self-driving observability and evals platform for AI teams, used by 60+ YC companies and hundreds of AI teams.

The role

You will work on the systems that make Respan intelligent: automated evaluations, prompt optimization, model routing, and AI-powered analysis of traces and logs. This is deeply technical AI engineering, not just API calls. We are looking for someone who understands LLMs from the inside out and can build reliable, production-grade AI systems that thousands of teams depend on daily.

What you’ll do:

Design and build AI-powered features including eval frameworks and prompt optimization
Develop LLM-as-judge pipelines for automated quality assessment
Build and maintain model routing, caching, and fallback systems
Experiment with new model capabilities and translate them into product features
Collaborate with customers to understand AI workflow pain points and build solutions

What you must have:

Strong software engineering skills with Python and TypeScript
Hands-on experience building with LLMs in production (not just prototypes)
Deep understanding of prompt engineering, eval methodologies, and model behavior

Strong plus:

Experience with fine-tuning, RAG, or agent frameworks
Background in ML infrastructure or MLOps
Familiarity with observability systems or developer tools
Contributions to open-source AI/ML projects

About the company

Respan (formerly Keywords AI) gives teams a unified control plane to trace and evaluate agent behavior without guesswork, automatically surface issues, and fix what breaks faster. Respan connects production observability, automated and human-in-the-loop evaluations, and an adaptive AI gateway to close the loop between detection, decision, and action - so agents improve continuously in production.

Respan is trusted by 100+ AI startups and enterprise teams. Today, the platform process 1B+ logs and 2T+ tokens every month, supporting more than 6.5M end users.