Description

About the Firm

We are a highly reputable mid-sized investment management firm applying advanced AI to improve how investment decisions are researched and executed. Our work centers on building systems that perform reliably in noisy, dynamic, real-world environments-not controlled benchmarks.

The Opportunity

Most AI systems break down outside clean demos. This role focuses on the harder layer underneath:

How do we engineer, validate, and stress-test agentic systems so they behave reliably over long horizons in uncertain environments?

You'll focus on harness engineering-the infrastructure, evaluation systems, and runtime scaffolding that make autonomous agents actually usable in production. This is less about isolated models and more about building the systems that prove they work .

What You'll Do

1. Design and build evaluation harnesses for long-running agent workflows (hours to days)
2. Develop infrastructure to simulate complex, noisy environments for agent testing
3. Create metrics and validation frameworks where success is ambiguous or delayed
4. Engineer agent runtime systems : orchestration, memory, retries, and failure handling
5. Build tools for observability, debugging, and introspection of agent behavior
6. Partner with researchers to translate prototypes into reliable, testable systems
7. Develop multi-agent test environments to evaluate coordination and failure modes
8. Continuously stress-test systems against edge cases, drift, and adversarial conditions

What We're Looking For

1. Senior/staff-level experience in AI/ML systems, infrastructure, or applied research
2. Strong background in systems engineering for AI , not just modeling
3. Experience with evaluation harnesses, benchmarking, or simulation frameworks
4. Familiarity with long-running, stateful agent systems and their failure modes
5. Ability to design robust testing strategies for non-deterministic systems
6. Track record of delivering in ambiguous, high-impact environments
7. Deep understanding of how models behave in production (not just in theory)
8. Comfort owning loosely defined problems end-to-end

Why This Role

1. Work on the hardest unsolved layer of agentic AI: making it reliable
2. Define how systems are evaluated-not just how they're built
3. Build infrastructure that directly impacts real-world decision-making
4. Operate in a focused, low-bureaucracy environment with rapid iteration
5. Tackle problems where correctness isn't obvious-and that's the point

Compensation

Competitive with senior/staff-level roles at leading technology companies, with performance-based upside.