Senior Software Engineer - AI Interaction Evaluator (Codex / Claude Code)

About the role

We are seeking a senior‑level software engineer to evaluate the quality of interactions with modern AI coding agents such as OpenAI Codex and Claude Code. This short‑term contract focuses on assessing how well the model mimics the reasoning and judgment of an experienced developer, rather than writing production code.

Key responsibilities

• Review end‑to‑end AI‑generated coding interactions and judge their usefulness, correctness, and alignment with strong engineering judgment.
• Provide clear, opinionated feedback on what works, what feels off, and how explanations can be improved.
• Distinguish between different quality levels of responses and help define standards for “great” AI‑assisted coding experiences.
• Collaborate with the team to shape evaluation criteria and contribute to the development of AI‑first IDE tools.

Required profile

• Staff‑ or principal‑level engineer (or equivalent experience) with a high bar for good engineering.
• Strong background in TypeScript/JavaScript or Python.
• Hands‑on experience using OpenAI Codex, Claude Code, and AI‑assisted development workflows.
• Comfortable making rigorous, subjective judgments about code quality and explanations.

Required skills

• TypeScript
• JavaScript
• Python
• OpenAI Codex
• Claude Code
• Cursor (or similar AI‑first IDE)

What we offer

• Remote contract, 10–20 hours per week.
• Compensation up to $200 USD per hour for US/Canada, $150 USD for EU/Latam, $100 USD for other locations.
• Engagement through early May with possible extension.
• Take‑home evaluation exercise and one behavioral interview.