Location: San Francisco preferred, remote considered
Duration: 10-12 weeks, Summer 2026
Compensation: Paid internship
Preference Model is building the next generation of training data to power the future of AI. Today's models are powerful but fail to reach their potential across diverse use cases because so many of the tasks that we want to use these models are out of distribution. Preference Model creates RL environments where models encounter research and engineering problems, iterate, and learn from realistic feedback loops.
Our founding team has previous experience on Anthropic’s data team building data infrastructure, tokenizers, and datasets behind the Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential. We are backed by a16z.
We're looking for PhD students and gifted undergrads to spend the summer building RL training environments for large language models.
Design and build RL environments that test LLM reasoning on ML, systems, and research problems
Write clean, production-grade Python (not notebooks)
Work with Docker, build reproducible environments, debug when things break
Translate ML papers and concepts into concrete training tasks
You're an undergrad or PhD student in CS, ML, math, physics, or a related field. You write real code, not just research prototypes. You read ML papers for fun in your free time.
Strong Python skills
Familiarity with how LLMs work, what they're good at, and where they fall short
Ability to work independently, take feedback, and iterate fast
You understand transformer internals and have worked with training or inference code
You've written CUDA kernels or worked with low-level GPU programming
You have a research area you know deeply (publications, public code, or strong coursework)
You read broadly across ML and can connect ideas from different subfields
You've built interactive environments, simulations, or complex software systems
Send your resume and a short note (2-3 sentences is fine) about what area of ML you're most interested in and why. Links to code, papers, or projects are more useful than a long cover letter.
Preference%20model
https://preference%20model.com