Scientific Evals

About us

Edison Scientific builds and deploys AI scientist agents to accelerate science and the development of new medicines. We are an ambitious team run by scientists and engineers from leading institutions across biology, physics, chemistry, and AI.

The role

We are seeking an ambitious, scientifically grounded person to join our team focused on developing rigorous evaluation frameworks, training data, and approaches for advancing AI capabilities in biology. This role sits at the intersection of scientific research, data, and machine learning, and is ideal for someone with deep scientific training who is excited to shape how frontier AI systems accelerate discovery.

Well-designed benchmarks are key to measuring abilities and charting progress for AI development. We have been defining the frontier of AI benchmarking for biology research since the first AI scientist agents were introduced, and are looking to bring people on board who want to help us continue pushing that frontier. In this role, you will:

• Design and build novel frontier benchmarks for measuring AI agent-driven science.

• Develop entirely new evaluation approaches that capture the realities of scientific research, drawing on your expertise as a scientist to translate the discovery process into evaluable agent tasks.

• Analyze model or agent work, identify failure modes, and directly contribute to iterative improvements to agents, evaluation tasks, and grading approaches.

• Collaborate with AI/ML researchers and engineers to translate scientific taste into training signal and agent improvements.

• Identify and rigorously curate important biological data that can be used for improving scientific agents.

• Coordinate data collection operations and manage workflows, including working with domain experts, tracking project progress, while maintaining visibility and thorough documentation.

Requirements:

• Graduate-level training in biology, computational biology, or a related field, with extensive hands-on research experience. We are particularly interested in people with both wet and dry lab experience.

• Understand or have working familiarity with benchmarking and evaluating AI systems.

• Energized by and willing to own ambitious, open-ended projects that require creativity, collaboration, and first-principles thinking to solve.

• Comfortable with Python and can build workflows for data processing, analysis, and experimentation.

• Thorough experience working with AI models and tools, including coding agents or Edison’s own agent platform, especially in the context of scientific research.

• Possess strong scientific taste/intuition and can articulate this into actionable and evaluable workflows.

• Organized and communicative, able to manage multiple workstreams and coordinate across teams.

• Detail-oriented and willing to take on high-value but occasionally tedious work.

Bonus Points for:

• Prior experience creating evaluation datasets, annotation guidelines, or working on human-in-the-loop data pipelines.

• Hands-on experience fine-tuning or evaluating large language models, or familiarity with RLHF and preference-based training.

• Publications or research experience in areas relevant to AI for science.

Why join us?

We're a fast-moving, mission-driven culture where smart people do their best work and actually enjoy doing it. We also offer full-time employees the following:

• Competitive salary and equity

• Full healthcare coverage; we pay 100% of premiums for you and your dependents

• Support for growing families, including a yearly new parent stipend and fertility coverage through Carrot

• Mental health support through Rula, our in-network therapist and psychiatrist network with fast availability

• 12 weeks of paid parental leave for maternity, paternity, and adoption

• Pet care support with a yearly employer-funded stipend for your animal companions

• Commuter benefits so you can pay for transit and parking with pre-tax dollars

• 401(k) company matching

• $300 health and wellness benefit quarterly

• Lunch is on us every day you're in the office, and dinner is on us when you're working late

• Regular team off-sites and company events

Edison Scientific is an equal opportunity employer. We do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other protected characteristic under applicable law.