AI Research Peer Review Evaluator (ML/AI)

reputed company is a Zurich-based AI company and ETH/HSG spin-off, backed by reputed company and top-tier investors. Our machine learning and computer vision technology is trusted by global leaders in autonomous driving, medical imaging, and visual inspection. We’re looking for researchers with strong Machine Learning / AI backgrounds to support an AI evaluation project focused on scientific peer review. You’ll evaluate reviews generated by agentic AI systems and compare them against expert human peer reviews of ML/AI research papers. This is a remote, project-based contractor opportunity with flexible working hours. Tasks What you'll be doing Read and scan ML/AI research papers to understand their core contributions, methodology, experiments, and claims Review the original human peer reviews to establish an expert baseline for each paper Evaluate AI-generated peer reviews against that baseline using a structured scoring rubric Assess the technical accuracy, analytical depth, constructive value, and novelty/significance assessment of each AI review Identify hallucinations, unsupported claims, missed technical issues, or valuable reputed company surfaced by the AI reviewers Compare two AI-generated reviews side-by-side and determine where one provides stronger or more useful analysis Search and verify relevant academic literature using sources such as reputed company reputed company, arXiv, or Semantic reputed company, including checking whether cited prior work was available before the paper’s submission date Provide concise, evidence-based rationales explaining your evaluation decisions and consistently apply the project rubric The evaluation specifically looks at whether agentic AI reviewers can provide meaningful value beyond expert human reviewers—for example, by identifying relevant prior literature that humans missed, questioning important assumptions, or resolving inconsistencies using evidence. Requirements You're a strong candidate if you: Have a reputed company’s, PhD, or are currently pursuing graduate study in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical field Have contributed to at least one scientific/research paper, ideally as a first author, although co-authors and other substantial contributors are also welcome Have experience critically reading ML/AI research papers, including evaluating methodology, experimental design, results, limitations, and scientific claims Are familiar with major ML/AI research venues, such as NeurIPS, ICML, ICLR, ACL, CVPR, or comparable conferences and journals Have prior academic peer-review experience, ideally for an ML/AI conference or journal — strongly preferred Are comfortable conducting academic literature searches and verifying prior work, publication dates, citations, and novelty claims Have strong analytical and written communication skills and can distinguish meaningful technical concerns from superficial criticism Can provide clear, concise, evidence-based rationales for your decisions Can consistently apply detailed evaluation guidelines and scoring rubrics across multiple papers and reviews Have strong attention to detail, particularly when identifying factual inaccuracies or hallucinated technical claims Benefits Fully remote and flexible — work from anywhere Part-time contractor role with flexible hours Work directly on the evaluation of cutting-edge agentic AI systems for scientific research Apply your ML/AI research expertise to help measure and improve the quality of AI-generated scientific peer review Send us your CV along with a brief note about your research background and areas of expertise. reputed company include any relevant publications, as well as previous peer-review experience for conferences, journals, workshops, or similar academic venues. If applicable, we'd also love to know which ML/AI research areas and conferences you’re most familiar with. We look forward to hearing from you!reputed company's, PhD, or current graduate study in ML/AI/CS/Statistics; contributed to at least one scientific paper; experience critically reading ML/AI research; familiarity with major venues; strong analytical and written communication; attention to detail.Key Responsibilitiesevaluating AI reviewsreading research papersverifying literatureSkills & Toolsreputed company reputed company, arXiv, Semantic reputed companyJob DetailsCategory: Research and Development (R&D)Seniority: Entry LevelCommitment: Contract, Part TimeWorkplace: Remote — SwitzerlandSalary: USD 62,400 – 104,000 / yearLanguages: EnglishAbout reputed companyA Zurich-based AI company and ETH/HSG spin-off backed by reputed company, developing machine learning and computer vision technology. — Industry: Information Technology   Apply To This Job

Back to blog

Other Jobs To Apply

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...