Job Overview: We are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. This role involves hands-on software engineering work, including development environment automation, issue triaging, and evaluating test coverage and quality.
Duties and Responsibilities: Analyze and triage GitHub issues across trending open-source libraries. Set up and configure code repositories, including Dockerization and environment setup. Evaluating unit test coverage and quality. Modify and run codebases locally to assess LLM performance in bug-fixing scenarios. Collaborate with researchers to design and identify repositories and issues that are challenging for LLMs. Opportunities to lead a team of junior engineers to collaborate on projects.
Required Qualifications: Minimum 3+ years of overall experience. Strong experience with at least one of the following languages: Ruby. Proficiency with Git, Docker, and basic software pipeline setup. Ability to understand and navigate complex codebases. Comfortable running, modifying, and testing real-world projects locally. Experience contributing to or evaluating open-source projects is a plus.
Additional Notes: Work in a fully remote environment. Opportunity to work on cutting-edge AI projects with leading LLM companies. Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST. Employment type: Contractor assignment (no medical/paid leave).
Info
Job Posting Disclaimer
All job postings on this site are shared for informational purposes only. The responsibility for the accuracy of job descriptions, requirements, qualifications, and other details rests entirely with the employer or organization offering the position. We do not verify or guarantee the authenticity of these listings.
Applicants are encouraged to perform their own due diligence and confirm all information directly with the employer before submitting an application.
We are not responsible for any actions, decisions, or outcomes resulting from applying to a job listed here. All interviews, selection processes, and job offers are conducted solely by the employer or organization.
Exercise caution and watch out for fraudulent job offers. Never provide sensitive personal information or make payments to secure a position.