FULLTIME
Technical AI Evaluation Speclalist (Code & Software Engineering)
Careerblock
Not specified · onsite · Posted 48d ago
Sourced from
Not disclosedFresherfulltimeNot specified
About a minute. You'll make an account on the way through.
agent · working
- Read this posting0.5s
- Found the form on Foundit · 9 questions to answer1.8s
- Decided the answers for all of them2.4s
- Tailor your résumé to this role
- Waiting for you
Section · 01
About this role
About The Opportunity
Careerbloc is a next-generation talent and AI evaluation platform empowering enterprises to hire with precision by assessing technical skills through real-world, code-first evaluations. We blend human expertise with AI-powered validation to deliver unbiased, scalable assessments for software engineering and technical roles across the globe—helping companies future-proof their tech teams and candidates showcase their true capabilities.
Role & Responsibilities
Must-Have
Careerbloc is a next-generation talent and AI evaluation platform empowering enterprises to hire with precision by assessing technical skills through real-world, code-first evaluations. We blend human expertise with AI-powered validation to deliver unbiased, scalable assessments for software engineering and technical roles across the globe—helping companies future-proof their tech teams and candidates showcase their true capabilities.
Role & Responsibilities
- Design, validate, and execute AI-driven evaluation frameworks for software engineering candidates—focusing on code quality, algorithmic efficiency, and system design patterns.
- Build and maintain scalable code evaluation rubrics using real-world programming challenges (Python, Java, JavaScript, SQL, etc.) aligned with industry-grade engineering standards.
- Analyze candidate submissions using static analysis tools, automated test suites, and manual code reviews to ensure fair, objective, and context-aware scoring.
- Collaborate with AI/ML engineers to fine-tune LLM-based evaluation models for assessing code correctness, readability, edge-case handling, and optimization.
- Identify and eliminate evaluation biases in test cases, improve scoring logic, and contribute to continuous feedback loops for model refinement.
- Document evaluation workflows, maintain test case libraries, and train QA/assessment teams on evaluation best practices and tooling.
Must-Have
- Python
- Java
- JavaScript
- SQL
- Git
- Unit Testing
- Code Review
- CI/CD Pipelines
- Experience with LLM evaluation frameworks (e.g., LangChain, RAG, LlamaIndex)
- Familiarity with code quality tools (SonarQube, Codacy, ESLint)
- Knowledge of SDE interview patterns (LeetCode, HackerRank-style assessments)
- 100% remote with flexible hours—work from anywhere in India.
- Impact-driven culture: your work directly shapes how millions of engineers are assessed globally.
- Continuous learning: access to AI/tech workshops, certification stipends, and peer mentorship.
Sourced from foundit
Section · 02
Skills
SDE interview patternsCode ReviewsonarqubeCI/CD PipelinesUnit testingeslintLLM evaluation frameworksPythonJavascriptSQLJavaGITCodacy