Talent.com
This job offer is not available in your country.
MCP & Tools Python Developer - Agent Evaluation Infrastructure

MCP & Tools Python Developer - Agent Evaluation Infrastructure

MindriftWorkFromHome, Canterbury, New Zealand
1 day ago
Job description

2 days ago Be among the first 25 applicants

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.

At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.

The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting‑edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real‑world expertise from across the globe.

We’re on the hunt for hands‑on Python engineers for a new project focused on developing Model Context Protocol (MCP) servers and internal tools for running and evaluating agent behavior. You’ll implement base methods for agent action verification, integrate with internal and client infrastructures, and help fill tooling gaps across the team.

Who we’re looking for :

Calling all security researchers, engineers, and penetration testers with a strong foundation in problem‑solving, offensive security, and AI‑related risk assessment.

We’re looking for someone who can bring a hands‑on approach to technical challenges, whether breaking into systems to expose weaknesses or building secure tools and processes. We value contributors with a passion for continuous learning, experimentation, and adaptability.

What you’ll be doing :

  • Developing and maintaining MCP‑compatible evaluation servers
  • Implementing logic to check agent actions against scenario definitions
  • Creating or extending tools that writers and QAs use to test agents
  • Working closely with infrastructure engineers to ensure compatibility
  • Occasionally helping with test writing or debug sessions when needed

Although we’re only looking for experts for this current project, contributors with consistent high‑quality submissions may receive an invitation for ongoing collaboration across future projects.

Requirements :

  • 4+ years of Python development experience, ideally in backend or tools
  • Solid experience building APIs, testing frameworks, or protocol‑based interfaces
  • Understanding of Docker, Linux CLI, and HTTP‑based communication
  • Ability to integrate new tools into existing infrastructures
  • Familiarity with how LLM agents are prompted, executed, and evaluated
  • Clear documentation and communication skills – you’ll work with QA and writers
  • We also value applicants who have :

  • Experience with Model Context Protocol (MCP) or similar structured agent‑server interfaces
  • Knowledge of FastAPI or similar async web frameworks
  • Experience working with LLM logs, scoring functions, or sandbox environments
  • Ability to support dev environments (devcontainers, CI configs, linters)
  • JS experience
  • Benefits :

  • Get paid for your expertise, with rates that can go up to $47 / hour depending on your skills, experience, and project needs
  • Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments
  • Participate in an advanced AI project and gain valuable experience to enhance your portfolio
  • Influence how future AI models understand and communicate in your field of expertise
  • Seniority level

  • Mid‑Senior level
  • Employment type

  • Part‑time
  • Job function

  • Other
  • Industries

  • IT Services and IT Consulting
  • #J-18808-Ljbffr

    Create a job alert for this search

    Developer • WorkFromHome, Canterbury, New Zealand

    Related jobs
    • Promoted
    AI Ops Engineer

    AI Ops Engineer

    One New ZealandWorkFromHome, Canterbury, New Zealand
    At One NZ we’re not just imagining the future, we’re building it.Our purpose? A better-connected Aotearoa New Zealand.Kia renarena te taukaea i Aotearoa New Zealand. Our ambition? To become the most...Show moreLast updated: 6 days ago
    • Promoted
    Staff Software Engineer, Machine Learning

    Staff Software Engineer, Machine Learning

    SmarterDxWorkFromHome, Canterbury, New Zealand
    Staff Software Engineer, Machine Learning.We are looking for a Staff Machine Learning Engineer to help us build a best in class ML ecosystem to advance our product suite with clinical AI.The ideal ...Show moreLast updated: 14 days ago
    • Promoted
    Net / SQL Developer

    Net / SQL Developer

    RICEFW Technologies,IncWorkFromHome, Canterbury, New Zealand
    Reputed US IT Staffing Company based in USA,.Duration : 24 months + possible extensions.Under the direction of client managers, the major responsibilities include performing web interface design, pr...Show moreLast updated: 10 days ago
    • Promoted
    Principal Machine Learning Engineer

    Principal Machine Learning Engineer

    Vital.ioWorkFromHome, Canterbury, New Zealand
    We are looking for a Principal Machine Learning Engineer to join our remote-first team, in a New Zealand & Australia friendly timezone. You will be leading from the front on our ML efforts, ensuring...Show moreLast updated: 29 days ago
    • Promoted
    Senior Software Engineer - Digital Workplace

    Senior Software Engineer - Digital Workplace

    CanonicalWorkFromHome, Canterbury, New Zealand
    Senior Software Engineer - Digital Workplace at Canonical.Join to apply for the Senior Software Engineer - Digital Workplace role at Canonical. Canonical is a leading provider of open source softwar...Show moreLast updated: 30+ days ago
    • Promoted
    Senior C++ Software Engineer

    Senior C++ Software Engineer

    CodeMatchWorkFromHome, Canterbury, New Zealand
    Are you ready to turbo charge your career and contribute to software product transformation, through cutting-edge technologies? Join my clients team in Christchurch as a C++ Software Engineering co...Show moreLast updated: 14 days ago
    • Promoted
    Python Developer | Upto $90 / hr Remote

    Python Developer | Upto $90 / hr Remote

    Crossing HurdlesWorkFromHome, Canterbury, New Zealand
    Python Developer | Upto $90 / hr Remote.Minimum 2 weeks, with potential for extension.Flexible, 10–20 hours / week with possible increase to 40 hours. Codex, Claude code) to perform realistic coding tas...Show moreLast updated: 2 days ago
    • Promoted
    SAP ABAP Developer

    SAP ABAP Developer

    TalentWorkFromHome, Canterbury, New Zealand
    Our client is seeking an SAP ABAP Developer for a 3-6 month remote contract across New Zealand.Proven expertise in developing applications using ABAP RESTful Application Programming Model within S4...Show moreLast updated: 29 days ago
    • Promoted
    Remarkable AI Expert

    Remarkable AI Expert

    Remarkable AIWorkFromHome, Canterbury, New Zealand
    Become a Customer Support Expert for brands you love.Chatdesk (DBA Remarkable AI) Experts are freelance customer support agents who help companies provide the best support for their customers throu...Show moreLast updated: 14 days ago
    • Promoted
    Senior DevOps Architect, Trilogy (Remote) - $200,000 / year USD

    Senior DevOps Architect, Trilogy (Remote) - $200,000 / year USD

    TrilogyWorkFromHome, Canterbury, New Zealand
    Senior DevOps Architect, Trilogy (Remote) - $200,000 / year USD.This range is provided by Trilogy.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more....Show moreLast updated: 3 days ago
    • Promoted
    Solution Architect

    Solution Architect

    FederatoWorkFromHome, Canterbury, New Zealand
    Federato is on a mission to defend the right to efficient, equitable insurance for all.We enable insurers to provide affordable coverage to people and organizations facing today’s challenges, inclu...Show moreLast updated: 14 days ago
    • Promoted
    IoT Data Engineer

    IoT Data Engineer

    CanonicalWorkFromHome, Canterbury, New Zealand
    Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is widely used in breakthrough enterprise initiat...Show moreLast updated: 30+ days ago
    • Promoted
    Engineering Senior Software Engineer New Zealand (Remote) FullTime

    Engineering Senior Software Engineer New Zealand (Remote) FullTime

    Leonardo Interactive PtyWorkFromHome, Canterbury, New Zealand
    Join the Revolution at Leonardo.Ai is an Australian tech startup.Our mission is to unleash the world's creativity with its groundbreaking AI-powered platform. We're seeking highly skilled Senior Sof...Show moreLast updated: 29 days ago
    • Promoted
    Dynamics 365 CE Support Functional Consultant

    Dynamics 365 CE Support Functional Consultant

    HSOWorkFromHome, Canterbury, New Zealand
    HSO is a leading Microsoft Partner with an established pedigree in Dynamics.Although we are relatively new to the ANZ region, we have quickly made our mark as a safe pair of hands for customers' D3...Show moreLast updated: 14 days ago
    • Promoted
    Data Ops Engineer

    Data Ops Engineer

    One New ZealandWorkFromHome, Canterbury, New Zealand
    Data Ops Engineer – Data & AI Platforms at One New Zealand.Define SLIs / SLOs for data freshness, completeness, timeliness, and pipeline success. Build dashboards, alerts, and traces, and ensure every...Show moreLast updated: 5 days ago
    • Promoted
    MuleSoft Developer

    MuleSoft Developer

    Momentum New ZealandWorkFromHome, Canterbury, New Zealand
    MuleSoft Developer | Contract | Anywhere in New Zealand | ASAP start.We are looking for three experienced MuleSoft Developers to join a leading programme of work on a contract basis.You’ll play a k...Show moreLast updated: 26 days ago
    • Promoted
    Embedded Linux Senior Software Engineer - Optimisation

    Embedded Linux Senior Software Engineer - Optimisation

    CanonicalWorkFromHome, Canterbury, New Zealand
    Embedded Linux Senior Software Engineer - Optimisation.Embedded Linux Senior Software Engineer - Optimisation.Embedded Linux Senior Software Engineer - Optimisation. Be among the first 25 applicants...Show moreLast updated: 29 days ago
    • Promoted
    Microsoft Dynamics 365 Developer (X+, F&O)

    Microsoft Dynamics 365 Developer (X+, F&O)

    WindcaveWorkFromHome, Canterbury, New Zealand
    Senior D365 Developer / X++ Developer.As part of our commitment to enhancing our Dynamics 365 capabilities, we are seeking a skilled X++ Developer / Dynamics 365 Developer to join our growing team ...Show moreLast updated: 3 days ago
    • Promoted
    Senior Software Engineer, Data Platform

    Senior Software Engineer, Data Platform

    SmarterDxWorkFromHome, Canterbury, New Zealand
    We are looking for a data and backend-oriented Senior Software Engineer to help us advance our clinical AI by designing and building core systems that handle, process, and analyze clinical data at ...Show moreLast updated: 14 days ago
    • Promoted
    Software Engineer - Python and K8s

    Software Engineer - Python and K8s

    CanonicalWorkFromHome, Canterbury, New Zealand
    Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is widely used in breakthrough enterprise initiat...Show moreLast updated: 30+ days ago