Talent.com
This job offer is not available in your country.
MCP & Tools Python Developer - Agent Evaluation Infrastructure

MCP & Tools Python Developer - Agent Evaluation Infrastructure

MindriftWorkFromHome, Wellington, New Zealand
15 hours ago
Job description

2 days ago Be among the first 25 applicants

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.

At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.

The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting‑edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real‑world expertise from across the globe.

We’re on the hunt for hands‑on Python engineers for a new project focused on developing Model Context Protocol (MCP) servers and internal tools for running and evaluating agent behavior. You’ll implement base methods for agent action verification, integrate with internal and client infrastructures, and help fill tooling gaps across the team.

Who we’re looking for :

Calling all security researchers, engineers, and penetration testers with a strong foundation in problem‑solving, offensive security, and AI‑related risk assessment.

We’re looking for someone who can bring a hands‑on approach to technical challenges, whether breaking into systems to expose weaknesses or building secure tools and processes. We value contributors with a passion for continuous learning, experimentation, and adaptability.

What you’ll be doing :

  • Developing and maintaining MCP‑compatible evaluation servers
  • Implementing logic to check agent actions against scenario definitions
  • Creating or extending tools that writers and QAs use to test agents
  • Working closely with infrastructure engineers to ensure compatibility
  • Occasionally helping with test writing or debug sessions when needed

Although we’re only looking for experts for this current project, contributors with consistent high‑quality submissions may receive an invitation for ongoing collaboration across future projects.

Requirements :

  • 4+ years of Python development experience, ideally in backend or tools
  • Solid experience building APIs, testing frameworks, or protocol‑based interfaces
  • Understanding of Docker, Linux CLI, and HTTP‑based communication
  • Ability to integrate new tools into existing infrastructures
  • Familiarity with how LLM agents are prompted, executed, and evaluated
  • Clear documentation and communication skills – you’ll work with QA and writers
  • We also value applicants who have :

  • Experience with Model Context Protocol (MCP) or similar structured agent‑server interfaces
  • Knowledge of FastAPI or similar async web frameworks
  • Experience working with LLM logs, scoring functions, or sandbox environments
  • Ability to support dev environments (devcontainers, CI configs, linters)
  • JS experience
  • Benefits :

  • Get paid for your expertise, with rates that can go up to $47 / hour depending on your skills, experience, and project needs
  • Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments
  • Participate in an advanced AI project and gain valuable experience to enhance your portfolio
  • Influence how future AI models understand and communicate in your field of expertise
  • Seniority level

  • Mid‑Senior level
  • Employment type

  • Part‑time
  • Job function

  • Other
  • Industries

  • IT Services and IT Consulting
  • #J-18808-Ljbffr

    Create a job alert for this search

    Developer • WorkFromHome, Wellington, New Zealand

    Related jobs
    • Promoted
    Senior DevOps Architect, Trilogy (Remote) - $200,000 / year USD

    Senior DevOps Architect, Trilogy (Remote) - $200,000 / year USD

    TrilogyWorkFromHome, Wellington, New Zealand
    Senior DevOps Architect, Trilogy (Remote) - $200,000 / year USD.This range is provided by Trilogy.Your actual pay will be based on your skills and experience — talk with your recruiter to learn more....Show moreLast updated: 2 days ago
    • Promoted
    Dynamics 365 F&O Analyst Programmer (Relocate to Malta / Big 4)

    Dynamics 365 F&O Analyst Programmer (Relocate to Malta / Big 4)

    Black Pen RecruitmentWellington, New Zealand
    Our client’s Microsoft Business Solutions team is a Microsoft Gold Partner and leader in Microsoft software implementations for medium to large organisations, providing their clients with the abili...Show moreLast updated: 30+ days ago
    • Promoted
    Data Architect

    Data Architect

    New Zealand GovernmentWellington, New Zealand
    What is the most important thing in the world, it is people, it is people, it is people.He tangata, he tangata, he tangata. WorkSafe New Zealand / Mahi Haumaru Aotearoa, is the primary workplace heal...Show moreLast updated: 1 day ago
    • Promoted
    Data Ops Engineer

    Data Ops Engineer

    One New ZealandWorkFromHome, Wellington, New Zealand
    Data Ops Engineer – Data & AI Platforms at One New Zealand.Define SLIs / SLOs for data freshness, completeness, timeliness, and pipeline success. Build dashboards, alerts, and traces, and ensure every...Show moreLast updated: 4 days ago
    • Promoted
    DevOps Engineer

    DevOps Engineer

    black.aiWorkFromHome, Wellington, New Zealand
    New Zealand (Remote), Auckland (Remote), Wellington (Remote).Join the Revolution at Leonardo.Ai, we’re building a generative AI platform that empowers millions to bring their ideas to life.Now part...Show moreLast updated: 5 days ago
    • Promoted
    Devops Engineer (Data)

    Devops Engineer (Data)

    Randstad New ZealandWellington, New Zealand
    OverviewDevops Engineer (Data) role at Randstad New Zealand.Contract position until June • • • •, based in Wellington, NZ. Immediate availability or short notice preferred.ResponsibilitiesDesigning, bu...Show moreLast updated: 8 days ago
    • Promoted
    Devops Engineer

    Devops Engineer

    Leonardo Interactive PtyWellington, New Zealand
    We're looking for a DevOps Engineer to join our Platform Tribe, the foundation that powers everything from Leonardo's AI generation features to enterprise APIs and mobile experiences.Leonardo's eng...Show moreLast updated: 2 days ago
    • Promoted
    Technology Mainframe Analyst (12-Month Fixed-Term)

    Technology Mainframe Analyst (12-Month Fixed-Term)

    Australia And New Zealand Banking Group LimitedWellington, New Zealand
    Select how often (in days) to receive an alert : .Department : Tech Wholesale & Enterprise Delivery (NZ).Location : Wellington or Auckland. Kia Hanga I Te Ao, E Ora Ai, E Tupu Ai, Te Tangata Me Te Kaing...Show moreLast updated: 9 days ago
    • Promoted
    Dynamics 365 Functional F&O Senior Consultant (Relocate to Malta / Big 4)

    Dynamics 365 Functional F&O Senior Consultant (Relocate to Malta / Big 4)

    Black Pen RecruitmentWellington, New Zealand
    Our client’s Microsoft Business Solutions team is a Microsoft Gold Partner and leader in Microsoft software implementations for medium to large organisations, providing their clients with the abili...Show moreLast updated: 30+ days ago
    • Promoted
    Analyst - System Transformation

    Analyst - System Transformation

    The Treasury - New ZealandWellington, New Zealand
    OverviewHave you got what it takes to shape Aotearoa New Zealand's tomorrow?.This is an exceptional opportunity to bring your analytics skills to transform Treasury's data capabilities.Joining our ...Show moreLast updated: 9 days ago
    • Promoted
    Integration Analyst Developer

    Integration Analyst Developer

    TeksystemsWellington, New Zealand
    Our client is a recognized global organisation looking for a Senior Developer to take part in their Global Digital Transformation. This is an exciting opportunity to contribute to different projects...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Devops Engineer

    Senior Devops Engineer

    New Zealand GovernmentWellington, New Zealand
    Mo tenei turanga mahi | About this role • • •Apply your DevOps, SRE, or Platform skills in a high-impact enabling role.Join a well-established team working at scale with SAFe methodology.Lead API and...Show moreLast updated: 30+ days ago
    • Promoted
    DevOps Engineer

    DevOps Engineer

    Leonardo.AiWorkFromHome, New Zealand
    Join the Revolution at Leonardo.Ai, we’re building a generative AI platform that empowers millions to bring their ideas to life. Now part of the Canva family, we’re scaling our engineering teams to ...Show moreLast updated: 2 days ago
    • Promoted
    Ai Developer

    Ai Developer

    DatacomNew Zealand
    Our WhyDatacom works with organisations and communities across Australia and New Zealand to make a difference in people's lives and help organisations use the power of tech to innovate and grow.Abo...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Devops Platform Engineer

    Senior Devops Platform Engineer

    Mott MacdonaldNew Zealand
    Recruiter contact : Simon Hampton.Mott MacDonald is a uniquely diverse employee-owned, engineering and development consultancy serving the public and private sector around the world.We are looking f...Show moreLast updated: 1 day ago
    • Promoted
    Senior Devops Engineer

    Senior Devops Engineer

    New Zealand Customs ServiceWellington, New Zealand
    New Zealand Traveller Declaration | Whakapuakanga Tangata Haere ki Aotearoa.The New Zealand Traveller Declaration (NZTD) is the digital declaration used to collect information from travellers comin...Show moreLast updated: 30+ days ago
    • Promoted
    Technology Mainframe Analyst (12-Month Fixed-Term)

    Technology Mainframe Analyst (12-Month Fixed-Term)

    Ko Matou About UsNew Zealand
    Kia Hanga I Te Ao, E Ora Ai, E Tupu Ai, Te Tangata Me Te Kainga.At ANZ, we're shaping a world where people and communities thrive, driven by a common goal : to improve the financial wellbeing and su...Show moreLast updated: 1 day ago
    • Promoted
    Python Developer | Upto $90 / hr Remote

    Python Developer | Upto $90 / hr Remote

    Crossing HurdlesWorkFromHome, Wellington, New Zealand
    Python Developer | Upto $90 / hr Remote.Minimum 2 weeks, with potential for extension.Flexible, 10–20 hours / week with possible increase to 40 hours. Codex, Claude code) to perform realistic coding tas...Show moreLast updated: 1 day ago
    • Promoted
    Devops Engineer (Data)

    Devops Engineer (Data)

    RandstadWellington, New Zealand
    Our client is looking for a highly skilled and experienced DevOps Engineer to join their team on a contract basis until June • • • •. This role is crucial for managing and optimising the data pipelines...Show moreLast updated: 10 days ago
    • Promoted
    • New!
    Ai Agent Evaluation Analyst

    Ai Agent Evaluation Analyst

    MindriftNew Zealand
    This opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of En...Show moreLast updated: 2 hours ago