Talent.com
This job offer is not available in your country.
MCP & Tools Python Developer - Agent Evaluation Infrastructure

MCP & Tools Python Developer - Agent Evaluation Infrastructure

MindriftWorkFromHome, Hawke's Bay, New Zealand
4 days ago
Job description

2 days ago Be among the first 25 applicants

This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.

At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI.

The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting‑edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into real‑world expertise from across the globe.

We’re on the hunt for hands‑on Python engineers for a new project focused on developing Model Context Protocol (MCP) servers and internal tools for running and evaluating agent behavior. You’ll implement base methods for agent action verification, integrate with internal and client infrastructures, and help fill tooling gaps across the team.

Who we’re looking for :

Calling all security researchers, engineers, and penetration testers with a strong foundation in problem‑solving, offensive security, and AI‑related risk assessment.

We’re looking for someone who can bring a hands‑on approach to technical challenges, whether breaking into systems to expose weaknesses or building secure tools and processes. We value contributors with a passion for continuous learning, experimentation, and adaptability.

What you’ll be doing :

  • Developing and maintaining MCP‑compatible evaluation servers
  • Implementing logic to check agent actions against scenario definitions
  • Creating or extending tools that writers and QAs use to test agents
  • Working closely with infrastructure engineers to ensure compatibility
  • Occasionally helping with test writing or debug sessions when needed

Although we’re only looking for experts for this current project, contributors with consistent high‑quality submissions may receive an invitation for ongoing collaboration across future projects.

Requirements :

  • 4+ years of Python development experience, ideally in backend or tools
  • Solid experience building APIs, testing frameworks, or protocol‑based interfaces
  • Understanding of Docker, Linux CLI, and HTTP‑based communication
  • Ability to integrate new tools into existing infrastructures
  • Familiarity with how LLM agents are prompted, executed, and evaluated
  • Clear documentation and communication skills – you’ll work with QA and writers
  • We also value applicants who have :

  • Experience with Model Context Protocol (MCP) or similar structured agent‑server interfaces
  • Knowledge of FastAPI or similar async web frameworks
  • Experience working with LLM logs, scoring functions, or sandbox environments
  • Ability to support dev environments (devcontainers, CI configs, linters)
  • JS experience
  • Benefits :

  • Get paid for your expertise, with rates that can go up to $47 / hour depending on your skills, experience, and project needs
  • Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments
  • Participate in an advanced AI project and gain valuable experience to enhance your portfolio
  • Influence how future AI models understand and communicate in your field of expertise
  • Seniority level

  • Mid‑Senior level
  • Employment type

  • Part‑time
  • Job function

  • Other
  • Industries

  • IT Services and IT Consulting
  • #J-18808-Ljbffr

    Create a job alert for this search

    Developer • WorkFromHome, Hawke's Bay, New Zealand

    Related jobs
    • Promoted
    Senior Software Engineer - Digital Workplace

    Senior Software Engineer - Digital Workplace

    CanonicalWorkFromHome, Hawke's Bay, New Zealand
    Senior Software Engineer - Digital Workplace.Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets.Our platform, Ubuntu, is w...Show moreLast updated: 30+ days ago
    • Promoted
    Python and Kubernetes Software Engineer - Data, AI / ML & Analytics

    Python and Kubernetes Software Engineer - Data, AI / ML & Analytics

    CanonicalWorkFromHome, Hawke's Bay, New Zealand
    Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is widely used in breakthrough enterprise initiat...Show moreLast updated: 30+ days ago
    • Promoted
    Senior Software Engineer, Data Platform

    Senior Software Engineer, Data Platform

    SmarterDxWorkFromHome, Hawke's Bay, New Zealand
    We are looking for a data and backend-oriented Senior Software Engineer to help us advance our clinical AI by designing and building core systems that handle, process, and analyze clinical data at ...Show moreLast updated: 17 days ago
    • Promoted
    Embedded Linux Senior Software Engineer - Optimisation

    Embedded Linux Senior Software Engineer - Optimisation

    CanonicalWorkFromHome, Hawke's Bay, New Zealand
    Embedded Linux Senior Software Engineer - Optimisation.Embedded Linux Senior Software Engineer - Optimisation.Embedded Linux Senior Software Engineer - Optimisation. Be among the first 25 applicants...Show moreLast updated: 30+ days ago
    • Promoted
    Staff / Principal Software Engineer - NZ Remote

    Staff / Principal Software Engineer - NZ Remote

    Auror LimitedWorkFromHome, Hawke's Bay, New Zealand
    Please note : This role is only available for candidates working remotely from NZ.We are unable to consider any applications outside of NZ. Auror is empowering the retail industry to tackle theft and...Show moreLast updated: 11 days ago
    • Promoted
    Senior Legal Counsel

    Senior Legal Counsel

    Unison NetworksHastings, Hawke's Bay, New Zealand
    Newly created position in Unison’s growing in-house legal team.Competitive remuneration package; contribution to relocation costs if required. Great team, supportive leadership, leading Hawke’s Bay-...Show moreLast updated: 14 days ago
    • Promoted
    Infrastructure Advocate

    Infrastructure Advocate

    Portainer.ioWorkFromHome, Hawke's Bay, New Zealand
    Portainer is looking for an Infrastructure Advocate, someone who’s lived the reality of enterprise IT operations and knows what it takes to run critical infrastructure under pressure.This isn’t dev...Show moreLast updated: 2 days ago
    • Promoted
    Freelance Software Developer (Python) - AI Trainer

    Freelance Software Developer (Python) - AI Trainer

    MindriftWorkFromHome, Hawke's Bay, New Zealand
    The Mindrift platform connects specialists with AI projects from major tech innovators.Our mission is to unlock the potential of Generative AI by tapping into real-world expertise from across the g...Show moreLast updated: 17 days ago
    • Promoted
    Implementation Engineer

    Implementation Engineer

    EngFlowWorkFromHome, Hawke's Bay, New Zealand
    Our cloud-based, distributed service optimizes developer workflows through remote execution and caching, improving efficiency, productivity, and product quality. Backed by top investors, EngFlow is ...Show moreLast updated: 17 days ago
    • Promoted
    Performance Engineer - Open Source

    Performance Engineer - Open Source

    CanonicalWorkFromHome, Hawke's Bay, New Zealand
    Join or sign in to find your next job.Performance Engineer - Open Source.Be among the first 25 applicants.Performance Engineer - Open Source. Canonical is a leading provider of open source software ...Show moreLast updated: 30+ days ago
    • Promoted
    Principal Machine Learning Engineer

    Principal Machine Learning Engineer

    Vital.ioWorkFromHome, Hawke's Bay, New Zealand
    We are looking for a Principal Machine Learning Engineer to join our remote-first team, in a New Zealand & Australia friendly timezone. You will be leading from the front on our ML efforts, ensuring...Show moreLast updated: 30+ days ago
    • Promoted
    Programming Analyst

    Programming Analyst

    DataAnnotationWorkFromHome, Hawke's Bay, New Zealand
    You will train AI models, measure the progress of AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. To apply, you will need to be proficient in either Pytho...Show moreLast updated: 11 days ago
    • Promoted
    Senior Principal Software Engineer - Platform Engineering

    Senior Principal Software Engineer - Platform Engineering

    AtlassianWorkFromHome, Hawke's Bay, New Zealand
    Senior Principal Software Engineer - Platform Engineering.Senior Principal Software Engineer - Platform Engineering.Senior Principal Software Engineer - Platform Engineering.Senior Principal Softwa...Show moreLast updated: 30+ days ago
    • Promoted
    Python and Kubernetes Software Engineer - Data, Workflows, AI / ML & Analytics

    Python and Kubernetes Software Engineer - Data, Workflows, AI / ML & Analytics

    CanonicalWorkFromHome, Hawke's Bay, New Zealand
    Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is widely used in public cloud, data science, AI,...Show moreLast updated: 30+ days ago
    • Promoted
    Python Developer | Upto $90 / hr Remote

    Python Developer | Upto $90 / hr Remote

    Crossing HurdlesWorkFromHome, Hawke's Bay, New Zealand
    Python Developer | Upto $90 / hr Remote.Minimum 2 weeks, with potential for extension.Flexible, 10–20 hours / week with possible increase to 40 hours. Codex, Claude code) to perform realistic coding tas...Show moreLast updated: 5 days ago
    • Promoted
    Software Engineer - Python and K8s

    Software Engineer - Python and K8s

    CanonicalWorkFromHome, Hawke's Bay, New Zealand
    Software Engineer - Python and K8s.Canonical is a leading provider of open source software and operating systems to the global enterprise and technology markets. Our platform, Ubuntu, is widely used...Show moreLast updated: 30+ days ago
    • Promoted
    Engineering Senior Software Engineer New Zealand (Remote) FullTime

    Engineering Senior Software Engineer New Zealand (Remote) FullTime

    Leonardo Interactive PtyWorkFromHome, Hawke's Bay, New Zealand
    Join the Revolution at Leonardo.Ai is an Australian tech startup.Our mission is to unleash the world's creativity with its groundbreaking AI-powered platform. We're seeking highly skilled Senior Sof...Show moreLast updated: 30+ days ago
    • Promoted
    Division CFO, Trilogy (Remote) - $400,000 / year USD

    Division CFO, Trilogy (Remote) - $400,000 / year USD

    TrilogyWorkFromHome, Hawke's Bay, New Zealand
    Division CFO, Trilogy (Remote) - $400,000 / year USD.Division CFO, Trilogy (Remote) - $400,000 / year USD.This range is provided by Trilogy. Your actual pay will be based on your skills and experience —...Show moreLast updated: 6 days ago