Senior Data Platforms Engineer

Simple Machines is a leading independent boutique technology firm with a global presence, including teams in London, Sydney, San Francisco, and New Zealand.

We specialise in creating technology solutions at the intersection of data, AI, machine learning, data engineering, and software engineering.

Our mission is to help enterprises, technology companies, and governments better connect with and understand their organisations, their people, their customers, and citizens.We are seeking a Senior Data Platforms Engineer to join our team.

The role is a dynamic, hands-on position focused on building real-time data pipelines and implementing data mesh architectures to enhance client data interactions.Technical ResponsibilitiesDeveloping Data Solutions : Implement and enhance data-driven solutions integrating with clients' systems using state-of-the-art tools such as Databricks, Snowflake, Google Cloud, and AWS.Data Pipeline Development : Develop and optimise high-performance, batch and real-time data pipelines employing advanced streaming technologies like Kafka, and Flink.Database and Storage Optimisation : Optimise and manage a broad array of database technologies, from traditional relational databases to modern NoSQL solutions.Big Data Processing & Analytics : Utilise big data frameworks such as Apache Spark and Apache Flink to address challenges associated with large-scale data processing and analysis.Cloud Data Management : Implement and oversee cloud-specific data services including AWS Redshift, S3, Google BigQuery, and Google Cloud Storage.Security and Compliance : Ensure all data practices comply with security policies and regulations, embedding security by design in the data infrastructure.Consulting ResponsibilitiesClient Advisory : Provide expert advice to clients on optimal data practices that align with their business requirements and project goals.Training and Empowerment : Educate client teams on the latest technologies and data management strategies, enabling them to efficiently utilise and maintain the solutions we have developed.Professional Development : Keep up with the latest industry trends and technological advancements, continually upgrading skills and achieving certifications in the technologies Simple Machines implements across its client base.Ideal Skills and ExperienceCore Data Engineering Tools & Technologies : Demonstrates proficiency in SQL and Spark, and familiarity with platforms such as Databricks and Snowflake.Data Storage Expertise : Knowledgeable in data warehousing technologies like BigQuery, Snowflake, and Databricks.Building and Managing Large-scale Data Systems : Experienced in developing and overseeing large-scale data pipelines and data-intensive applications within production environments.Data Modelling Expertise : Proficient in data modelling, understanding the implications and trade-offs of various methodologies and approaches.Infrastructure Configuration for Data Systems : Competent in setting up data system infrastructures, favouring infrastructure-as-code practices using tools such as Terraform and Pulumi.Programming Languages : Proficient in Python and SQL, with additional experience in programming languages like Java, Scala, GoLang, and Rust considered advantageous.CI / CD Implementation : Knowledgeable about continuous integration and continuous deployment practices using tools like GitHub Actions and ArgoCD.Testing Tools and Frameworks : Experienced with data quality and testing frameworks such as DBT, Great Expectations, and Soda.Commercial Application of Data Engineering Expertise : Demonstrated experience in applying data engineering skills across various industries and organisations in a commercial context.Agile Delivery and Project Management : Skilled in agile, scrum, and kanban project delivery methods, ensuring efficient and effective solution development.Consulting and Advisory Skills : Experienced in a consultancy or professional services setting, offering expert advice and crafting customised solutions that address client needs.Professional Experience and QualificationsProfessional Experience : At least 8+ years of data engineering or equivalent experience in a commercial, enterprise, or start-up environment.Educational Background : Degree or equivalent experience in computer science or a related field.Right to Work : Must have full New Zealand working rights and reside in Wellington.

#J-18808-Ljbffr