7 Hadoop jobs in New Zealand

AgileEngine is a top-ranking provider of software solutions to Fortune 500, Global 500, and Future 50 companies. Listed on Inc. 5000 among the fastest-growing US companies, we are always open to talented software, UX, and data experts in the Americas, Europe, and Asia.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment daily, there’s no better place — guaranteed! :)

What you will do

Collaborate closely with Data Scientists to understand their data requirements, model development needs, and performance optimization goals;

Design, build, and maintain scalable data pipelines and infrastructure to support ML model development and deployment;

Collect, process, and prepare large datasets for analysis, feature engineering, and model training;

Implement data quality checks, validation, and monitoring to ensure the reliability and accuracy of data used in ML models;

Optimize data storage, retrieval, and processing performance for ML workloads;

Develop and maintain ETL (Extract, Transform, Load) processes and workflows;

Manage data versioning, storage, and access control for ML datasets;

Collaborate with DevOps and IT teams to ensure smooth deployment and scaling of ML models in production environments;

Stay up-to-date with emerging technologies, tools, and best practices in data engineering and ML infrastructure.

Must haves 

Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field;

Proven experience as a Data Engineer, preferably in a role supporting Data Scientists;

Strong proficiency in programming languages such as Python, Java, or Scala;

Proficiency with data storage and processing technologies, including SQL databases, NoSQL databases, and distributed computing frameworks (e.g., Hadoop, Spark);

Experience with data warehousing solutions and data integration tools;

Experience with data pipeline and workflow management tools: AWS glue, Airflow, etc;

Knowledge of ML frameworks (e.g., TensorFlow, PyTorch) and ML model deployment;

Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud) and containerization (e.g., Docker, Kubernetes);

Excellent problem-solving skills and a collaborative mindset;

Excellent oral and written communication skills with a keen sense of customer service.

Nice to haves

Experience in ad/marketing environment

Experience working with Data Lakes

Experience with Scala

Experience with Glue/EMR/Redshift/Airflow

Experience with Terraform.

The benefits of joining us

Professional growth

Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.

Competitive compensation

We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.

A selection of exciting projects

Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.

Flextime

Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.

AgileEngine is a top-ranking provider of software solutions to Fortune 500, Global 500, and Future 50 companies. Listed on Inc. 5000 among the fastest-growing US companies, we are always open to talented software, UX, and data experts in the Americas, Europe, and Asia.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment daily, there’s no better place — guaranteed! :)

What you will do

Collaborate closely with Data Scientists to understand their data requirements, model development needs, and performance optimization goals;

Design, build, and maintain scalable data pipelines and infrastructure to support ML model development and deployment;

Collect, process, and prepare large datasets for analysis, feature engineering, and model training;

Implement data quality checks, validation, and monitoring to ensure the reliability and accuracy of data used in ML models;

Optimize data storage, retrieval, and processing performance for ML workloads;

Develop and maintain ETL (Extract, Transform, Load) processes and workflows;

Manage data versioning, storage, and access control for ML datasets;

Collaborate with DevOps and IT teams to ensure smooth deployment and scaling of ML models in production environments;

Stay up-to-date with emerging technologies, tools, and best practices in data engineering and ML infrastructure.

Must haves 

Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field;

Proven experience as a Data Engineer, preferably in a role supporting Data Scientists;

Strong proficiency in programming languages such as Python, Java, or Scala;

Proficiency with data storage and processing technologies, including SQL databases, NoSQL databases, and distributed computing frameworks (e.g., Hadoop, Spark);

Experience with data warehousing solutions and data integration tools;

Experience with data pipeline and workflow management tools: AWS glue, Airflow, etc;

Knowledge of ML frameworks (e.g., TensorFlow, PyTorch) and ML model deployment;

Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud) and containerization (e.g., Docker, Kubernetes);

Excellent problem-solving skills and a collaborative mindset;

Excellent oral and written communication skills with a keen sense of customer service.

Nice to haves

Experience in ad/marketing environment

Experience working with Data Lakes

Experience with Scala

Experience with Glue/EMR/Redshift/Airflow

Experience with Terraform.

The benefits of joining us

Professional growth

Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.

Competitive compensation

We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.

A selection of exciting projects

Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.

Flextime

Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.

AgileEngine is a top-ranking provider of software solutions to Fortune 500, Global 500, and Future 50 companies. Listed on Inc. 5000 among the fastest-growing US companies, we are always open to talented software, UX, and data experts in the Americas, Europe, and Asia.

If you like a challenging environment where you’re working with the best and are encouraged to learn and experiment daily, there’s no better place — guaranteed! :)

What you will do

Collaborate closely with Data Scientists to understand their data requirements, model development needs, and performance optimization goals;

Design, build, and maintain scalable data pipelines and infrastructure to support ML model development and deployment;

Collect, process, and prepare large datasets for analysis, feature engineering, and model training;

Implement data quality checks, validation, and monitoring to ensure the reliability and accuracy of data used in ML models;

Optimize data storage, retrieval, and processing performance for ML workloads;

Develop and maintain ETL (Extract, Transform, Load) processes and workflows;

Manage data versioning, storage, and access control for ML datasets;

Collaborate with DevOps and IT teams to ensure smooth deployment and scaling of ML models in production environments;

Stay up-to-date with emerging technologies, tools, and best practices in data engineering and ML infrastructure.

Must haves 

Bachelor’s or Master’s degree in Computer Science, Data Science, or a related field;

Proven experience as a Data Engineer, preferably in a role supporting Data Scientists;

Strong proficiency in programming languages such as Python, Java, or Scala;

Proficiency with data storage and processing technologies, including SQL databases, NoSQL databases, and distributed computing frameworks (e.g., Hadoop, Spark);

Experience with data warehousing solutions and data integration tools;

Experience with data pipeline and workflow management tools: AWS glue, Airflow, etc;

Knowledge of ML frameworks (e.g., TensorFlow, PyTorch) and ML model deployment;

Familiarity with cloud platforms (e.g., AWS, Azure, Google Cloud) and containerization (e.g., Docker, Kubernetes);

Excellent problem-solving skills and a collaborative mindset;

Excellent oral and written communication skills with a keen sense of customer service.

Nice to haves

Experience in ad/marketing environment

Experience working with Data Lakes

Experience with Scala

Experience with Glue/EMR/Redshift/Airflow

Experience with Terraform.

The benefits of joining us

Professional growth

Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.

Competitive compensation

We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.

A selection of exciting projects

Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.

Flextime

Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.

**Overview**
Are you insatiably curious, deeply passionate about the realm of databases and analytics, and ready to tackle complex challenges in a dynamic environment in the era of AI? If so, we invite you to join our team as a Cloud & AI Solution Engineer in Data Platform for enterprise customers at Microsoft. Here, you'll be at the forefront of cutting-edge projects that leverage the latest technologies to drive meaningful impact. Join us and be part of a team that thrives on collaboration, creativity, and continuous learning.
Databases & Analytics is a growth opportunity for Microsoft Azure, as well as its partners and customers. It includes a rich portfolio of products including IaaS and PaaS services on the Azure Platform in the age of AI. These technologies empower customers to build, deploy, and manage database and analytics applications in a cloud-native way.
**About the role**
As a Data Platform Solution Engineer (SE), you will play a pivotal role in helping enterprises unlock the full potential of Microsoft's cloud database and analytics stack across every stage of deployment. You'll collaborate closely with engineering leaders and platform teams to accelerate the Fabric Data Platform, including Azure Databases and Analytics, through hands-on engagements like Proof of Concepts, hackathons, and architecture workshops. This opportunity will allow you to accelerate your career growth, develop deep business acumen, hone your technical skills, and become adept at solution design and deployment. You'll guide customers through secure, scalable solution design, influence technical decisions, and accelerate database and analytics migration into their deployment workflows. In summary, you'll help customers modernize their data platform and realize the full value of Microsoft's platform, all while enjoying flexible work opportunities.
As a trusted technical advisor, you'll guide customers through secure, scalable solution design, influence technical decisions, and accelerate database and analytics migration into their deployment workflows. In summary, you'll help customers modernize their data platform and realize the full value of Microsoft's platform.
**Responsibilities**
+ Drive technical sales with decision makers using demos and PoCs to influence solution design and enable production deployments.
+ Lead hands-on engagements-hackathons and architecture workshops-to accelerate adoption of Microsoft's cloud platforms.
+ Build trusted relationships with platform leads, co-designing secure, scalable architectures and solutions
+ Resolve technical blockers and objections, collaborating with engineering to share insights and improve products.
+ Maintain deep expertise in Analytics Portfolio: Microsoft Fabric (OneLake, DW, real-time intelligence, BI, Copilot), Azure Databricks, Purview Data Governance and Azure Databases: SQL DB, Cosmos DB, PostgreSQL.
+ Maintain and grow expertise in on-prem EDW (Teradata, Netezza, Exadata), Hadoop & BI solutions.
+ Represent Microsoft through thought leadership in cloud Database & Analytics communities and customer forums
**Qualifications**
+ 5+ years technical pre-sales or technical consulting experience OR Bachelor's Degree in Computer Science, Information Technology, or related field AND 4+ years technical pre-sales or technical consulting experience OR Master's Degree in Computer Science, Information Technology, or related field AND 3+ year(s) technical pre-sales or technical consulting experience OR equivalent experience
+ Expert on Azure Databases (SQL DB, Cosmos DB, PostgreSQL) from migration & modernize and creating new AI apps.
+ Expert on Azure Analytics (Fabric, Azure Databricks, Purview) and competitors (BigQuery, Redshift, Snowflake) in data warehouse, data lake, big data, analytics, real-time intelligent, and reporting using integrated Data Security & Governance.
+ Proven ability to lead technical engagements (e.g., hackathons, PoCs, MVPs) that drive production-scale outcomes.
Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations ( .

Description
AWS Infrastructure Services owns the design, planning, delivery, and operation of all AWS global infrastructure. In other words, we're the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and cooling equipment that ensure our customers have continual access to the innovation they rely on. We work on the most challenging problems, with thousands of variables impacting the supply chain - and we're looking for talented people who want to help.
You'll join a diverse team of software, hardware, and network engineers, supply chain specialists, security experts, operations managers, and other vital roles. You'll collaborate with people across AWS to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. And you'll experience an inclusive culture that welcomes bold ideas and empowers you to own them to completion.
We have an immediate opening for a Mechanical Colo Regional Engineer in Auckland. If you meet these qualifications, exude passion, and enjoy the challenge of innovative projects at hyper-scale, this job is for you! As a Mechanical Colo Regional Engineer, you will provide full life-cycle support to Data Centers from design inception through site improvement and maintenance. You will be the 'go to' engineering resource for your region when technical advice is needed, and will use your subject matter expertise and engage with diverse teams.
Amazon's vision is to be the world's most customer-centric company, and this role is key to that vision. As a Mechanical Colo Regional Engineer, you will be leading projects to fit out the data centers we support to meet ever-evolving customer needs as we continue expanding our fleet to hyper-scale. As an ideal candidate you:
* Possess Strong Engineering Judgement and are able to provide recommendations despite uncertainty
* Are detail and data oriented
* Have experience managing engineering projects and consultants.
* Build trust and relationships with different stakeholders (e.g., Operations, Commissioning, Construction and Design)
* Be inclined to get into the field to see things up close.
Key job responsibilities
* Work with colo vendors to review the design for new Data Centers in your region, including design input, submittal review and incorporating AWS products to ensure the design is meeting AWS expectations.
* Act as resident engineer during new colo construction projects. Support construction, commissioning, and handover.
* Troubleshoot, conduct Root Cause Analysis (RCA) and create Corrective Action (CA) documentation for site/equipment failures.
* Directly support operational issues with ad-hoc training, complex operating procedure reviews, including essential equipment, and event support.
* Provide technical support to the design for existing data center upgrades and design-solutions, which add capacity, improve availability, and increase efficiency.
* Supporting operating partners to lead, review, and approve designs for existing co-location (colo) data center upgrades which improve availability/efficiency.
* Interface with operating partners, data center design engineering team, server hardware team, environmental health and safety team to promote standards that maintain consistency and reliability in services delivered by operating partners.
* Work on concurrent projects, sometimes in multiple geographical regions.
* Initiate and lead engineering site audits within leased or colo data centers. Produce reports outlining risks with recommended mitigations and remediation's.
A day in the life
Each day you will interact with different teams responsible for all aspects of the data centers. You will prioritize your activities to support data center capacity availability and safety focusing on the actions that are most impactful. You will have the opportunity to work on projects locally and globally.
About the team
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn't followed a traditional path, or includes alternative experiences, don't let it stop you from applying.
Why AWS?
Amazon Web Services (AWS) is the world's most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating - that's why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Inclusive Team Culture
AWS values curiosity and connection. Our employee-led and company-sponsored affinity groups promote inclusion and empower our people to take pride in what makes us unique. Our inclusion events foster stronger, more collaborative teams. Our continual innovation is fueled by the bold ideas, fresh perspectives, and passionate voices our teams bring to everything we do.
Mentorship & Career Growth
We're continuously raising our performance bar as we strive to become Earth's Best Employer. That's why you'll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there's nothing we can't achieve in the cloud.
Basic Qualifications
- Bachelor's Degree in Mechanical Engineering or equivalent experience.
- Direct experience with the design, construction, operation, or maintenance of mission critical facilities, especially data centers.
- 6+ cumulative years of experience with industrial or commercial engineering in Mission Critical facilities including but not limited to: data centers, power generation, oil / gas facilities. (Experienced Engineer)
Preferred Qualifications
- Experience reading, interpreting, and creating construction drawings, specifications, and submittal documents.
- Ability to carry design concepts through exploration, development, and into deployment/mass production
- Basic understanding of both mechanical and electrical equipment/design related to data centers (Including but not limited to: uninterruptible power sources , diesel generators, electrical switch-gear, power distribution units, variable frequency drives, automatic/static transfer switches, chillers (air-cooled and water-cooled), pumps, cooling towers, heat exchangers, CRAHs, air economizers, etc.)
- Have fluent knowledge of continuous operating redundant electrical systems, cooling systems, air flow containment systems and building management systems. (Including but not limited to: uninterruptible power sources, AC/DC conversion, P&ID loops, diesel generators systems and complex arrangements, direct evaporative cooling systems, etc.). IDE statement:
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit for more information. If the country/region you're applying in isn't listed, please contact your Recruiting Partner.