Solution Lead – Big Data and Data Engineering

Role Introduction

Reports to: Senior Solution Lead

Join Cathay as a Solution Lead in our Business Intelligence Competency Centre (BICC) Solution Centre. You will provide subject-matter technical competence in designing, implementing, maintaining, and administrating big data and machine learning infrastructure, defining solutions to bridge gaps and ensuring the project scope is maintained, and solutions implemented in line with design.

Thrive in a dynamic and agile environment that embraces AI and cutting-edge technologies. Our culture fosters teamwork and progressive thinking, while our internal mobility program offers diverse career opportunities.

Leverage your expertise to lead technical delivery in the BICC Solution Centre, ensuring we deliver on our brand promise and drive organizational success together with our wide-reaching flight operation teams around the world.

Key Responsibilities

• Capture and communicate business direction, challenges, and requirements.
• Design, build and support large-scale batch and real-time data pipelines with data processing frameworks on Amazon Cloud Platform and hybrid cloud environments.
• Collaborate with application development teams, architects, Data Scientists and Machine Learning experts and stakeholders.
• Design, develop, implement, and support IT products and applications to meet the customer and IT needs.
• Provide an escalation point for solution analysts to deliver business solutions that include technology implementations and/or adaptations.
• Generate ideas and develop new approaches to improve overall system design and/or streamline delivery of product/system with adherence to security policy and data governance framework.
• Contribute to team resource planning and lead a team of Solution Analysts and vendor on implementation
• Keep abreast of the latest trends in technology, industry, and corresponding business domains
• Participate in Product Vertical Teams to continuously experiment, iterate, and deliver product objectives by adopting Agile development practices

Requirements

• Degree qualification in Information Technology related discipline, master’s preferred
• 8+ years of related IT experience, with over 5+ years of experience in the production deployment and lifecycle management of machine learning models. This includes skills in areas such as model deployment, monitoring, A/B testing, and operationalizing ML pipelines using cloud-based platforms like AWS SageMaker, Azure ML Studio, or similar tools. Expertise in this domain is a must
• Proficient in machine learning frameworks essential for machine learning operation activities
• Hands-on experience in implementing and managing continuous integration (CI) and continuous deployment (CD) pipelines for big data and machine learning applications using tools like Jenkins, GitLab CI/CD, etc
• Familiarity with containerization and orchestration technologies, particularly Docker and Kubernetes, for deploying and managing big data and machine learning applications. Preferably using AWS EKS and OpenShift
• Proficiency in infrastructure provisioning and configuration management using tools like Terraform, Ansible, AWS CDK, and AWS CloudFormation, etc
• Experience in designing and maintaining data pipelines. Knowledge of PySpark, Python, Hadoop ecosystems, open-source development frameworks, etc
• Experience in cloud-based design and development preferably using AWS
• Experience in agile development processes and data-driven development

Personal & Application Information

Cathay Pacific is an Equal Opportunities Employer. Personal data provided by job applicants will be used strictly in accordance with our Applicant Personal Information Collection Statement and for recruitment purposes only. Candidates not notified within eight weeks may consider their application unsuccessful. We keep records of your data for no longer than is necessary for the purpose for which we obtained them and any other permitted linked purposes. If your application is unsuccessful, we will keep your details on file for as long as is necessary to process your application or for the purposes of further job opportunities if you agree to such longer periods.