Lead Data Engineer

Eligibility: Australian PR or Citizenship required

Were partnering with a major player in Financial Services to find a Lead Data Engineer who combines strong technical expertise with leadership skills to drive major transformations. This role is perfect for a seasoned Data Engineer with a background in ETL, Big Data, and AWS, whos ready to lead a small team while remaining hands-on.

The Role This is a unique opportunity to join a high-performing data team, working across multiple business units to deliver innovative solutions. Youll lead a team of 2-3 data engineers, set them up for success, and take ownership of their deliverables. As a hands-on technical lead, youll work closely with an engineering manager and product owner to drive automation projects, regulatory initiatives, and new solutions.

Day-to-Day Responsibilities:

• Design and develop end-to-end data solutions in a Spark, SQL, and AWS environment.
• Build, optimize, and manage ETL pipelines to produce critical regulatory reports.
• Lead solution design workshops and ensure alignment with business goals.
• Uplift and mentor team members while maintaining accountability for delivery.
• Drive platform efficiency through automation and AI, utilizing Python.
• Collaborate with stakeholders to identify and solve data challenges.

• ETL Expertise: Strong hands-on experience with tools like Talend, SSIS, Informatica, or similar.
• Big Data Skills: Expertise in Spark, Hadoop, Hive, and Presto. Familiarity with modern storage formats like Parquet, ORC, Avro.
• Cloud Knowledge: Extensive experience with AWS services (EC2, S3, RDS, EMR).
• Kafka: Experience building real-time data pipelines and streaming applications, enabling tasks like real-time analytics, data ingestion, and event-driven architectures. 
• SQL and SparkSQL: Advanced skills, including performance tuning.
• Data Modelling: Proficient in creating efficient, scalable data models.
• Automation: Hands-on with Python for enhancing processes and implementing AI-driven solutions in the future. 

• Proven leadership experience managing teams of data engineers.
• A strong background in end-to-end data pipeline development.
• Strategic thinker who can also roll up their sleeves and contribute hands-on.
• Experience working in fast-paced environments with tight regulatory and business demands.
• Sydney-based and able to work in a hybrid model (3 days in office).

Information :

• Company : Preacta Recruitment
• Position : Lead Data Engineer
• Location : Sydney CBD NSW
• Country : AU