Aug 16, 2019

Senior Data Engineer

  • Prospa
  • Sydney NSW, Australia
Full time Data Engineer

Job Description

What you will do as a Senior Data Engineer at Prospa :-

Maintenance and expansion of the current data warehouse and data pipelines, including batched jobs, streaming processing, CI/CD and data ingestions.

Finding new ways of using technologies to either create or enhance our codebase and frameworks for data processing

Manage and monitor job executions and alerting, improve the dependency workflow to allow parallelism, concurrency and best usage of cluster resources and infrastructure.

Be part of architectural and design sessions, sharing your experience and knowledge openly with the wider team

Share your knowledge and expertise, inputting product “grooming” sessions and stand-ups

Redesign our legacy products into the current & future platform

Ability to execute an end-to-end project/data solution 

What you need to have for this role :

Your primary skillset for this role is to code on Python, Spark and SQL to create efficient and scalable software

Hands on experience on ETL, data pipelines and implementing technologies from ground up using cloud technologies

Good knowledge on data warehouse, data modeling techniques, and job workflow management

Experience on Azure is an advantage over the other cloud vendors with CI/CD

Git + Workflows to support code review and code release of software

Be an Agile mindset and follow its principles

Streaming processing background is another advantage

Creating reusable toolsets and libraries that the team can leverage to get to value fast

Working with diverse stakeholders to push for data driven approaches, rapid experimentation and investment in data capabilities

Must have as a fellow Prosperian :

Focuses on the Customer: Ability to build trust and respect with customers and channel partners

Excellent communications and presentation skills: Ability to establish immediate rapport with partners, customers and related industry representatives and become a trusted advisor

Results Driven Self-starter: Exemplary organisational skills coupled with a high degree of attention to detail and a proactive “can-do” attitude.

Executes Perfectly & Takes Accountability for Actions: Excellent at executing agreed decisions without undue delay. Takes responsibility for own actions or decisions, regardless of outcomes or success.

Fosters Teamwork, Collaboration & Respect for Others: Seeks and values diverse contributions, experiences and ideas of others. Flexible team player who thrives in fast-paced environments and is able to work autonomously.

Role

Senior Big Data Engineer