What you will do as a Senior Data Engineer at Prospa :-
Maintenance and expansion of the current data warehouse and data pipelines, including batched jobs, streaming processing, CI/CD and data ingestions.
Finding new ways of using technologies to either create or enhance our codebase and frameworks for data processing
Manage and monitor job executions and alerting, improve the dependency workflow to allow parallelism, concurrency and best usage of cluster resources and infrastructure.
Be part of architectural and design sessions, sharing your experience and knowledge openly with the wider team
Share your knowledge and expertise, inputting product “grooming” sessions and stand-ups
Redesign our legacy products into the current & future platform
Ability to execute an end-to-end project/data solution
What you need to have for this role :
Your primary skillset for this role is to code on Python, Spark and SQL to create efficient and scalable software
Hands on experience on ETL, data pipelines and implementing technologies from ground up using cloud technologies
Good knowledge on data warehouse, data modeling techniques, and job workflow management
Experience on Azure is an advantage over the other cloud vendors with CI/CD
Git + Workflows to support code review and code release of software
Be an Agile mindset and follow its principles
Streaming processing background is another advantage
Creating reusable toolsets and libraries that the team can leverage to get to value fast
Working with diverse stakeholders to push for data driven approaches, rapid experimentation and investment in data capabilities
Must have as a fellow Prosperian :
Focuses on the Customer: Ability to build trust and respect with customers and channel partners
Excellent communications and presentation skills: Ability to establish immediate rapport with partners, customers and related industry representatives and become a trusted advisor
Results Driven Self-starter: Exemplary organisational skills coupled with a high degree of attention to detail and a proactive “can-do” attitude.
Executes Perfectly & Takes Accountability for Actions: Excellent at executing agreed decisions without undue delay. Takes responsibility for own actions or decisions, regardless of outcomes or success.
Fosters Teamwork, Collaboration & Respect for Others: Seeks and values diverse contributions, experiences and ideas of others. Flexible team player who thrives in fast-paced environments and is able to work autonomously.