Location: London (Hybrid) I am working with a Tier 1 Financial Services organisation who are looking for an experienced Senior Big Data Engineer to join their Credit Risk IT team, working on GCP implementation. As a Big Data Engineer, you will play a crucial role in designing, developing, and implementing large-scale data processing systems using cutting-edge technologies. The role requires a passion for working with big data, possess extensive knowledge of Spark, Hive and Python, and have a solid understanding of UNIX or similar.
- Design, develop, and maintain large-scale data processing systems using Spark, Hive, HDFS, and Python.
- Collaborate with cross-functional teams to identify and implement data engineering solutions to meet business requirements.
- Optimise and improve existing data pipelines and processes for better performance and scalability.
- Perform data analysis, troubleshooting, and debugging to ensure data quality and reliability.
- Develop and maintain ETL processes for data ingestion, transformation, and storage.
- Collaborate with data scientists and analysts to enable efficient data exploration and analysis.
- Provide technical leadership and mentorship to junior team members.
- Stay up to date with the latest advancements in big data technologies and contribute to the improvement of existing systems.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Proven experience as a Big Data Engineer, with a strong hands-on delivery track record.
- Extensive knowledge and experience with Spark, Hive, HDFS, and Python programming.
- In-depth understanding of UNIX or similar and shell scripting.
- Proficiency in using git and SDLC tooling for version control and collaborative development.
- Familiarity with systems programming and cloud platforms, particularly Google Cloud, is advantageous.
- Solid understanding of data engineering best practices, data modelling, and data warehousing concepts.
The role offers a competitive salary, comprehensive benefits package, and opportunities to work on challenging projects that make a real impact.
If you are interested in this role, please apply direct!