A company based in Johannesburg is looking for a skilled Data Engineer to join their DataWorks team. This role involves creating and improving data systems that help businesses make better decisions using data. You will work in a hybrid setup, meaning you can work both from the office and remotely sometimes.

As a Data Engineer, you will build and manage systems that collect, process, and store large amounts of data. Your main tasks will include designing and maintaining data pipelines that move data between different systems, such as databases, cloud services, and other software. You will work with tools like Microsoft Fabric, AWS, and Databricks to create solutions that are fast, reliable, and easy to scale.

You will also combine data from different sources, like internal company systems and external platforms, into one central place for analysis. This involves setting up data warehouses, data lakes, and other storage systems to organize the data properly. Another important part of your job is making sure the data is accurate, complete, and ready for use by analysts, scientists, and business teams.

You will work closely with data analysts, business intelligence developers, and data scientists to understand what they need from the data. Your goal is to create solutions that help them answer questions, spot trends, and make data-driven decisions. Additionally, you will optimize existing data systems to make them faster, more efficient, and easier to manage.

To succeed in this role, you need at least seven years of experience working with data engineering or building data platforms. A degree in Computer Science, Engineering, Information Technology, or a related field is preferred, though not always required. You should have strong experience working with cloud-based technologies, especially AWS services like Glue, Redshift, S3, Lambda, EMR, and Athena, or platforms like Microsoft Fabric and Databricks.

You will need to be comfortable working with big data tools such as Apache Spark, Hadoop, or Kafka. Programming skills in Python, Pandas, and SQL are essential for processing and transforming data. Experience in the financial services industry is a plus but not required.

This is a permanent job with a competitive salary of up to R1.2 million per year, depending on your experience. The role is hybrid, meaning you can work from the office and remotely. E-Merge IT Recruitment is a specialized agency that helps match the right people with the right companies, so you will have support throughout the hiring process.