Recruitment Room Team

Senior Data Engineer (6-Month Contract) (JHB Hybrid)

Johannesburg – Gauteng – South Africa
1 week ago
Contract
Application ends: September 19, 2025
Deadline date:
September 19, 2025

Job Description


ENVIRONMENT:
OUR client, a reputable Foundation with a mission to strengthen the entrepreneurial ecosystem, has an exciting 6-month renewable contract project-based role for an experienced Senior Data Engineer. The project entails building an enterprise data platform to transform AGGP_SA’s entrepreneurship ecosystem through automated data capture and real-time analytics, consolidating data from 5 disparate systems. The ideal candidate must have 5-8 years Data Engineering experience including utilsing Python/Java/Scala, Apache Spark, Kafka, or similar streaming technologies, AWS & preferably Azure, API integration and RESTful services, SQL and NoSQL databases. Any prior experience in the Non-profit/Social Impact Sector will prove beneficial.
 
DUTIES:
  • Data Pipeline Development (40%): Build real-time ETL pipelines for Asana, SharePoint, AcceleratorApp, Fluxx, and Jibble integration.
  • Data Architecture (30%): Design and implement data lake/warehouse architecture supporting 300,000+ beneficiaries.
  • API Integration (20%): Develop robust API connections with error handling and retry logic.
  • Performance Optimization (10%): Ensure <5-minute data synchronization and support 100,000 records/hour processing.
Deliverables –
    • Fully functional data pipeline integrating all 5 systems.
    • Data lake architecture documentation.
    • Automated data quality monitoring system.
    • Performance benchmarks achieved (<5 min sync).
 
REQUIREMENTS:
  • 5-8 Years Data Engineering experience.
  • Python/Java/Scala for Data Engineering (5+ Years).
  • Apache Spark, Kafka, or similar streaming technologies.
  • Cloud platforms (Azure preferred, AWS acceptable).
  • API integration and RESTful services.
  • SQL and NoSQL databases.
  • Data lake/warehouse architecture (Delta Lake, Databricks).
  • ETL/ELT pipeline development.
  • Real-time data processing.
  • Previous non-profit/social impact sector experience beneficial.
  • Proven track record of multi-system integrations.
  • Experience with data volumes of 1M+ records.
 
Desirable Skills –
  • Microsoft Azure Data Factory.
  • Experience with CRM data models.
  • TolaData or similar impact measurement platforms.
  • SharePoint API/Graph API experience. 
  • Predictive Analytics pipeline setup. 

Related Jobs