Data Engineer

Roles and Responsibilities

The primary role of a Data Engineer position is to participate in the planning, building and operating of data platforms that make data available to other software services, data scientists, and data analysts. The position has the following responsibilities:

1. Participate in the planning process with senior engineers to align data platform architecture with business requirements
2. Provision the collection of data from various sources as required
3. Structure and configure data pipelines, databases, and data warehouses
4. Structure and configure cloud services in the AWS ecosystem that provide data engineering functionalities
5. Work with data scientists and analysts to optimise data structures with ETL/ELT tasks based on the modelling requirements of the scientific and analytical processes
6. Automate data collection, transformation, life cycling, and disposal tasks
7. Work to ensure well-executed data governance and compliance with data privacy and security standards as required
8. Work with application developers to integrate data platforms with business applications

Knowledge and Technical Skill Requirements

Mandatory

• Proficiency in programming with Python at an advanced level
• Experience working with Jupyter Notebooks and/or Databricks
• Experience with and advanced working knowledge of relational and non-relational databases, SQL scripting and associated query languages
• Proficiency in using Linux operating systems programmatically
• Working knowledge of big data technologies such as Apache Hadoop, Apache Spark, Hive, Presto and Apache Kafka

Preferred

• Working with big data cloud services such as AWS EMR, AWS Glue and AWS MSK for Apache Kafka, AWS Lambda and the various AWS storage solutions
• Working with Elasticsearch and Kibana
• Knowledge of data collection, transmission, and ingestion processes with MQTT, Sparkplug and SCADA systems in the context of Industrial Internet of Things (IIoT) technologies
• Working knowledge of message queues, stream processing and event stores
• Working knowledge of mathematical modelling, data science and data analytics applications and processes

Experience and Qualification Requirements

• Degree or certification in a mathematical, scientific, engineering, or associated interdisciplinary field or equivalent experience
• 2 years of professional data engineering experience

Munyongo is an industrial, financial, data and software engineering company with additional expertise in mathematical modelling and operations research. We believe that science, technology, engineering, and mathematics, can play a big role in securing the futures of our society and environment; we offer our knowledge and skills to endeavours working to do the same.

Munyongo has the potential to be an interesting place to work and perhaps even build a career. The projects in our portfolio are diverse in application yet similar in the underlying knowledge and capabilities required to execute them well.