# Data Engineer
A Data Engineer is responsible for implementing, building, testing, and maintaining the infrastructure and architecture required for the efficient and reliable processing of various types of data. The selected candidate works with Data Analysts, Data Scientists, and other stakeholders to understand data requirements, identify data sources, and implement the necessary processes for data transformation and loading into target systems. They are also responsible for ensuring the integrity, availability, and security of the data throughout its lifecycle, and for continuously optimizing the data pipelines to meet evolving business needs.
## Duties & Role
- Strong understanding of data engineering concepts, including data warehousing, ETL/ELT processes, and data governance
- Experience in planning and maintaining data lakes and pipeline processes, both batch-wise and (near) real-time
- Experience using data engineering tools to support development of data science, data analytics, and AI applications for both structured and unstructured data
- Experience with monitoring and logging tools to ensure the reliability and performance of data pipelines and AI systems
- Solid understanding of containerization and orchestration tools such as Docker and Kubernetes
- Experience with cloud platforms such as AWS, Google Cloud, or Azure
- Solid understanding of good software development principles such as version control, CI/CD, and unit and functional testing
- Strong proficiency in Python for building and integrating various data engineering components
- Solid understanding of data security best practices, including data encryption, access controls, authentication and authorization mechanisms, and compliance requirements