IOTLink

Mô tả Công việc

Title: Data Engineer

Location: District 1

Main Task:

• Data Quality Assurance: implement frameworks to ensure the accuracy, completeness, and consistency of data. This involves creating and maintaining automated tests to validate data at various stages of the pipeline
• Data Integration: be responsible for integrating data from multiple sources, ensuring that the data is accurate and complete when stored. This involves connecting to external databases, APIs, and other storage solutions
• Data Governance: establish data governance policies to ensure data accuracy, consistency, and compliance with regulations. This includes defining data standards and monitoring adherence to these standards.
• Data Warehousing: They develop and optimize data warehousing solutions, ensuring that data is stored in a way that maintains its integrity and is easily accessible for analysis.
• Monitoring and Maintenance: Continuous monitoring and maintenance of data pipelines are essential to detect and resolve any issues that might affect data integrity. This includes regular audits and updates to the data infrastructure
• Collaboration with Stakeholders: work closely with infrastructure & system team and data analysts to understand their data requirements and provide optimized data sets. This collaboration helps ensure that the data used for analysis is complete and reliable.

Yêu Cầu Công Việc

Qualification

• 2-3 years’ experience working with data as a data engineer.
• Strong SQL query writing skills including ability to address query performance issues.
• This is a technical role with software engineering experience.
• Experience using data integration, transformation, workflow management, and ETL tools.
• Experience with database systems such as MSSQL, Oracle, MySQL, PostgreSQL
• Having knowledge of data cloud such as AWS/ GCP is a big plus.
• Experience identifying the right tools and designing a data warehouse from scratch.
• Experience with data warehouse design including understanding Star schema and other design approaches.
• Programming skills in Python, practical experience with .NET