Position Overview 

We are seeking an experienced and highly skilled Azure Data Engineer to join our team who is responsible for data engineering, designing, developing, and maintaining the data platform and systems required for data storage, processing, and analysis. 

Responsibilities 

• Design, develop, and implement data solutions using Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Event Hub, and other Azure services. 

• Build and maintain efficient data pipelines and ETL processes to support data integration and analysis. 

• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. 

• Optimize and tune data processes for performance, reliability, and scalability. 

• Ensure data security and compliance with industry standards and regulations. 

• Troubleshoot and resolve data-related issues and discrepancies. 

• Create and maintain documentation for data solutions, including architecture diagrams, data flow diagrams, and technical specifications. 

• Stay up-to-date with the latest Azure technologies and best practices for data engineering. 

Required Skills 

• 8+ years of experience as an Azure Data Engineer or similar role. 

• Strong expertise in Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Event Hub. 

• Proficiency in programming languages such as PySpark, Scala, or SQL. 

• Experience with data modeling, data warehousing, and big data technologies. 

• Knowledge of Delta Lake and other data storage formats. 

• Familiarity with data integration tools and techniques. 

• Excellent problem-solving skills and attention to detail. 

• Knowledge of data governance and data quality best practices. 

• Strong communication and collaboration skills. 

• Certification in Microsoft Azure Data Engineering or related areas is a plus 

• Bachelor’s degree in Computer Science, Information Technology, or a related field. 

Azure DevOps (Data) Engineer  

Position Overview 

We are seeking a skilled and innovative DevOps Engineer to join our team, focused on implementing Continuous Integration and Continuous Deployment (CI/CD) for Azure data services, including Azure Databricks and Azure Data Factory (ADF). This role will play a critical part in modernizing our data operations and ensuring seamless integration and automation within our Azure environment. 

Responsibilities 

• Design, build, and maintain CI/CD pipelines for Azure Databricks and Azure Data Factory. 

• Collaborate with data engineers and software development teams to integrate code changes into the deployment pipeline. 

• Automate deployment processes and workflows to enhance efficiency and reduce manual intervention. 

• Monitor and troubleshoot issues in the CI/CD pipeline, ensuring minimal downtime and quick resolution. 

• Implement best practices for asset bundling and version control within Azure DevOps. 

• Maintain and optimize Azure DevOps environment for scalability and performance. 

• Ensure security and compliance standards are met in all deployment processes. 

• Document processes, procedures, and configurations for CI/CD pipelines and Azure DevOps. 

Skillset 

• Proven experience with Azure DevOps and implementing CI/CD pipelines.

• Proficiency in Databricks Asset Bundles (DABs).

• Strong knowledge of Azure data services, specifically Azure Databricks and Azure Data Factory. 

• Experience with scripting and automation tools (e.g., PowerShell, Python). 

• Experience in implementing selective deployment for ADF data pipelines. 

• Understanding of containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus. 

• Excellent problem-solving skills and ability to troubleshoot complex CI/CD issues. 

• Strong communication skills and ability to collaborate effectively with cross-functional teams. 

• Knowledge of security best practices for CI/CD pipelines and Azure environments. 

• Detail-oriented with a focus on delivering high-quality, reliable automation solutions. 

• Bachelor's degree in Computer Science, Information Technology, or related field. 

• Minimum of 5 years of experience in DevOps or related roles.