Data Engineer (Azure Data Factory, Talend, AWS Glue)
Area of expertise:
- Day-to-day you will:
- Translate business requirements to technical solutions leveraging strong business acumen
- Analyze current business practices, processes and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics PaaS Services
- Support the planning and implementation of data design services, providing sizing and configuration assistance and performing needs assessments.
- Deliver of architectures for transformations and modernizations of enterprise data solutions using Azure cloud data technologies.
- Design and Build Modern Data Pipelines and Data Streams.
- Design and Build Data Service APIs.
- Develop and maintain data warehouse schematics, layouts, architectures and relational/non-relational databases for data access and Advanced Analytics.
- Expose data to end users using PowerBI, Azure API Apps or any other modern visualization platform or experience
- Implement effective metrics and monitoring processes.
- 3-5 years overall experience and at least 2 years working as Data Engineer
- Demonstrated experience of turning business use cases and requirements to technical solutions.
- Experience in business processing mapping of data and analytics solutions.
- Ability to conduct data profiling, cataloging, and mapping for technical design and construction of technical data flows.
- The ability to apply such methods to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration.
- Experienced in Data Transformation using ETL/ELT tools such as AWS Glue, Azure Data Factory, Talend, EAI
- Knowledge in business intelligence tools such as Power BI, Tableau, Qlik, Cognos TM1
- Knowledge of Azure Data Factory, Azure Data Lake, Azure SQL DW, and Azure SQL, Azure App Service is required. Azure IoT, Azure HDInsight + Spark, Azure Cosmos DB, Azure Databricks, Azure Stream Analytics is a plus.
- Experienced in Cloud Data-related tool such as Microsoft Azure, Amazon S3
- Ability to leverage on variety of programming languages & data crawling/processing tools to ensure data reliability, quality & efficiency
- Knowledge in data modelling and database management such as performance tuning of the Enterprise Data Warehouse, Data Mart, and Business Intelligence Reporting environments, and support the integration of those systems with other applications
- Knowledge of Python is a plus.
- Experience preparing data for Data Science and Machine Learning.
- Experience preparing data for use in Azure Machine Learning and/or Azure Databricks is a plus.
- Demonstrated experience preparing data and building data pipelines for AI Use Cases (text, voice, image, etc.…).
- Designing and building Data Pipelines using streams of IOT data.
- Knowledge of Lambda and Kappa architecture patterns.
- Knowledge of Master Data Management (MDM) and Data Quality tools and processes
- Strong team collaboration and experience working with remote teams.
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code fundamentals.
- Working experience with Visual Studio, PowerShell Scripting, and ARM templates.
- Azure certifications (AZ-900, AZ-200 and 201)
Share this job:
Avanade leads in providing innovative digital services, business solutions and design-led experiences for its clients, delivered through the power of people and the Microsoft ecosystem. Our professionals combine technology, business and industry expertise to build and deploy solutions to realize results for clients and their customers. Avanade has 29,000 digitally connected people across 23 countries, bringing clients the best thinking through a collaborative culture that honors diversity and reflects the communities in which we operate. We welcome all, and seek talented individuals who can bring their whole self to work, build inclusive teams and encourage diversity inside and outside the organization. Majority owned by Accenture, Avanade was founded in 2000 by Accenture LLP and Microsoft Corporation. Learn more at www.avanade.com.