Azure Data Engineering - Databricks
Area of expertise:
Job DescriptionAs a Data Engineering Consultant at Avanade, you will use comprehensive modern data engineer techniques and methods with Advanced Analytics to support business decisions for Avanade clients. You can collect, aggregate, and analyze structured/unstructured data from multiple internal and external sources and patterns, insights, and trends to decision-makers. Your goal is to support the use of data-driven insights to help our clients achieve business outcomes and objectives. In this role, you will collect, aggregate, store, and reconcile data in support of Client business decisions. You will help design and build data pipelines, data streams, reporting tools, information dashboards, data service APIs, data generators, and other end-user information portals and insight tools. You will be a critical part of the data supply chain, ensuring that stakeholders can access and manipulate data for routine and ad hoc analysis to drive business outcomes using Advanced Analytics.
- Strong experience as an Azure Data Engineer and must have Azure Databricks experience.
- Demonstrated experience of turning business use cases and requirements into technical solutions.
- Expert level understanding on Azure Data Factory, Azure Data Lake, Azure SQL DW, and Azure SQL, Azure App Service is required.
- Experience in business processing mapping of data and analytics solutions.
- Ability to conduct data profiling, cataloging, and mapping for technical design and construction of technical data flows.
- The ability to apply such methods to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration.
- Azure IoT, Azure HDInsight + Spark, Azure Cosmos DB, Azure Stream Analytics is a plus.
- Knowledge of Python is a plus.
- Experience preparing data for Data Science and Machine Learning.
- Experience preparing data for use in Azure Machine Learning and/or Azure Databricks is a plus.
- Demonstrated experience preparing data and building data pipelines for AI Use Cases (text, voice, image, etc.…).
- Designing and building Data Pipelines using streams of IoT data.
- Knowledge of Lambda and Kappa architecture patterns.
- Knowledge of Master Data Management (MDM) and Data Quality tools and processes
- Strong team collaboration and experience working with remote teams.
- Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code fundamentals.
- Working experience with Visual Studio, PowerShell Scripting, and ARM templates.
- Your future colleagues will help you with questions and challenges. Moreover, there is a pleasant, informal working atmosphere;
- Knowledge development is our focus, and we ensure this both by offering formal training opportunities as well as informal evenings where peers share their knowledge;
- With your passion, you can use the latest Microsoft technology to make a difference for our customers in the Enterprise market;
- We have a strong focus on the career and development opportunities of all our people. Everyone has a career advisor who will work with you to create a plan and guide you in choosing from the many training opportunities we offer;
- We offer the opportunity to be socially involved during workhours and to dedicate yourself to a good cause;
- We have very attractive benefits, not unimportant to mention of course :-): lease car, bonus scheme, phone and laptop, good pension plan, possibility to work part-time (min 32 hours) and many other extras.
Share this job:
Avanade leads in providing innovative digital services, business solutions and design-led experiences for its clients, delivered through the power of people and the Microsoft ecosystem. Our professionals combine technology, business and industry expertise to build and deploy solutions to realize results for clients and their customers. Avanade has 29,000 digitally connected people across 23 countries, bringing clients the best thinking through a collaborative culture that honors diversity and reflects the communities in which we operate. We welcome all, and seek talented individuals who can bring their whole self to work, build inclusive teams and encourage diversity inside and outside the organization. Majority owned by Accenture, Avanade was founded in 2000 by Accenture LLP and Microsoft Corporation. Learn more at www.avanade.com.