Data Support Engineer
Job Description
KEY ACCOUNTABILITIES:
- The Azure Data Support engineer focuses on data-related tasks in Azure.
- Manage, monitor, and ensure the security and privacy of data to satisfy business needs.
- Monitor real time and batch processes to ensure data accuracy.
- Monitor azure pipelines and troubleshoot where required.
- Enhance existing pipelines, databricks notebooks as and when required.
- Involved in development stages of new pipelines as and when required.
- Troubleshoot pipelines, real time replication jobs and ensuring minimum data lag.
- Available to work on a shift basis to cover monitoring during weekends. (one weekend out of three)
- Act as an ambassador for DP World at all times when working promoting and demonstrating positive behaviours in harmony with DP World's Principles, values and culture ensuring the highest level of safety is applied in all activities understanding and following DP World's Code of Conduct and Ethics policies
- Perform other related duties as assigned
- Responsible for monitoring and enhancing existing data pipelines using Microsoft Stack.
- Responsible for enhancement of existing data platforms.
- Experience with Cloud Platforms such as Azure, AWS , Google Cloud etc.
- Experience with Azure data Lake, Azure data lake Analytics,Azure SQL Database, Azure, Data BricksandAzure SQL Data warehouse.
- Good understanding ofSpark Architectureincluding Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
- Good understanding ofBig Data Hadoopand Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, andKafka(distributed stream-processing) .
- Experience in Database Design and development with Business Intelligence usingSQL Server 2014/2016, Integration Services (SSIS), DTS Packages, SQL Server Analysis Services (SSAS),DAX, OLAP Cubes, Star Schema and Snowflake Schema.
- Monitoring of pipelines inADFand experience with Azure SQL, Blob storage, Azure SQL Data warehouse.
- Experience in a support environment working with real time data replication will be a plus.
QUALIFICATIONS, EXPERIENCE AND SKILLS:
Qualification:
- Bachelor/master's in computer science/IT or equivalent.
- Azure certifications will be an added advantage (Certification in AZ-900 and/or AZ-204, AZ-303, AZ-304 or AZ-400 , DP200 & DP201).
- ITIL certification a plus.
Experience: 5 – 8 Years
Must Have Skills:
- Azure Data lake, Data factory, Azure Databricks
- Azure SQL database, Azure SQL Datawarehouse.
- Hadoop ecosystem.
- Azure analytics services.
- Programming Python, R, Spark SQL
Good to Have Skills:
- MSBI (SSIS, SSAS, SSRS), Oracle, SQL, PL/SQL
- Data Visualization, Power BI
- Data Migration
Job Details
Employment Types:
Full time
Industry:
Courier / Freight / Transportation
Function:
IT
Roles:
ERP / CRM – Support Engineer , Software Engineer / Programmer
Skills:
Data Support Engineer
