Azure Databricks Engineer Innover Infotech
Innover Infotech
Office Location
Full Time
Experience: 5 - 5 years required
Pay:
Salary Information not included
Type: Full Time
Location: Hyderabad
Skills: big data, Data Engineering, Python, Scala, SQL, azure devops, Distributed Systems, Data Modeling, data warehousing, Performance tuning, optimization, Power Bi, Data Visualization, azure databricks, Analytics Solutions, Etl Pipelines, data architectures, Azure Cloud services, ETL, Data Pipeline Frameworks, Delta Lake, Parquet Formats, CICD pipelines, Big Data architecture, Problemsolving, RealTime Data Streaming, Machine learning frameworks
About Innover Infotech
Job Description
You are an experienced Azure Databricks Engineer who will be responsible for designing, developing, and maintaining scalable data pipelines and supporting data infrastructure in an Azure cloud environment. Your key responsibilities will include designing ETL pipelines using Azure Databricks, building robust data architectures on Azure, collaborating with stakeholders to define data requirements, optimizing data pipelines for performance and reliability, implementing data transformations and cleansing processes, managing Databricks clusters, and leveraging Azure services for data orchestration and storage. You must possess 5-10 years of experience in data engineering or a related field with extensive hands-on experience in Azure Databricks and Apache Spark. Strong knowledge of Azure cloud services such as Azure Data Lake, Data Factory, Azure SQL, and Azure Synapse Analytics is required. Experience with Python, Scala, or SQL for data manipulation, ETL frameworks, Delta Lake, Parquet formats, Azure DevOps, CI/CD pipelines, big data architecture, and distributed systems is essential. Knowledge of data modeling, performance tuning, and optimization of big data solutions is expected, along with problem-solving skills and the ability to work in a collaborative environment. Preferred qualifications include experience with real-time data streaming tools, Azure certifications, machine learning frameworks, integration with Databricks, and data visualization tools like Power BI. A bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field is required for this role.,