Data Platform Engineer Yugen Analytics Pvt Ltd

About Yugen Analytics Pvt Ltd

Founded in 2020 by ex-Silicon Valley, IIT Alumni with prior experience in Tech Consulting and Product Management, Yugen is an early-stage startup in the Data Science and Machine Learning Engineering space. We imagine the future to be shaped by the convergence and simultaneous adoption of Algorithms, Engineering and Operations, and Responsible AI. Our mission, therefore, is to help effectuate and expedite that for our client partners.

Job Description

ROle :- Data Platform Engineer
Experince :- 3-5 Years
Location :- Bangalore (WFH)
NP :- 30 Days max .
Salary:- 25-30LPA
Manadatory Skills:- Airflow, pyspark
No. of Interview Rounds:- 4
Function: Software Engineering → Big Data / DWH / ETL

Data AnalysisData StreamingETLNoSQLAmazon DynamoDBFlinkKafkaRedisSparkSQL
We are looking for Platform Engineers with a deep sense of ownership and a high bias for action. This position is for the Data and ML Platform ay Yugen, where we are trying to build an end-to-end solution to help enterprises

Responsibilities:

Drive insights from their data (Data Platform)
Accelerate the pace of Machine Learning at scale (ML Training, Deployments)
Build innovative and defensible AI workflows (Generative AI, LLMs)
Build highly-scalable and secure data infrastructure that enabled Analytics and Machine Learning (Model training, Model Inference) use cases
Build transformation systems for various data stores such as analytics, logging, application metrics and click stream events.
Building tools and applications that reduce manual efforts and eliminate friction to access data and manage data infrastructure.
Review and influence design and architecture with stability, maintainability, and scale in mind
Identify patterns and provide solutions to a class of problems
Be able to multi-task, prioritize and handle dependencies with minimal oversight.

Requirements:

Good understanding of nuances of distributed systems, scalability, and availability
At least 2 years of experience in Spark/Scala/Java
At least 1 year of experience in Kafka and Flint/Spark Streaming
At least 2 years of experience in Airflow
Experience building ETL pipelines on large data (TB+ scale)
Strong database and storage fundamentals (includes NoSQL DBs/Key value stores)
Experience with cloud deployments (AWS, GCP)
Basic working knowledge of Kubernetes