Big Data QA Engineer Idyllic Services

About Idyllic Services

Job Description

Job Title: Senior QA Data Engineer Location: Bangalore / Pune (Hybrid) Experience: 5+ years Employment Type: Full-time Were currently hiring a Senior QA Data Engineer with strong experience in data testing, automation, and cloud technologies. This role is ideal for someone who enjoys working on complex data pipelines and is passionate about delivering high-quality, reliable data solutions. Responsibilities: Build and maintain automated test scripts for data workflows and ETL pipelines using Python and PySpark Perform data validations using SQL and work with NoSQL databases for test coverage Collaborate with DevOps teams to validate and enhance CI/CD pipelines (preferably using Jenkins) Support testing of data integrations across various AWS services like Glue, Lambda, S3, RDS, and Iceberg Design data validation frameworks to ensure data quality and integrity across the pipeline Monitor data pipeline performance and proactively flag issues or anomalies Contribute to documentation of test plans, strategies, and best practices What Were Looking For: Strong hands-on experience with Python, PySpark, and SQL Background in data pipeline/ETL testing Experience with data testing automation tools and frameworks Familiarity with working on CI/CD tools like Jenkins Understanding of AWS cloud services, or willingness to ramp up quickly Experience validating both relational and NoSQL data stores Nice to Have: Exposure to tools like Airflow or Spark AWS certification (e.g. Developer Associate) is a plus,