Data Engineer

Job Title: Data Engineer

• Experience: 5–10 Years
• Location: Pan India
• Joining Time: Immediate to 15 Days Maximum

About The Role:

Indus Net Technologies is looking for an experienced Data Engineer to design, build, and maintain robust data pipelines leveraging Databricks and Azure Cloud Services. The ideal candidate will have hands-on expertise in processing large-scale IoT sensor data from manufacturing equipment, ensuring data quality, integrity, and high performance across multi-plant operations.

Key Responsibilities:

• Design and implement ETL/ELT pipelines using Databricks and PySpark.
• Process and transform real-time IoT sensor data from multiple manufacturing plants.
• Manage data storage, versioning, and optimization using Delta Lake.
• Integrate data from diverse sources (SQL Server, JSON, CSV, APIs, etc.).
• Write and optimize complex SQL queries and stored procedures for performance.
• Implement data quality monitoring, testing, and CI/CD pipelines using Azure DevOps.
• Collaborate closely with manufacturing and analytics teams to deliver actionable IoT data insights.

Top 4 Mandatory Skills:

• Databricks & Apache Spark (PySpark) – Minimum 3 years of hands-on experience
• Azure Cloud Services – Experience with ADF, Blob Storage, and DevOps
• Python – Advanced proficiency with OOP concepts, pandas, and robust error handling
• Delta Lake – Strong knowledge of data versioning, time travel, and optimization

Good-To-Have Skills:

• Advanced SQL and stored procedure development
• Experience in JSON data processing and schema handling
• Git-based version control and CI/CD integration using Azure DevOps
• Exposure to manufacturing or industrial IoT data patterns

Preferred Domain Knowledge:

• Industrial IoT and time-series data management
• Machine data and equipment performance monitoring
• Multi-plant data integration and central data architecture
• Data quality frameworks, downtime, and defect analytics

Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or a related field
• 4–6 years of proven experience in Data Engineering
• 2+ years of strong exposure to Databricks/Spark ecosystem
• Azure Data Engineer Associate or Databricks certification is preferred

Soft Skills:

Excellent communication skills in English (written and verbal)

Strong analytical and problem-solving mindset

Collaborative and proactive approach with cross-functional teams

High attention to detail and commitment to data accuracy