Hands-on experience in data integration/ETL development.
Strong hands-on experience with SSIS – package design, control flow, data flow, error handling, and deployment.
Strong hands-on experience with Apache Kafka – producers, consumers, brokers, topics, partitions, Kafka Connect, and Schema Registry.
Proficiency in SQL and relational databases (SQL Server; exposure to Oracle/PostgreSQL/MySQL is a plus).
Experience with data modeling, data warehousing concepts, and ETL/ELT design patterns.
Familiarity with message serialization formats (Avro, JSON, Protobuf).
Experience integrating data across on-premise and cloud environments (Azure/AWS preferred).
Understanding of CDC (Change Data Capture) tools and techniques.
Strong debugging, performance tuning, and troubleshooting skills.
Good understanding of data governance, security, and compliance practices in enterprise environments.
Excellent communication and stakeholder management skills.
Experience with Azure Data Factory, Databricks, or AWS Glue.
Exposure to Confluent Kafka or managed Kafka services.
Knowledge of Python or .NET (C#) for custom integration scripting.
Experience with CI/CD pipelines for data engineering deployments.
Familiarity with monitoring tools (e.g., Grafana, Prometheus, Splunk).
Relevant certifications in Kafka, SQL Server, or Cloud platforms.
Qualifications:
Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Job Description:
Design, develop, and maintain robust ETL pipelines using SSIS for batch data integration across enterprise systems.
Build and manage real-time data streaming pipelines using Apache Kafka, including producers, consumers, topics, and Kafka Connect.
Architect end-to-end data integration solutions that combine batch (SSIS) and streaming (Kafka) approaches.
Collaborate with data architects, business analysts, and application teams to gather integration requirements and translate them into technical designs.
Optimize existing ETL/ELT workflows for performance, scalability, and reliability.
Implement data quality checks, error handling, logging, and monitoring across integration pipelines.
Manage schema design, topic partitioning, and data serialization (Avro/JSON/Protobuf) within Kafka ecosystems.
Work closely with DBAs and infrastructure teams to ensure integration solutions align with database and network architecture.
Troubleshoot production issues related to data pipelines, latency, and throughput.
Document technical designs, data flow diagrams, and integration standards.
Mentor junior team members and enforce best practices in data integration development.
Ensure compliance with data governance, security, and audit requirements.