Role
Location
Experience
Location Type
Team
Senior Data Engineer
Job Description
We are seeking a Senior Data Engineer with a minimum of 10 years of experience to join our core product development center in India. You will be a key player in designing, building, and deploying innovative data solutions that help our private equity clients make smarter investment decisions.
This role requires a proven expert in data modeling, SQL, and programming languages like Python and Java. You will lead the development of robust data systems and pipelines using technologies like Azure Data Factory, Apache Airflow, Snowflake, and Azure Synapse Analytics. The ideal candidate has strong problem-solving skills and a collaborative mindset to work with data scientists and architects to ensure the integrity and efficiency of our data systems.
Key Responsibilities
- Data Analysis and Organization: Analyze and organize raw data to derive actionable insights.
- Data Systems and Pipelines: Build and maintain robust data systems and pipelines to ensure seamless data flow.
- Business Needs Assessment: Evaluate business needs and objectives to design appropriate data solutions.
- Trend Interpretation: Interpret trends and patterns to support business decision-making.
- Complex Data Analysis: Conduct complex data analyses and report on findings to stakeholders.
- Data Preparation: Prepare data for prescriptive and predictive modeling.
- Algorithm and Prototype Development: Build algorithms and prototypes to enhance data processing capabilities.
- Data Integration: Combine raw information from various sources to create comprehensive data sets.
- Data Quality Enhancement: Explore and implement ways to improve data quality and reliability.
- Data Acquisition Opportunities: Identify opportunities for new data acquisition to enhance analytics capabilities.
- Analytical Tool Development: Develop analytical tools and programs to support data-driven decision-making.
- Collaboration: Collaborate with data scientists and architects on multiple projects to ensure the success of data initiatives.
Requirements and Skills:
- Previous experience as a data engineer or in a similar role.
- Proven expertise with data models, data mining, and segmentation techniques.
- Hands-on experience with SQL database design.
- Proficiency in programming languages such as Java ,Python and SQL
- Proficiency in databases like Oracle,SQL Server, MySQL, PostGresql for structured data
- Proficiency in Snowflake , Azure Synapse Analytics, Amazon Redshift
- Expert knowledge of Azure Data Factory, Apache Airflow, Blob storage, ADLS, Azure function apps, Azure logic apps, Azure DevOps for CI/CD
- Solid understanding of Data warehouse concepts and Dimensional Modeling
- Must have experience implementing data quality frameworks/tools, security practices ( data encryption, access controls, compliance etc.,)
- Should have good understanding of version control using Github
- Good knowledge using data visualization tools like PowerBI
- Good understanding of Machine Learning and AI frameworks
- Strong numerical and analytical skills.
Education:
- Degree in Computer Science, IT, or a related field; a Master’s degree is a plus.
- Data engineering certification is a plus.
Soft Skills:
- Excellent problem-solving skills.
- Strong communication (oral & written) and collaboration abilities.
- Ability to work effectively in a fast-paced, dynamic environment.
- Ability to guide and mentor junior team members and manage onshore/offshore resources for the successful delivery of the projects
- Excellent time management skills
