Big Data Engineer

We are seeking an experienced Big Data Engineer with expertise in Scala, Spark, and Hadoop to design, develop, and maintain scalable data solutions. The ideal candidate will have a strong background in big data technologies and a proven track record of building and optimizing data pipelines and analytics solutions.

Key Responsibilities:

• Design and Develop: Create and implement scalable and high-performance data pipelines using Scala, Apache Spark, and Hadoop. Develop data models and architecture to support analytics and business intelligence.
• Data Integration: Integrate data from various sources, including structured and unstructured data, and ensure data quality and consistency across the ecosystem.
• Optimization: Optimize data processing workflows and queries to improve performance and reduce latency. Troubleshoot and resolve issues related to data processing and integration.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide solutions that meet their needs.
• Maintenance and Monitoring: Monitor the performance of data pipelines and systems, and implement improvements as necessary. Ensure the reliability and scalability of data solutions.
• Documentation: Document data pipelines, architecture, and processes to ensure clarity and maintainability. Provide support and training to other team members as needed.
• Innovation: Stay up-to-date with the latest developments in big data technologies and practices. Recommend and implement new tools and techniques to enhance our data capabilities.