Lead Data Engineer - Python

Descrição da empresa

Strider is your hub for finding remote jobs in the US. Top US companies are hiring for full-time, long-term roles on Strider. We make it easy, too! Sign up, build your profile, indicate your interests, and then Strider goes to work to find the perfect opportunity for you.

Our talent team works with you one-on-one to help you showcase your strengths and teach you how to succeed in the international job market. You can also access Strider Benefits which offers exclusive perks and discounts for healthcare, English classes, and more so that you can get perks even when working remotely.

Atividades e Responsabilidades

- Develop effective data pipelines and ETL processes for data lake integration
- Optimize data infrastructure considering data volume, velocity, and variety
- Ensure data architecture performance, reliability, and scalability
- Implement data governance practices to ensure data quality, integrity, and security
- Adopt optimal engineering practices, methodologies, procedures, and technologies
- Work with cross-functional teams to develop data solutions that meet business needs
- Stay updated with data engineering trends
- Share insights with the team and organization
- Effectively communicate technical concepts, solutions, and recommendations to both technical and non-technical stakeholders

Requisitos

Must-haves

- 7+ years of experience in data engineering
- Leadership experience
- AWS experience
- Apache Spark experience
- Experience building scalable data pipelines and ETL processes from scratch
- Strong proficiency with Python
- Proficiency with SQL and relational database technologies
- Expertise in distributed systems, data processing frameworks
- Expertise in data lake and cloud computing platforms
- Deep knowledge of data modeling and data warehousing concepts
- Familiarity with data governance, access controls, security, and compliance principles
- Ability to optimize data pipelines for performance, scalability, and reliability
- Strong problem-solving and analytical skills for complex data engineering challenges
- Excellent communication skills in both spoken and written English
- Bachelor's Degree in Computer Engineering, Computer Science, or equivalent

Nice-to-haves

- Experience with real-time data processing and streaming frameworks
- Knowledge of modern data lake house technologies
- Experience with Docker and Kubernetes
- Experience with Terraform, Airflow, Pandas, PySpark
- Experience with NoSQL databases
- Experience with data visualization tools and data exploration techniques
- Actively participating in the data engineering community
(e.g. making contributions to open-source data engineering projects)