Project Description:
Looking to add a Senior Data Engineer [Temporary Contract] to our cross-functional Search product team. This team of data scientists, analysts, data engineers, software engineers and SDETs is focused on improving our search capability and customer search experience. The Senior Data Engineer will use frameworks and tools to perform the ETL and propose abstractions of those methods to aid in solving the problems associated with data ingestion.
Required skills:
Python, NoSQL/MySQL, Redshift, Kinesis, Airflow, AWS, Docker, fluent in English, and Elasticsearch.
*Expert SQL skills.
*4 + years experience with:
***Scaling and optimizing schemas.
***Performance tuning ETL pipelines.
***Building pipelines for processing large amounts of data.
***Proficiency with Python, Scala and other scripting languages.
Experience with:
*MySQL and Redshift.
*NoSQL data stores, methods and approaches.
*Kinesis or other data streaming services.
*Airflow or other pipeline workflow management tools.
*EMR, Spark and ElasticSearch.
*Docker or other container management tools.
*Developing infrastructure as code (IAC).
*Ability to effectively work and communicate with cross-departmental partners and non-technical teams.
Responsibilities:
*Handle data engineering tasks in a team focused on improving search functionality and customer search experience.
*Design, develop, and own ETL pipelines that deliver data with measurable quality.
*Scope, architect, build, release, and maintain data oriented projects, considering performance, stability, and an error-free operation.
*Identify and resolve pipeline issues while discovering opportunities for improvement.
*Architect scalable and reliable solutions to move data across systems from multiple products in nearly real-time.
*Continuously improve our data platform and keep the technology stack current.
*Solve critical issues in complex designs or coding schemes.
*Monitor metrics, analyze data, and partner with other internal teams to solve difficult problems creating a better customer experience.
Nice to Have:
Nice to Have Experience with:
*Segment customer data platform with integration to Braze.
*Terraform.
*Tableau.
*Django.
*Flask.
Contract Details:
*Duration: 3 Months (possibility of extension)
*Location: Remote
*U.S. Working Hours - Must be available 9am-3pm Pacific Time for team meeting and collaboration
Job Type: Temporary
Contract length: 9 months
Pay: R390,00 - R400,00 per hour
Expected hours: 40 per week
