Senior Data Engineer - GCP

About the Role

Key Responsibilities

• Design, develop, and optimize data pipelines on GCP using BigQuery, Dataflow, and Pub/Sub
• Build and maintain microservices to support data processing and analytics
• Develop real-time and batch data ingestion frameworks using Kafka
• Work with NoSQL databases (e.g., Firestore, Cassandra, MongoDB) to support high-throughput data applications
• Ensure data integrity, governance, and security best practices in the data pipeline
• Optimize performance, scalability, and cost efficiency of data solutions
• Collaborate with data scientists, analysts, and software engineers to deliver end-to-end data products
• Implement monitoring, logging, and alerting for data pipelines and services
• Participate in agile development methodologies, contribute to project planning, and drive project success
• Explore new tools, frameworks, and techniques to improve digital solutions and foster innovation
• Create and maintain technical documentation including system designs, architecture diagrams, and code documentation

Requirements

• 5+ years of overall experience in large-scale software development with at least 2+ years in a technical lead role
• 5+ years of experience in data engineering focusing on microservices-based data solutions
• 5+ years of designing and developing highly scalable Microservices, Webservices, and Rest APIs
• 3+ years of hands-on experience with GCP services like BigQuery, Dataflow, Pub/Sub, and Cloud Storage
• 3+ years of containerization experience with Docker and Kubernetes, and CI/CD pipelines
• 3+ years of experience developing within an agile development environment
• Experience with NoSQL databases and data processing frameworks
• Strong problem-solving skills and ability to work in an agile environment

Nice to Have

• Experience designing and developing in a Test-Driven Development environment
• Proficiency in Python or Java for data processing
• Experience with monitoring and logging tools like Prometheus, Stackdriver, or ELK
• Understanding of distributed computing and data partitioning strategies
• Knowledge of machine learning pipelines and integration with data workflows
• Experience with performance tuning and cost optimization in cloud environments
• Familiarity with Apache Beam or Flink for large-scale data processing
• Experience with Infrastructure as Code tools like Terraform

Qualifications

• Bachelor's degree or equivalent experience (HS diploma + 4 years relevant experience)

Benefits & Perks

• Affordable medical plan options
• 401(k) plan with matching contributions
• Employee stock purchase plan
• No-cost wellness programs including screenings, tobacco cessation, weight management, counseling, and financial coaching
• Paid time off, flexible work schedules, family leave, dependent care resources
• Colleague assistance programs, tuition assistance, retiree medical access
• Comprehensive and competitive pay and benefits package