Big Data Engineer

About the Role

Key Responsibilities

• Design, develop, and implement data processing solutions using HDFS, Hive, and Spark
• Ensure solutions are highly usable, scalable, and maintainable
• Understand the impacts of data layer performance factors and collaborate with the Data Architect to implement mitigating physical modeling solutions
• Perform code and design reviews to ensure performance, maintainability, and standards
• Work with the Data Architect to define data security roles, groups, and policies
• Enforce Hadoop design standards, reusable objects, tools, best practices, and related development methodologies
• Tune dataset design for optimal performance
• Develop, execute, and troubleshoot complex report SQL
• Lead moderately complex initiatives and deliverables within technical domain environments
• Collaborate and consult with peers and colleagues to resolve technical challenges and achieve goals

Requirements

• 5+ years of experience in data processing and big data technologies
• Experience with HDFS, Hive, and Spark
• Knowledge of data security roles, groups, and policies
• Ability to perform code and design reviews
• Experience in tuning dataset design for performance
• Proficiency in developing and troubleshooting complex SQL reports
• Strong collaboration and communication skills