Big Data Engineer

About the Role

Key Responsibilities

• Design, develop, and implement data processing solutions using HDFS, Hive, and Spark.
• Ensure solutions are highly usable, scalable, and maintainable.
• Understand the impacts of data layer performance factors and collaborate with the Data Architect to implement mitigating physical modeling solutions.
• Perform code and design reviews to ensure performance, maintainability, and standards.
• Work with the Data Architect to define data security roles, groups, and policies.
• Enforce Hadoop design standards, reusable objects, tools, best practices, and related development methodologies for the organization.
• Partner with Business Stakeholders and developers to ensure deliverables meet business expectations.
• Tune dataset design for optimal performance.
• Develop, execute, and troubleshoot complex report SQL.
• Lead moderately complex initiatives and deliverables within technical domain environments.
• Collaborate and consult with peers and colleagues to resolve technical challenges and achieve goals.

Requirements

• Experience with data processing solutions using HDFS, Hive, and Spark.
• Knowledge of data security roles, groups, and policies.
• Ability to perform code and design reviews for performance and standards.
• Experience in tuning dataset design for performance optimization.
• Proficiency in developing, executing, and troubleshooting complex report SQL.
• Ability to lead technical initiatives and collaborate with teams.