Cloud Data Engineer

JOB SUMMARY

Responsible for designing and implementing robust data pipelines. Must maintain high data quality and security in cloud environments, automating tasks with Python and Bash scripting, and continuously research new technologies to enhance scalability. This role focuses on the development and implementation of data engineering solutions that enhance data warehousing capabilities and maintain legacy and new systems.

DUTIES & RESPONSIBILITIES

• Design and implement data pipelines to move and transform data from various sources into cloud-based data warehouses.
• Develop and maintain ETL processes using tools such as Fivetran, Matillion, SSIS, SSAS, SSRS and Snowflake Dataflow on AWS.
• Optimize data storage and retrieval by designing and implementing data partitioning, indexing, and compression strategies.
• Build and maintain data processing infrastructure on cloud platforms such as AWS.
• Collaborate with business and analysts to ensure data quality, accuracy, and availability for analysis and reporting.
• Manage and lead Complex System Design, Integration and Development.
• Monitor and troubleshoot data pipelines and infrastructure to ensure high availability and performance.
• Implement security and access controls to protect sensitive data in cloud-based environments.
• Automate data processing tasks using scripting languages such as Python or Bash
• Research and evaluate new technologies and tools to improve data processing efficiency and scalability.
• Mentor and train junior data engineers and provide technical guidance to cross-functional teams.
• Perform other job-related functions as assigned.

QUALIFICATIONS

DEGREE TYPE:

Bachelor's Degree

FIELD(S) OF STUDY:

Computer Science, Engineering or related field

EXPERIENCE

• Minimum of 4 years’ experience developing, validating, and implementing data warehouses, data systems or cloud-based data solutions.
• Experience in data engineering with a focus on data warehousing and legacy systems.
• Experience with multiple data modeling approaches, SQL, and scripting programming languages like Python and PowerShell.

COMPETENCIES/SKILLS

• Good understanding of OLAP Cubes, Dimensional model.
• Ability to lead and coach team members.
• Strong communication skills and stakeholder engagement abilities; driving results through innovation, process improvement and creativity.
• Able to prioritize and execute tasks in a high-pressure, fast-paced environment.
• Demonstrated understanding of agile methodologies, QA practices and user experience, with strong digital & technical skills.
• Experienced with collaboration tools (Jira, Confluence, Lucid Chart;).
• Strong understanding of databases such as Microsoft SQL Server, Oracle, MySQL, HANA, Snowflake.
• Experience with traditional on-prem virtual and physical infrastructure, as well as cloud-based environments, preferably AWS.
• Excellent understanding of Windows, and Linux based infrastructure.
• Working knowledge of various tools, open-source technologies, and cloud services