Senior Cloud Data Platform Engineer

Job Overview

Job Description

Senior Cloud Data Platform Engineer needs 5+ years of implementing data applications or data platforms with BigData/Hadoop, Python/Java/Spark full stack, etc
Senior Cloud Data Platform Engineer requires:
• 5+ years of implementing data applications or data platforms with BigData/Hadoop, Python/Java/Spark full stack, etc.
• Hybrid role must be in office 2 days a week,
• Locations; NY,NY; Iselin, NJ; Charlotte, NC
• 5+ year support experience of Big Data technologies in Hadoop ecosystem Hive, HDFS, MapReduce, Spark, Yarn, Kafka, Pig, HBase, Sqoop, Elastic Search, Kerberos.
• 5+ year experience with ETL/ELT tools such as AWS Glue ETL, Talend, Datastage etc.
• 5+ year experience with Orchestration tools such as Airflow, Autosys etc.
• Experience with Jira and Agile methodology is a plus;
• Experience with understanding DevOps or CI/CD process w.r.t to infrastructure management activities
Senior Cloud Data Platform Engineer duties:
• Design, develop and deliver cloud datastore solutions and develop automation pipelines to migrate data sets from On-prem to Cloud platforms. Practice Infrastructure as code to develop automation routines and integration flows to manage state of the datastore platform systems
• Provision secures from start datastores and enable them with required security controls including encryption, masking, certificate/keys rotation etc.
• Collaborates with developers, analysts, various system administrators to identify business requirements in designing efficient datastore solutions and interfaces.
• Identifies and documents all system constraints, implications, and consequences of various proposed system changes.
• Reviews technical documentation to guide system users and to assist with the ongoing operation, maintenance, and development of the system. Evaluates the efficiency and effectiveness of application operations and troubleshooting problems.