NJ Employer Sr. Hadoop Data Administrator in Franklin Lakes, New Jersey
POSITION SUMMARY We are looking for an experienced Hadoop Administrator who will be responsible for administering the Hadoop cluster, apply new versions and patches through Ambari. Assist in implementing security for Hadoop big projects. Assist Build team with Environment requirements Design and Develop new processes for better maintenance of the environments. Help Application and Operations team to troubleshoot performance and configuration issues. Participate in the on-call rotation. He/She will be competent to complete most of the Big Data ecosystem Admin and design tasks independently and should be able to work with little guidance. ESSENTIAL FUNCTIONS Responsible for implementation and ongoing administration of Hadoop infrastructure. Aligning with the systems engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments. Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users. Cluster maintenance as well as creation and removal of nodes Performance tuning of Hadoop clusters and Hadoop MapReduce routines. Screen Hadoop cluster job performances and capacity planning Monitor Hadoop cluster connectivity and security * Manage and review Hadoop log files. * File system management and monitoring. * HDFS support and maintenance. Collaborating with application teams to install operating system and Hadoop updates, patches, version upgrades when required. Develop new processes for better maintenance of the environments. Setup best practices for monitoring the cluster * Implement security for Hadoop big data. Help Application and Operations team to troubleshoot the performance issues. * Point of Contact for Vendor escalation. Participate in the on-call rotation to support a 7x24x365 for Production support. Mentor Junior developers on the team regarding better ways for existing development (performance tuning) and the new Hadoop ecosystem tools Point of Contact for Vendor escalation * NoSQL Environment responsibilities * Software installation and configuration * Database backup and recovery * Database connectivity and security * Performance monitoring and tuning * Disk space management * Software patches and upgrades * Automate manual tasks * Process Support * Develop Infrastructure documentation. * Responsible for code migrations from Dev. to QA to Production * Providing operational instructions for deployments. * Executes and provides feedback for operational policies, procedure, processes and standards * Support any process that needs attention in Production environment. QUALIFICATIONS Bachelor s degree and 8+ years of experience required. Experience in working with Linux platform and setting up HA. Experience with NOSQL data bases a plus. In depth understanding of Linux, Knowledge of Java. In depth knowledge of Hortonworks Hadoop ecosystem tools, Kerberos, Ranger, Hadoop Authentication/Authorization, NoSQL databases. NoSQL data modeling techniques and map reduce design patterns *LI-KM2.