Reference # 19-00043 Title Big Data Engineer
Location Scottsdale, ARIZONA
Position Type Right to Hire
Experience Level Right to Hire
Start Date / End Date 25-06-2018 --- 28-12-2018
Description Senior Data Engineer

Overall Purpose
This position designs, develops, tests and maintains distributed data processing applications on a Big Data Platform.

Must Have
  • Hadoop
  • Scala (2+ Years)
  • Java (5+ Years)
  • Spark (1+ Years)
  • Kafka (1+ Years)
  • Hbase and/or Solr
  • Real-Time and Batch Architecture

Essential Functions
  • Design, develop, document, test, and debug new and existing software systems and applications.
  • Serve as technical expert on development projects.
  • Drives full development life cycle including requirements analysis and design.
  • Write technical specifications based on conceptual design and stated business requirements.
  • Support, maintain, and document software functionality.
  • Identify and evaluate new technologies for implementation.
  • Analyze code to find causes of errors and revise programs as needed.
  • Drive software design meetings and analyze user needs to determine technical requirements.
  • Consult with end user to prototype, refine, test, and debug programs to meet needs.
  • Complies with all security policies and procedures, to ensure that the highest level of system and data confidentiality, integrity and availability is maintained.
Desired Skills

  • Ability to use Unix/Linux command line programs and create/edit scripts.
  • Able to analyze results of automated deployment scripts and makes modifications.
  • Experience with Spark Map/Reduce distributed processing
  • Able to prepare detail design documents from requirements.
  • Demonstrable understanding of distributed, component based Real-Time and Batch architecture
  • A solid understanding of event driven / asynchronous programming and multi-threading
  • Demonstrable technical design skills
  • Familiar with TDD/BDD.
  • Common Developer Skills
    • Competent (5+ years) Java or Scala
    • Experience with Kafka messaging system
    • Experience with HBase and/or SOLR
    • Competent with SQL or PL/SQL and relational database concepts.
    • Experience with source code control systems (GIT)
    • Experience with secure programming techniques is highly desirable
    • Able to perform code reviews to ensure the code meets the acceptance criteria.