HDP Developer Java

Our classes are always live and instructor led from our Exton, PA or EPIC Partner locations. Springhouse AnywhereLive options require Internet Access. Select classes are Guaranteed to Run (GTR). View our complete schedule policies.

 

 

 

 

Overview

​This advanced course provides Java programmers a deep-dive into Hadoop application development. Students will learn how to design and develop efficient and effective MapReduce applications for Hadoop using the Hortonworks Data Platform, including how to implement combiners, partitioners, secondary sorts, custom input and output formats, joining large datasets, unit testing, and developing UDFs for Pig and Hive. Labs are run on a 7-node cluster running in a virtual machine that students can keep for use after the training.

Intended Audience

​Experienced Java software engineers who need to develop Java MapReduce applications for Hadoop.


At Completion

After attending class, students will be able to:

  • Describe Hadoop 2 and the Hadoop Distributed File System
  • Describe the YARN framework
  • Develop and run a Java MapReduce application on YARN
  • Use combiners and in-map aggregation
  • Write a custom partitioner to avoid data skew on reducers
  • Perform a secondary sort
  • Recognize use cases for built-in input and output formats
  • Write a custom MapReduce input and output format
  • Optimize a MapReduce job
  • Configure MapReduce to optimize mappers and reducers
  • Develop a custom RawComparator class
  • Distribute files as LocalResources
  • Describe and perform join techniques in Hadoop
  • Perform unit tests using the UnitMR API
  • Describe the basic architecture of HBase
  • Write an HBase MapReduce application
  • List use cases for Pig and Hive
  • Write a simple Pig script to explore and transform big data
  • Write a Pig UDF (User-Defined Function) in Java
  • Write a Hive UDF in Java
  • Use JobControl class to create a MapReduce workflow
  • Use Oozie to define and schedule workflows

Prerequisites

​Students must have experience developing Java applications and using a Java IDE. Labs are completed using the Eclipse IDE and Gradle. No prior Hadoop knowledge is required.


Exams & Certifications


Materials

  • ​50% Lecture/Discussion
  • 50% Hands-on Labs

Course Outline

​Hands-On Labs

  • Configuring a Hadoop Development Environment
  • Putting data into HDFS using Java
  • Write a distributed grep MapReduce application
  • Write an inverted index MapReduce application
  • Configure and use a combiner
  • Writing custom combiners and partitioners
  • Globally sort output using the TotalOrderPartitioner
  • Writing a MapReduce job to sort data using a composite key
  • Writing a custom InputFormat class
  • Writing a custom OutputFormat class
  • Compute a simple moving average of stock price data
  • Use data compression
  • Define a RawComparator
  • Perform a map-side join
  • Using a Bloom filter
  • Unit testing a MapReduce job
  • Importing data into HBase
  • Writing an HBase MapReduce job
  • Writing User-Defined Pig and Hive functions
  • Defining an Oozie workflow

 

 

HDP Developer Javahttp://springhouse.com/course-catalog/HW HDP JHDP Developer Java

Get More Information
Name:

Phone:  

Email:  

Comments:

Help us prove you're not a robot:
 

 ‭(Hidden)‬ Catalog-Item Reuse

Microsoft Gold Partner

PMI R.E.P.

AXELOS Limited

The Microsoft Gold CPLS logo is a mark of Microsoft, Inc.

The PMI R.E.P. logo is a mark of the Project Management Institute, Inc.

ITIL® is a registered trade mark of AXELOS Limited.
IT Infrastructure Library® is a registered trade mark of AXELOS Limited
The Swirl logo™ is a registered trade mark of AXELOS Limited
Accredited course material is property of ITSM Academy.

Connect with us

Springhouse Education & Consulting Services

Corporate HQ:Eagleview Corporate Park
707 Eagleview Boulevard
Suite 207
Exton, PA 19341

610-321-3500 - info@springhouse.com