The Oozie Orchestration Framework : From 0 to 1

About Course

• Prerequisites: Working with Oozie requires some basic knowledge of the Hadoop eco-system and running MapReduce jobs

   Oozie is like the formidable, yet super-efficient admin assistant who can get things done for you, if you know how to ask

  formidable, yet super-efficient: Oozie is formidable because it is entirely written in XML, which is hard to debug when things go wrong. However, once you've figured out how to work with it, it's like magic. Complex dependencies, managing a multitude of jobs at different time schedules, managing entire data pipelines are all made easy with Oozie

  get things done for you: Oozie allows you to manage Hadoop jobs as well as Java programs, scripts and any other executable with the same basic set up. It manages your dependencies cleanly and logically. 

  if you know how to ask: Knowing the right configurations parameters which gets the job done, that is the key to mastering Oozie

Topics covered in this course

• Section 1: Introduction
  	Lecture 1	You, This Course and Us	01:38
  Section 2: A Brief Overview Of Oozie
  	Lecture 2	What is Oozie?	11:16
  	Lecture 3	Oozie architectural components	10:45
  Section 3: Oozie Install And Set Up
  	Lecture 4	Installing Oozie on your machine	16:29
  Section 4: Workflows: A Directed Acyclic Graph Of Tasks
  	Lecture 5	Running MapReduce on the command line	04:40
  	Lecture 6	The lifecycle of a Workflow	06:12
  	Lecture 7	Running our first Oozie Workflow MapReduce application	11:15
  	Lecture 8	The job.properties file	08:45
  	Lecture 9	The workflow.xml file	12:06
  	Lecture 10	A Shell action Workflow	07:46
  	Lecture 11	Control nodes, Action nodes and Global configurations within Workflows	09:57
  Section 5: Coordinators: Managing Workflows
  	Lecture 12	Running our first Coordinator application	12:27
  	Lecture 13	A time-triggered Coordinator definition	08:52
  	Lecture 14	Coordinator control mechanisms	07:09
  	Lecture 15	Data availability triggers	10:03
  	Lecture 16	Running a Coordinator which waits for input data	06:11
  	Lecture 17	Coordinator configuration to use data triggers	15:25
  Section 6: Bundles: A Collection Of Coordinators For Data Pipelines
  	Lecture 18	Bundles and why we need them	09:15
  	Lecture 19	The Bundle kick-off time	11:12
  Section 7: Installing Hadoop in a Local Environment
  	Lecture 20	Hadoop Install Modes	08:32
  	Lecture 21	Setup a Virtual Linux Instance (For Windows users)	15:31
  	Lecture 22	Hadoop Standalone mode Install	09:33
  	Lecture 23	Hadoop Pseudo-Distributed mode Install	14:25

What you will get from this course?

• • Install and set up Oozie
  • Configure Workflows to run jobs on Hadoop
  • Configure time-triggered and data-triggered Workflows
  • Configure data pipelines using Bundles

  • Workflow Management: Workflow specifications, Action nodes, Control nodes, Global configuration, real examples with MapReduce and Shell actions which you can run and tweak

  • Time-based and data-based triggers for Workflows: Coordinator specification, Mimicing simple cron jobs, specifying time and data availability triggers for Workflows, dealing with backlog, running time-triggered and data-triggered coordinator actions

  • Data Pipelines using Bundles: Bundle specification, the kick-off time for bundles, running a bundle on Oozie

Who should buy this course?

• • Engineers, analysts and sysadmins who are interested in big data processing on Hadoop
  • Beginners who have no knowledge of the Hadoop eco-system can not purchase this course
  • Students should have basic knowledge of the Hadoop eco-system and should be able to run MapReduce jobs on Hadoop