Hi there,
Just as background, I'm running HDP 2.5 on a set of 7 VMs. I know 2.5 isn't supported right now but I can't imagine this error is specific to 2.5.
I set up a transfer job to move one Postgres table into Hive. The jobs spins for a while and then get stuck with the following error:
2016-10-19 15:41:24,202 FATAL [IPC Server handler 8 on 34487] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1476904741845_0006_m_000000_0 - exited : java.io.IOException: Split class org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not found
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2114)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)
... 7 more
2016-10-19 15:41:24,202 INFO [IPC Server handler 8 on 34487] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1476904741845_0006_m_000000_0: Error: java.io.IOException: Split class org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not found
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2114)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)
... 7 more
2016-10-19 15:41:24,203 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1476904741845_0006_m_000000_0: Error: java.io.IOException: Split class org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not found
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:426)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2114)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)
... 7 more
Here's the relevant snippet from Toad's log file:
2016-10-19 15:41:41 ERROR AbstractYARNOoozieSqoopJob:151 - Oozie job error
java.lang.Exception: Oozie job 0000001-161019153618324-oozie-oozi-W ends with KILLED.
at com.dell.tfh.control.service.transfer.yarn.AbstractYARNOoozieSqoopJob.waitUntilJobIsFinished(AbstractYARNOoozieSqoopJob.java:235)
at com.dell.tfh.control.service.transfer.yarn.AbstractYARNOoozieSqoopJob.run(AbstractYARNOoozieSqoopJob.java:142)
at com.dell.tfh.control.service.transfer.yarn.YARNOozieSqoopImportJob.run(YARNOozieSqoopImportJob.java:183)
at com.dell.tfh.control.service.TransferService.runTransferJob(TransferService.java:300)
at com.dell.tfh.control.service.TransferService.executeTransferItem(TransferService.java:284)
at com.dell.tfh.control.service.TransferService.executeItem(TransferService.java:239)
at com.dell.tfh.control.service.TransferService.execute(TransferService.java:155)
at com.dell.tfh.gui.transfer.handler.TransferExecuteJob.runJob(TransferExecuteJob.java:96)
at com.dell.tfh.gui.commons.jobs.AbstractToadJob.run(AbstractToadJob.java:162)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
As far as I can tell from my limited experience, oozie has the proper shared libs available:
[oozie@<hostname> ~]$ oozie admin -shareliblist
[Available ShareLib]
hive
distcp
mapreduce-streaming
spark
oozie
hcatalog
hive2
sqoop
pig
spark_orig
Any ideas?
Thanks,
JP