ClassNotFoundException:HoodieInputFormat

[daikon12]

hi , When I followed the steps of quickstart (https://uber.github.io/hudi/quickstart.html ) ,I have a problem：

and I found HoodieInputFormat.class belongs to this jar:

So I introduced this dependency in run_sync_tool.sh , but It did not work.

Is there any good solution to this problem ? Thank you very much

[vinothchandar]

Hi.. is this from running the sync tool? Can you throw some light on your setup?

[vinothchandar]

@daikon12 do you still have this issue?

[daikon12]

@daikon12 do you still have this issue?
Sorry for the late reply . Yes ， It still does not work . Would you like to offer some ideas

[vinothchandar]

It cannot find the HoodieInputFormat which is contained in the hoodie-hadoop-mr-bundle. Did you add the hoodie-hadoop-mr-hundle jar to your hive server?

We can also move this discussion to the dev mailing list, so we can respond quicker..
http://hudi.incubator.apache.org/community.html

[daikon12]

@vinothchandar thx your idea。
But I imported the package manually ，The detailed commands are shown below：

But It doesn't seem to work。

[vinothchandar]

I think we removed the suffix in 0.4.5

All you need is the hive bundle jar. It has the mr stuff internally

$ ls -l packaging/hoodie-hive-bundle/target/
total 101144
drwxr-xr-x  3 vinoth  xxxx        96 Mar 15 11:18 classes
drwxr-xr-x  3 vinoth  xxxx        96 Mar 15 11:18 generated-sources
-rw-r--r--  1 vinoth  xxxx  50725597 Mar 15 11:18 hoodie-hive-bundle-0.4.6-SNAPSHOT.jar
drwxr-xr-x  3 vinoth  xxxx        96 Mar 15 11:18 maven-archiver
drwxr-xr-x  3 vinoth  xxxx        96 Mar 15 11:18 maven-status
-rw-r--r--  1 vinoth  xxxx      4220 Mar 15 11:18 original-hoodie-hive-bundle-0.4.6-SNAPSHOT.jar
-rw-r--r--  1 vinoth  xxxx       971 Mar 15 11:18 rat.txt

I will double check and update the docs..

Can you try running with just the packaging/hoodie-hive-bundle/target/hoodie-hive-bundle-0.4.6-SNAPSHOT.jar off master

[vinothchandar]

Btw, you can use the https://github.com/apache/incubator-hudi/blob/master/hoodie-hive/run_sync_tool.sh tool to run it instead of doing it by hand?

[daikon12]

Btw, you can use the https://github.com/apache/incubator-hudi/blob/master/hoodie-hive/run_sync_tool.sh tool to run it instead of doing it by hand?
When I run this script,The program output looks like this :

Maybe there is a colon missing .

[daikon12]

Btw, you can use the https://github.com/apache/incubator-hudi/blob/master/hoodie-hive/run_sync_tool.sh tool to run it instead of doing it by hand?

So I runit manually again and it works fine until I got the following error:

I need to import many such jars manually

[vinothchandar]

You mean the run_sync_tool.sh gives you this error? Thats odd. Are the Hive jars themselves there? antlr should be provided by Hive I think?
@bvaradar as well to take a look..

[bvaradar]

@daikon12 :

The run_hive_sync tool failed because there were multiple standalone jdbc jars found in your hive installation. We have not seen this case in our installations. I have created a PR (#609) to handle this case. Can you please apply this patch, try this out and let us know if this solved the issue.

Thanks,
Balaji.V

[Achuth17]

I faced the antlr recognition exception as well, Fixed it by adding a few jars (like the antlr jar) manually. Just thinking out loud, Why can't we pick all the jars from "{HIVE_HOME}/lib/" instead of selectively picking?

[bvaradar]

@Achuth17 : This was done on purpose. There are lot of different hadoop/hive environments out there. With Hudi having to interoperate across different query engines, it was practical to choose a set of hadoop/hive releases for which we can certify different hoodie tools/jars. As part of this certification, we found that blindly picking all jars results in package conflicts with this ones present in the installations.

[bvaradar]

@daikon12 : Is run_hive_sync tool working fine now ?

[daikon12]

@daikon12 : Is run_hive_sync tool working fine now ?
Thank you @vinothchandar @bvaradar You inspire me , This problem is probably caused by my version of hadoop and hive , Now I only have one client of parcels way . I will try another version later to see if it works.

[bvaradar]

Closing due to inactivity