DS connectivity with Hadoop

mydsworld · Post by **mydsworld** » Thu Jun 04, 2015 12:16 am

Please let me know If DS 11.3 can connect to Hadoop file system (Cloudera 5) and to Hive/HBase tables. If 'Yes' how to configure.

Thanks in advance.

ray.wurlod · Post by **ray.wurlod** » Thu Jun 04, 2015 1:05 am

Yes. Use the HDFS stage.

mydsworld · Post by **mydsworld** » Thu Jun 04, 2015 1:37 am

Hi Ray,
Thanks for your reply. I havn't used it ever, so wondering using HDFS stage I can access files on HDFS. Will I be able to access tables in Hive or HBase using 'HDFS stage' ?

Thanks.

mydsworld · Post by **mydsworld** » Thu Jun 04, 2015 4:45 am

Ray or Others,

Is there any stage called 'HDFS stage' in DS 11.3. I thought it is 'Big Data' File stage to access files sitting on Hadoop.

Curious to know if one can set up the following.

1. ODBC for Hive tables and calling it in Datastage
2. JDBC for HBase tables and calling it in Datastage.

Thanks.

chulett · Post by **chulett** » Thu Jun 04, 2015 6:18 am

The 'HDFS stage' is the Big Data File stage.

mydsworld · Post by **mydsworld** » Thu Jun 04, 2015 6:51 am

Thanks for the clarification. Still curious about the following.

ODBC/JDBC for Hive/HBase tables and calling it in Datastage

chulett · Post by **chulett** » Thu Jun 04, 2015 7:05 am

If those access methods are supported they can be 'called' from DataStage. The JDBC side would take extra shenanigans because, well... Java. You could always ask your official support provider as well.

JPalatianos · Post by **JPalatianos** » Tue Dec 06, 2016 9:39 am

Hi,
I was just curious if you were ever able to configure your connectivity to Cloudera using the "Big Data File Stage"?

We have just been tasked with a similar exercise connecting to Cloudera from our 11.5 DataStage installation.

Thanks - - John

atulgoel · Post by **atulgoel** » Fri Dec 23, 2016 6:26 am

Hi .. just wanted to know if you are able to read the hive tables or hdfs files using Big data file stage...Even I have a similar requirement and doing research on the configuration part.