Reading Partitioned data from File Connector stage
Posted: Thu Sep 14, 2017 8:30 am
Hi,
I am able to read the data from HDFS files.
the folder structure as below.
/data/projectname/zonename/dbname/tablename/partfilexxxxx*
Question:
How do I read the read if the data is stored in partition on business_effective_date like below
/data/projectname/zonename/dbname/tablename/effective_date=20170915/partfilexxxxx*
/data/projectname/zonename/dbname/tablename/effective_date=20170916/partfilexxxxx*
Should I read them separately or is there a way to read from all the effective_date sub folders at once using file connector stage?
I am able to read the data from HDFS files.
the folder structure as below.
/data/projectname/zonename/dbname/tablename/partfilexxxxx*
Question:
How do I read the read if the data is stored in partition on business_effective_date like below
/data/projectname/zonename/dbname/tablename/effective_date=20170915/partfilexxxxx*
/data/projectname/zonename/dbname/tablename/effective_date=20170916/partfilexxxxx*
Should I read them separately or is there a way to read from all the effective_date sub folders at once using file connector stage?