Can Data set be accessed from any external editor

sudeepmantri · Post by **sudeepmantri** » Tue May 05, 2009 4:56 am

Hi

Have a requirement to access Data set from Java editor.

Could some one provide pointers on how this can be achived?

Regards,
Sudeep

priyadarshikunal · Post by **priyadarshikunal** » Tue May 05, 2009 7:18 am

what kind of access?

if access means reading date then yes only if your editor is smart enough to decode Datastage's internal format. And I don't know any such editor.

If you want to get statistics for datasets then you might be able to do that using datastage functions/executables. But I never used that.

Its documented and also I prefer to use filesets if the intermediate data is needed by any other external program.

sudeepmantri · Post by **sudeepmantri** » Tue May 05, 2009 7:36 am

Basically there are orchadmin utilities by which I can read a dataset from command line or dump it into a file (may be a CSV or something).

I just wanted to know if there's any other option by virtue of which I dont need to dump data into another file and from the command prompt or STDOUT itself I can read the formatted data properly.

P.S. I also need first line column name which is a Property exhibited only by a Sequential file stage.

Sainath.Srinivasan · Post by **Sainath.Srinivasan** » Tue May 05, 2009 10:23 am

Maybe you can use orchadmin and pipe to stdout.

sudeepmantri · Post by **sudeepmantri** » Tue May 05, 2009 9:36 pm

Exactly....But the problem is it wont preserve the formatting of the data and I also need the first line as column name (there are some other requirements too)...Seems these can only be achieved only when I write from dataset to a sequential file..Which basically I cant do at this stage of the game!! :D

ray.wurlod · Post by **ray.wurlod** » Tue May 05, 2009 11:30 pm

A Data Set consists of multiple physical files. On that basis at the very least the answer is "no".

There are no columns, so there are no column headings. The storage schema is in the control file, not in the data files. Therefore the answer is still "no".

Data are in binary format. For example, an int32 occupies precisely four bytes. While some editors can "read" that, the question really is "can you?". Therefore I'm staying with the original "no" answer.

sudeepmantri · Post by **sudeepmantri** » Thu May 07, 2009 9:18 pm

Thanks Ray...Will look for some other optins then..As of now heart and mind both are going for XML files!!

datastage_learner · Post by **datastage_learner** » Thu May 07, 2009 10:28 pm

create a job that reads datasets and gets you either seq. files or xml files as output and then thru. your java prog. u can read those... am I right?

datastage_learner · Post by **datastage_learner** » Thu May 07, 2009 10:30 pm

Moreover datasets are bound by the no. nodes you had used during partitioning or processing... there may be 4 or more such nodes that the dataset is distributed over... so how can you read such files?

ray.wurlod · Post by **ray.wurlod** » Thu May 07, 2009 11:38 pm

datastage_learner wrote:u can read those... am I right?

No.

Whatever Sudeep does will not enable U to read anything.