Delimited file parallel extraction.

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ramakrishna459
Participant
Posts: 37
Joined: Tue Mar 26, 2013 12:02 pm
Location: INDIA

Delimited file parallel extraction.

Post by ramakrishna459 »

Can we extract the delimited text file using sequential file in parallel mode?
I know we can extract fixed width files.If we couldnt extract the delimited file paralley...what constraints doesnt alow to extract parallely.

Thanks
Rk
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You can. No constraints, although it's slightly more efficient when a fixed-length format is available.

The issue is that, with delimited files, the reader has to approximate the cutoff locations for each Nth of the file, and then scan for the next line terminator. With fixed width this is a simple byte-offset calculation.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
stuartjvnorton
Participant
Posts: 527
Joined: Thu Apr 19, 2007 1:25 am
Location: Melbourne

Post by stuartjvnorton »

Enlightenment lies within, my friend...
http://publibfp.boulder.ibm.com/epubs/pdf/c1938270.pdf
Post Reply