Splitting One File to Multiple Files

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Yea, but the duplicates need to be removed plus some more manipulation.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Ultramundane
Participant
Posts: 407
Joined: Mon Jun 27, 2005 8:54 am
Location: Walker, Michigan
Contact:

Post by Ultramundane »

I did not see any mention of duplicates.

Can you tell me the additional manipulation. I ran the job with the sample data and it works.

I must be missing something. :?
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

No i mean the product (first column) is being repeated, need to get only unique values to build file names.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Ultramundane
Participant
Posts: 407
Joined: Mon Jun 27, 2005 8:54 am
Location: Walker, Michigan
Contact:

Post by Ultramundane »

Awk keeps track of the open files and once opened it will only append to the files. Using this method you do not need to get the unique filenames. Awk will overwrite/create when opened (used first time) and will append for the rest.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

120267 wrote:Ray,

The count of the products are not Defined, It is dynamic.If we get 60 products we have to split as 60 files with product name as the file name.We may get more than 100 products also.Is there any solution with out implementing using loop.
I answered the question that was asked.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Ultramundane wrote:Awk keeps track of the open files and once opened it will only append to the files. Using this method you do not need to get the unique filenames. Awk will overwrite/create when opened (used first time) and will append for the rest.
O yea thats right. Just needs testing it. Nice...
Last edited by DSguru2B on Wed Dec 20, 2006 2:49 pm, edited 1 time in total.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
Ultramundane
Participant
Posts: 407
Joined: Mon Jun 27, 2005 8:54 am
Location: Walker, Michigan
Contact:

Post by Ultramundane »

I just created 500 products in random order and many with different codes and in random order it worked. I got multiple entries and the files got created.

Sorry I am not premium member yet and missed the beef of your message.
Post Reply