DSXchange

how to read multiple stages in datastage one by one

my req is like i have 3 input file stages with me , where i want to start reading the second input stage only when the 1st get over same for the 3rd when 2nd get over...

is this possible in data stage to di it in same job.

No. It might help to know what's in these three input stages, however, to determine a best course of action for you.

input stages are seqfile stages and the data is normal business data ,i just want to check wheater it is possible to read the files one by one .
when ever i m trying to do the testing every time stages start reading on same time ... do i need to setup any constarints to achieve this result.

No, they will always run in parallel. Again, why three separate stages? Is the metadata different? Can you not read all three in a single stage using a File Pattern?

Even that may be in parallel.

You need a single File property and a Filter Command to cat the three files together.

Assuming you can. We still don't know the reasoning behind 'needing' three separate stages or the serial processing.

Assuming you need to do some data validation of each file, and that's the reason you can't combined 3 files into 1 (using cat command etc.) .

you can have a single job with multi instance, to append into a single file, from the three(or more) instance of job.

thanks for the valuable suggestions....

no the input meta is different but i need to combine the output in a single file with the common columns from the input files.

so is it not possible in a single job to read one after the other...???

Server job it is certainly possible and easy to implement.

Parallel , possible, tricky. and probably not very useful.

i don't have PX environment now so can't test.
try this

take the last records from 1st file as refrence input to the second file using lookup stage, so that the second file will wait untill it get the last record from 1st file.

thanks keshav,
yes u r right by this way we can go on...

can u please tell us how it is possible in Server job....

U is not posting in this forum today. And certainly not in this thread. (Though I note that U has made a post today.)

The second person personal pronoun in English is spelled "you". DSXchange is not a mobile telephone nor is there any requirement to limit your messages to 72 or 144 characters. Please maintain a professional standard of written English on DSXchange, not least because it helps those whose first language is not English.

Read 1st file, and write into a sequential file(file_1), and in the output of the file_1 read the 2nd file, and append to file_1 , again in the output read 3rd file and append to File_1

Thanks keshav,

Yes you are right by this way we can go on...

Could you please tell us how it is possible in Server job....?


thanks for the guiding me...

:D

hellboy wrote:Could you please tell us how it is possible in Server job....?

Server job questions really belong in the server job forum.

This helps future searchers.