DSXchange

I have three files:
- first file:
0,data
1,data
2,data
etc
-second file:
0,data
0,data
...
1,data
1,data
...
etc
-third file:
0,data
0,data
...
1,data
1,data
1,data
etc

And I need to merge that files by taking for each record from first file, all records with the same key from the second file and after that, all records from third file with the same key. In output file I need only data without keys (ids). How to do that?
Thanks in advance.

Best regards,
Piotrek

Your question has the answer. User Merge stage.

Sainath.Srinivasan wrote:Your question has the answer. User Merge stage.

I don't think so. I tried doing it by merge stage, but I didn't get the expected output. As I know merge will take data from first file and merge it with every record in the second with the same key. But I need only once to put data from first file, then only data from second file and then from third file. And so on for each record from first file.
Correct me if I am wrong about merge stage.

Concatenate the 3 files and then sort.

Sainath.Srinivasan wrote:Concatenate the 3 files and then sort.

But in output file there would be no ID field. ID field in source files is only for merging data. So it would be impossible to sort.

Hi,

If you dont want the ID field in the ouput, pass your merged output thru a transformer and eliminate the ID field in it and then write to your final output.

How can I concatenate files? As I said before merge doesn't solve my problem.

Hi

Please give me a sample output of the merge file. I am not sure whether u want the merged file to look as:

$ cat merge
0,data0 0,data0 0,data0
1,data1 1,data1
2,data2

$ cat file1
0,data0
1,data1
2,data2

$ cat file2
0,data0
1,data1

$ cat file3
0,data0

$ paste file1 file2 file3 > merge


thanks
Pramod

In this example I need to get this output:
file1.data0
file2.data0
file3.data0
file1.data1
file2.data1
file1.data2

I would like to do it in DS job.