Page 1 of 1

merging two files with different metadata

Posted: Thu Sep 07, 2006 6:06 am
by vinodhraj
Hi,

Is it possible in Datastage to merge files with different metadata having common key columns?

Thanks

Vinod

Re: merging two files with different metadata

Posted: Thu Sep 07, 2006 6:12 am
by balajisr
vinodhraj wrote:Hi,

Is it possible in Datastage to merge files with different metadata having common key columns?

Thanks

Vinod
Use Merge stage.

Posted: Thu Sep 07, 2006 6:02 pm
by kumar_s
Either you can use merger or lookup based on the volume of data. Since your concertn is on metadata, I guess you are on Concatination, if so just use cat file1 file2 > new filename command in any of the unix command execution option, like Execute command activity or Before/After job subroutine.

why you do that

Posted: Sun Sep 10, 2006 11:50 pm
by changming
cat two file. but why you do that? how do you deal with a file with two parts of different meta data?

Posted: Mon Sep 11, 2006 1:39 am
by vinodhraj
Thanks.

I am working with datastage server jobs, I can't use merge stage in server since I have to mention different key name for the same column.

Is it possible in unix ?

Somehow I have done this using Set Operators in SQL..

I didn't prefer lookup becoz, there is no relation between these tables though the column name were same.

Moreover SET Operators were better in performance wise.

Thanks

Vinod

Posted: Mon Sep 11, 2006 3:43 am
by kumar_s
If there is no relationship between two table, how can you merge two table without cartision product.
Can you explain your need with some example?

Posted: Mon Sep 11, 2006 7:26 am
by vinodhraj
Kumar,

Using Union we can do this i.e if the column name is similar.

Since I am joining with differnent metadata, which ever metadata is mismatching, I will mention them as empty string providing common alias name.

for eg:

table a

name
deptno
sal


table b

name
deptname

select name, deptno,to_char(sal),'' deptname from table a
union all
select name, '' deptno,'' sal,deptname from table b


hope it helps

Vinod

Posted: Mon Sep 11, 2006 4:01 pm
by ray.wurlod
vinodhraj wrote:I am working with datastage server jobs, I can't use merge stage in server since I have to mention different key name for the same column.
Vinod
This is not correct. The column names (when working with sequential files) are totally irrelevant. Adjust the metadata so that you do have a common key column name.

Posted: Mon Sep 11, 2006 7:02 pm
by kumar_s
Using transformer, you can create columns with empty strings to match the metadata. Or use script for the same purpose. 'Cat' the two files.

Posted: Mon Sep 11, 2006 9:44 pm
by ray.wurlod
cat is NOT the same as join.

Posted: Mon Sep 11, 2006 10:30 pm
by DSguru2B
cat is synonymous to union.

Posted: Mon Sep 11, 2006 11:45 pm
by ray.wurlod
Neither is cat synonymous to UNION. A UNION removes duplicates; cat does not. Nor is UNION a join.

Posted: Tue Sep 12, 2006 10:37 am
by DSguru2B
Actually i mean UNION ALL. Cat is synonymous to UNION ALL. And yes my statement above doesnt hold true as UNION does remove duplicates. For some reason i left out the 'ALL'. :oops:

Posted: Tue Sep 12, 2006 4:35 pm
by ray.wurlod
Learn not to rush, grasshopper. :D

Posted: Tue Sep 12, 2006 4:49 pm
by DSguru2B
Well caught. :wink: