Page 1 of 1

replacement for lookup stage

Posted: Fri Mar 09, 2007 9:00 am
by kirankota79
i want to replace the values of 4 columns with other values......so iam using the original file as input to the lookupstage and the new values sequential file as a lookup file to the lookup stage. the file has 10million records. Datastage job is aborting saying that the space is not sufficient. i understand that since it has millions of records ......lookup stage dont allow it. but can anybody tell me the replacement for it...so that i can proceed...


thanks in advance......

Posted: Fri Mar 09, 2007 9:09 am
by trobinson
Join Stage

Posted: Fri Mar 09, 2007 9:24 am
by kirankota79
i want to be more clear......what iam doing is a swap technique. i want to swap the 4 collumns. for example.....the values of columns at row 1 should be swapped with the values of columns at row 100.
I have generated some random rownumbers with a C++ routine equal to the number of the records in the file. for example:
1 1990
2 3028
3 56
4 898...........etc. so the values at row 1 should be swapped with row 1990 and so on.............



i dont think the join stage can do this? Am i right.

Posted: Fri Mar 09, 2007 9:26 am
by DSguru2B
Use the transformer, re-arrange the columns. No join is required in this case.

Posted: Fri Mar 09, 2007 9:26 am
by kcbland
You need to define a primary key so that a row can be found in a reference. Lookup, Join, Merge, heck even a database join, require you to identify how to connect rows. If you mean data is positional within the data file, then simply use the row number as the primary key and create a column to hold that value.

Posted: Fri Mar 09, 2007 9:28 am
by kumar_s
Do a join based on first column in the Left file and second column as key in right file. Make sure you rename the columns and datatypes to be identical.

Posted: Fri Mar 09, 2007 9:31 am
by DSguru2B
I must be missing something here. If the columns within a single row are to be re-arranged, no join is required. Just re-arrange them in the transformer. Right :?:

Posted: Fri Mar 09, 2007 9:34 am
by kumar_s
Different rows.
the values of columns at row 1 should be swapped with the values of columns at row 100.

Posted: Fri Mar 09, 2007 9:34 am
by DSguru2B
kumar_s wrote:Different rows.
the values of columns at row 1 should be swapped with the values of columns at row 100.
Thank you Kumar. I should just shutup today. Sorry guys. :oops:

Posted: Fri Mar 09, 2007 9:38 am
by kumar_s
May be you can just start. :wink: