replacement for lookup stage

kirankota79 · Post by **kirankota79** » Fri Mar 09, 2007 9:00 am

i want to replace the values of 4 columns with other values......so iam using the original file as input to the lookupstage and the new values sequential file as a lookup file to the lookup stage. the file has 10million records. Datastage job is aborting saying that the space is not sufficient. i understand that since it has millions of records ......lookup stage dont allow it. but can anybody tell me the replacement for it...so that i can proceed...

thanks in advance......

trobinson · Post by **trobinson** » Fri Mar 09, 2007 9:09 am

Join Stage

kirankota79 · Post by **kirankota79** » Fri Mar 09, 2007 9:24 am

i want to be more clear......what iam doing is a swap technique. i want to swap the 4 collumns. for example.....the values of columns at row 1 should be swapped with the values of columns at row 100.
I have generated some random rownumbers with a C++ routine equal to the number of the records in the file. for example:
1 1990
2 3028
3 56
4 898...........etc. so the values at row 1 should be swapped with row 1990 and so on.............

i dont think the join stage can do this? Am i right.

DSguru2B · Post by **DSguru2B** » Fri Mar 09, 2007 9:26 am

Use the transformer, re-arrange the columns. No join is required in this case.

kcbland · Post by **kcbland** » Fri Mar 09, 2007 9:26 am

You need to define a primary key so that a row can be found in a reference. Lookup, Join, Merge, heck even a database join, require you to identify how to connect rows. If you mean data is positional within the data file, then simply use the row number as the primary key and create a column to hold that value.

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 9:28 am

Do a join based on first column in the Left file and second column as key in right file. Make sure you rename the columns and datatypes to be identical.

DSguru2B · Post by **DSguru2B** » Fri Mar 09, 2007 9:31 am

I must be missing something here. If the columns within a single row are to be re-arranged, no join is required. Just re-arrange them in the transformer. Right

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 9:34 am

Different rows.

the values of columns at row 1 should be swapped with the values of columns at row 100.

DSguru2B · Post by **DSguru2B** » Fri Mar 09, 2007 9:34 am

kumar_s wrote:Different rows.
the values of columns at row 1 should be swapped with the values of columns at row 100.

Thank you Kumar. I should just shutup today. Sorry guys.

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 9:38 am

May be you can just start.