DSXchange

Thanks for the replies. Actually we are trying to come up with a best approach to cleanse or translate our data. I was suggesting the first approach menthioned in my first post but there was an argument for sending the whole record to a routine and that routine will lookup to the hashed file to get ...

Guys I would like to know which one will perform better for the below given scenorio. Say I have a file with 500,000 records and each record has 10 columns. Out of the 10 columns 4 columns need to be validated. I have a hashed file with one record for each coulmn validation rule. Scenario 1 --------...

Criag

Sorry for the confusion. here means here is DSXchange. It's the script written by Kim Duke(Runjob.ksh) long back.

Thanks
Kaps

Criag

Sorry for the confusion. here means here is DSXchange. It's the script written by Kim Duke(Runjob.ksh) long back.

Thanks
Kaps

Thanks for the ideas. I was able to run it successfully like this.... runjob.sh project_name job_name -param prm_1=val1 -param prm_2=val2 But what I am looking for is a way to get these parameters from parameter file dynamically when I run the job instead of mentioning them one by one. I know we can...

Thanks for the reply. do you see "shift" in the code after every parameter I believe that takes care of parameter assignment. I don't think that causes the problem.

Guys when I run job from command line using the scipt given below I am getting "Invalid arguments:dsjob -run" error... I took the piece of code which assigns the parameters from parameter file from one of the post here... It looks like thats where I am having problem. This is the shell I a...

Amey, Thanks...Your logic works fine. Only thing is I need to create as many links as the number of cleanse fields. But better than having many Hashed file stages. Naveen, I have not tested ur solution yet. I will do it and let you know. Meanwhile if u had a chance to test it Please post the result ...

Thanks Amey and Naveen for your time...

It looks like Amey has given the solution. Let me test it and keep you guys posted.

Thanks

thats just an example I was describing...I can't create a table for every field which need to be cleansed.

Naveen- Here you go... [code]Source Record (File) -------------------- Source ID Gender State Cntry_Code ----------------------------------------------------------- SRC1 Male Nebraska United States RuleTable(DataBase) -------------------- Source ID Source Value Cleansed Value -----------------------...

Thanks Naveen... BUT the Problem is, when I join the source and hashed file based on keys I agree TField4 will get RField4 but How do I get TField5 which should be again RField4 (and value of this RField4 will be different then the previous RField4 Because now my source value is SField5 which was SF...

Naveen Thanks. Well...Let me try to put in different way. Source Record -------------- SField1 SField2(Source ID) SField3 SField4 SField5 SField6 RuleTable(DataBase) --------- RField1(Source ID) RField3(Source Value) RField4(Cleanse Value) Job Design ---------- SeqFile(Source)--->Transformer--->Targ...

No. I don't think hash file is capable of doing this functionality. If I have to use hash file and if my file has 10 column in a record which need to be cleansed then I need 10 hash file stages from my transformer. Am I correct ?
Please clarify my if you think I am wrong.

Thanks for the input. I am still not clear about how I implement this logic. I have done similar thing in Informatica easily using a transformation called un-connected lookup which will get all the record from database to cache when the first fetch is executed. so subsequent get to that lookup will ...