DSXchange

hi kcbland: At the moment we are only using one CPU we havent gone parallel yet using server. Doing two hash lookup's based on two key's from different tables. In both lookup's there is a common key lookup. Job Design: Two Hash's going into transformer. Seq file (Input) going into Transformer and an...

Hi All: We have done a couple of hash lookups on our server jobs...however there are millions and millions of rows and hence it takes for ever. Need to improve performance. My question: Is it better to join files or do a Merge on key columns instead of Hash lookups....? Whichever is better can you g...

Hello Ray:

Yes in the row splitter I load the table definition and it does read characters that are delimited and put in the respective columns.

Thanks,
MJ

Thanks Ray: But can you tell me how to avoid these warnings coming in the Logs...? Is there a defect in the row splitter...? Errors in log file are as follows: JobName..Row_Splitter_For_Table_Name_ 1 : read_delimited() - row 270, line 1, too many columns in record JobName..Row_Splitter_For_Table_Nam...

One more thing: The messages in the file were 244,000. I tried to view the log for the job, but on the Director, it says "Retrieving" and the process hangs. This is possibly due to a huge log file. In future we will have to run the jobs with 600,000 Messages in Message queue and the possib...

Hi throbinson: Thanks for your reply. However, my job encounters no bottlenecks or any such thing. Its working fine. What I would like to know that in MQ stage the setting Wait time = 0 (not to wait just read) is the best or Wait time = -1 (specifies indefinite wait period (Default)) is a better opt...

Hello All: I have designed a job that reads messages form the MQ Message queue and then seperates the header and data. The data which is delimited is read and parsed into a variable. That variable in which the data is parsed is broken into number of columns using row splitter then dumped into transf...

Thank you all for your replies.

Yes, using the vi editor is the best way to view the input file content.

MJ

Thanks Ray:

Went and added the other columns and now I do not get the 1st. error. As per the second error just called the routine and remapped the Indata and now its working fine.

Appreciate all your guidance. :D

MJ

Hello All: I am getting errors in the Director. Can anyone explain what the following error's mean: ERROR 1: Parse_2_1a..Row_Splitter_158: read_delimited() - row 83, line 3, too many columns in record ERROR 2: Program "DSU.RoutineName": Line 18, Variable "InData" previously undef...

Hello All: Thank you for all your responses. I guess I had better explain why the target has more columns than the source. Well the target will be getting data from other sources and my job is to concentrate on 4 sources see that its data is extracted transformed and loaded to this target. The thing...

Hello All: I have a source file that has 17 columns and a target file that has 25 columns. In the transfomer I match the 17 columns into respective columns in the targets 25 columns? How do I take care of the rest 8 columns? They show up Red in the transformer since they are not present in the sourc...

Hello All:

Thank you all SO much.

It was a combination of all that was mentioned.

Regards,
MJ

Hello All: I am able to compile and run my job sucessfully. Output is fine. But after some time when I try to go back to that job and try to open it I get the following error. "Failed to open RT_LOG2085 file." This has begun to happen to some of my saved jobs and some of my colleagues jobs...

Hi: Thank you for your responses. Yes I am able to view the output column by copying the output line to a word file and see that the data is showing up correctly. However, I would like to know the maximum length of these fields. I think that the maximum message length (MAXMsgLength) is 4 MB or 10485...