DSXchange

Hi Ray, I bought the premium membership yesterday 26th Nov. But i am still not able to see the content you posted usder premium. I got a mail from rick stating that i will get another mail of confirmation. But how long do you think it will take to get this membership. Should i also contact editor@ l...

Hi All, One of our ETL batch is running for 8 hours. This has 6 Match jobs in it. The match uses Unduplicate match and the matches are for Individual, Org , Address, Phone etc. Each of them run for more than hour. But the Frequency file is generated from a row generator with 1 row and all the column...

Thanks All for your replys.

I am going temporarily with the parallel seqeunce in a loop.
Also working on the script.

PS: The scripts is becoming bigger and bigger everytime i test it.

regards,
Samyam

Thanks for your reply.
The loop will call it sequentially one after another.

I need it to be triggered parallely.

Regards,
Samyam

Hi All,

I would like to trigger multiple instances of a job say 'n' times.
I need 'n' to be a dynamic value which i can choose at run time.

How do i do this?

Regards,
Samyam

There are no performance issues at all.
was just a bit worried to see the memory usage high all the time, but not anymore.

Thanks for all your responses.

Regards,
Samyam

There are 6 ISD jobs which are always runnig thats it.

Hi All, The top -m command gives the below result. top - 14:02:20 up 40 days, 4:40, 5 users, load average: 3.91, 2.80, 2.71 Tasks: 1117 total, 1 running, 1106 sleeping, 0 stopped, 10 zombie Cpu(s): 0.2%us, 0.5%sy, 0.0%ni, 99.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 98871628k total, 98052996k used, ...

Hi,

Its red hat linux 2.6.32

Regards,
Samyam

Thanks for the reply.
I will check all of this and see if I can come to some conclusion.

Hi,

I am just checking the health of our DS servers for a week now.
The reports consistantly show high memory utilization usually around 90%.
The memory on the servers is 128 GB

Is this something to be worried about? like bad design or need of performance tuning?

Regards,
Samyam

Thanks Ray.
will surely check and get back to you.

Hi All,

I am looking for the documentation of the ISD stage.
never could find a good one.

Could anyone suggest a place to look for it.

Regards,
Samyam

Hi All, Sorry about the delayed response. We tried a lot of options and the one of them gave us a good performance improvement. We Split the input file into 4 files of the size 4GB each and triggered the same job 4 times in parallel reading the the 4 different files. It came down to 1 hour processin...

Hi,

I have job where it reads from a txt file. One of the column in the file is xml.
the file is 15gb and has 40 million records. The job runs for 3 hours.

Is there any way i can improve its performance?

Regards,
Samyam