same problem

attili · Post by **attili** » Fri Aug 18, 2006 2:22 pm

Hey guys we had the same problem over here. The job ran for 14 hours but not yer yeilded nothing (usually it takes just 2 hours) can any one throw some light on it.

chulett · Post by **chulett** » Fri Aug 18, 2006 2:26 pm

Err... what 'same problem'? Start over from the beginning. Or post a link to the other conversation 'cuz we have no idea what you are talking about.

meena · Post by **meena** » Fri Aug 18, 2006 6:08 pm

Hi attili
Explain about your job with little detail.What it is doing?What is your source and target. You posted a new topic and it is difficult to check for "same problem".

DSguru2B · Post by **DSguru2B** » Fri Aug 18, 2006 10:46 pm

I like the heading, heh, 'same problem'. Wow, Even the RMM stage cannot tackle this one. It just gets better and better here doesnt it

ray.wurlod · Post by **ray.wurlod** » Fri Aug 18, 2006 11:04 pm

RMM version 2.0 needs a recursive option, methinks.

attili · Post by **attili** » Sat Aug 19, 2006 8:20 am

Sorry guys for the confusion..actually I am new to this forum...i was confused with post newtopic/reply.

We have a job which actullay runs at 2500 rows per second....all of a shudden the row count has shuddenly dropped phenominally..now it is running at 9 rows per second which is not acceptable? statistics are updated after every run. What could be the posssible reason?

cam some one help me in this regard.

Thanks for the reply
Attili

chulett · Post by **chulett** » Sat Aug 19, 2006 8:27 am

Ok... at least now we have a clue what this is about.

However, that's not nearly enough information for anyone to help you. Post as many details as you can: job design, data volumes, source and target, everything you can think of. Perhaps then someone can help.

attili · Post by **attili** » Sat Aug 19, 2006 11:13 am

Hi Craig:

The jobs is designed for loading a peoplesoft table( Oracle DB). The desings is..client will send us a FLAT file and we have few lookups to compare data in the flat file with that of the lookup(CRC logic is been implemented to avoid duplicate values) then load it to the final table.

thats it...a simple design.

I think this info is enough.

Thanks and regards
Attili

chulett · Post by **chulett** » Sat Aug 19, 2006 2:44 pm

No, still not really enough... but we'll keep digging. And this:

attili wrote:(CRC logic is been implemented to avoid duplicate values)

This concerns me, as CRC32 is not a mechanism to 'avoid duplicates' but for CDD or Change Data Detection. Be that as it may...

Which part do you think has slowed down? Are your lookups done with hashed files or some other mechanism? Are you using the OCI stage to write to Oracle? What are your Array Size and Transaction Size settings? What update action are you using? Has your target table grown substantially since it used to run faster? Are your updates using an index? See, lots of little tidbits to know to be able to help you.

If you haven't tried this, make a copy of your job and hack parts off to see where the slowdown is. For example, pop off the target OCI stage and replace it with a Sequential File stage. If it speeds back up, then you know you have a database issue. Still slow? Lookups.

ray.wurlod · Post by **ray.wurlod** » Sat Aug 19, 2006 10:13 pm

attili wrote:We have a job which actullay runs at 2500 rows per second....all of a shudden the row count has shuddenly dropped phenominally..now it is running at 9 rows per second which is not acceptable? statistics are updated after every run. What could be the posssible reason?

What has changed?

"Nothing" is not the correct answer.

chulett · Post by **chulett** » Sat Aug 19, 2006 10:25 pm

I saved that question for you, Ray.

DSguru2B · Post by **DSguru2B** » Mon Aug 21, 2006 1:05 pm

I know right, thats Ray's signature question. And i just love his initial reply, "Nothing is not the correct answer", really cool.
Coming back to the OP query, check if the table you are loading changed. Check with the DBA if he put any triggers on it. If a trigger fires on every insert, that tends to slow down the inserts. And if the record size is really huge, then you can expect a drastic change as you have seen. Please have a chat with your DBA.

NBALA · Post by **NBALA** » Mon Aug 21, 2006 2:21 pm

Can Moderator Change the Post Topic ?..Same Problem ..no one can guess what is inside !

attili · Post by **attili** » Mon Aug 21, 2006 7:56 pm

Sorry for the confusion in the Topic Heading...as I am new to forums ...I tried posting a reply and ended up in posting a new topic..and I assure this will not happen again.

Comming to the discussion...thanks alot to every in sahring your valuble suggesions.

Regards
Attili