Negative Elapsed Time (- rows/sec link count)

Aparna_A · Post by **Aparna_A** » Thu Nov 23, 2006 10:18 am

Hi ,

Can somebody pls. let me know what is the meaning of getting negative row count on the Links between two stages.

[ex: -13037 rows/sec]

Thanks,

ArndW · Post by **ArndW** » Thu Nov 23, 2006 10:33 am

I've heard here that people have seen it but have not personally seen it. could it be that the job started on one day and finished on the next; since the formula could use "(EndTime-Startime)" as part of the computation that would cause the error on a day rollover unless it is accounted for.

Aparna_A · Post by **Aparna_A** » Thu Nov 23, 2006 10:39 am

Actually, the job has finished in a flash of a second and there was no change over in the day on the server.

ArndW · Post by **ArndW** » Thu Nov 23, 2006 10:45 am

Well, there goes that theory. Some part of DS can get the OS time in milliseconds, but on some systems those values aren't always too accurate (I recall HP-UX issues years ago); so if the job finished within 1 second and the millisecond portions of the start and end times don't come from OS correctly, i.e. end time is before start time then you could get a negative row count and it would probably be pretty hefty, too.

Is it reproduceable? Then it could be submitted to your support provider.

Aparna_A · Post by **Aparna_A** » Thu Nov 23, 2006 11:15 am

I have run the job again and the following are the details as seen in the job monitor. These are the statistics for a DB2 output link.

start time: 11/23/2006 12:06:19 PM
end time: 11/23/2006 12:05:43 PM
elapsed time: -01:59:24
Link Count : -18106 rows/sec.
Row Count: 651827

How can the end time on the link be less than the start time..

ArndW · Post by **ArndW** » Thu Nov 23, 2006 11:52 am

Aparna - you must have some really special hardware that the rest of us only know of as "quantum fuzzy hyper spatial computing" or QFHSC. I need one of those

Sounds like you have a reprodoceable test case for you support provider. What hardware and OS are you on?

Actually, could it be that DS is getting the end time from the DB2 server? If it is on a different box perhaps you could check the system time and see if that might be accounting for the difference.

Aparna_A · Post by **Aparna_A** » Thu Nov 23, 2006 1:49 pm

yes Arnd,

DB2 is on a remote server. There is a time difference between the DataStage server and the database remote server.
The datastage server time is ahead by about 1 minute and 13 seconds than the database server.

But is it not that the start time and end time are reported by DataStage server and not Database server? pls. let me know..

thanks

ArndW · Post by **ArndW** » Fri Nov 24, 2006 3:58 am

Aparna,

normally I would have said that start/stop times are those of the server, but you have discovered that this is not the case for the DB2 reporting on PX jobs. It seems to take the start time from the DS server and the end time from the DB2 server. This sounds like a bug which you should report through your support provider. It is an important and significant error in my opinion, since many decisions are based on rows/second and if you don't synchronize your UNIX times then pretty big discrepancies would be reported by DS (imagine if the time difference were reversed in your case, you'd be getting HUGE performance reported for small jobs).

ray.wurlod · Post by **ray.wurlod** » Fri Nov 24, 2006 12:55 pm

Yet another reason not to rely on rows/sec as a throughput measure!!
Consider the impact if the times differed in the opposite direction - you'd be complaining falsely about poor performance.