Should I use DataSets?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

kduke
Charter Member
Charter Member
Posts: 5227
Joined: Thu May 29, 2003 9:47 am
Location: Dallas, TX
Contact:

Post by kduke »

Still 4,000 rows per second is very slow.
Mamu Kim
dougcl
Premium Member
Premium Member
Posts: 137
Joined: Thu Jun 24, 2010 4:28 pm

Post by dougcl »

chulett wrote:You marked your post as using the UNIX system for your server, hence the confusion. ...

Actually I marked my post as using the UNIX system for our server, because we are. Raftsman is just here stirring up trouble :)
dougcl
Premium Member
Premium Member
Posts: 137
Joined: Thu Jun 24, 2010 4:28 pm

Post by dougcl »

Raftsman wrote: When I read, I get approx 4000 row/sec. from a dataset. When I read from Oracle, I get 30000 rows/sec.
Hi just ran some tests here. I have a table with 180 byte rows. Reading 10M rows into dev/null from Oracle runs at 50K rows/sec. Reading 10M rows from a dataset into dev/null runs at 170K rows per second.

Please note we are storing datasets on a SAN.

I haven't done much write testing to Oracle yet, but our write performance to the dataset easily keeps up with the Oracle reads.

Thanks,
Doug
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

dougcl wrote:
chulett wrote:You marked your post as using the UNIX system for your server, hence the confusion. ...
Actually I marked my post as using the UNIX system for our server, because we are. Raftsman is just here stirring up trouble :)
Sorry, failed to recognize the hijack! :x
-craig

"You can never have too many knives" -- Logan Nine Fingers
Raftsman
Premium Member
Premium Member
Posts: 335
Joined: Thu May 26, 2005 8:56 am
Location: Ottawa, Canada

Post by Raftsman »

I didn't hijack, I was giving my two cents regarding performance. I realize they are two different operating systems but the subject is the same. As for stirring up trouble, no. I would prefer to use datasets.

So in summary, I wasn't hijacking the thread. I would figure that Windows and Unix would have some similarities.
Jim Stewart
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

OK, wrong word then. Double-sorry. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply