CoSORT

jatayl · Post by **jatayl** » Tue Apr 11, 2006 11:49 am

Anyone use CoSORT products in conjunction with DataStage Enterprise Edition?

Thanks,
Jason

kumar_s · Post by **kumar_s** » Wed Apr 12, 2006 8:58 am

Perhaps you may ask the issues or doubts you have, so that you may get the answer?

jatayl · Post by **jatayl** » Wed Apr 12, 2006 12:02 pm

kumar_s wrote:Perhaps you may ask the issues or doubts you have, so that you may get the answer?

Ok, my question is CoSORT says their product works with DataStage Server Edition, so I was wondering if anyone had used it with Enterprise Edition. I didn't know if it was possible.

Thanks.

ray.wurlod · Post by **ray.wurlod** » Wed Apr 12, 2006 4:32 pm

There is a CoSORT plug-in stage that gives access to CoSORT functionality.

"Plug-in", of course, simply means "optionally installed". In this case - due to licensing issues - it also means "separately chargeable".

I can not comment on its use, as I have not used it. However, if you visit the sister site of DSXchange, you can learn about the plug-in stage there.

vmcburney · Post by **vmcburney** » Wed Apr 12, 2006 9:22 pm

So far CoSort have worked mainly with the server edition for developing the plugin and determining compatibility and measuring performance improvements. There is no doubt the CoSort sorting, filtering and aggregation is many times faster then a server job. As much as 100 times faster. What I am dubious about is whether it is an advantage to Enterprise Edition customers. If I had the time I would run benchmark comparisons.

Since they don't have a parallel plugin you would need to implement it as a server shared container in a parallel job or as an external job run from a Sequence job stage or routine. My investigation of DataStage server v enterprise: some performance stats indicates the parallel sort, aggregate and database stages are many times faster then the server edition equivelent. So the ROI of choosing CoSort is not as strong and the lack of a native parallel stage is a major drawback.

Probably worth evaluating if you have very large amounts of source data in text files. Enterprise Edition isn't great at sequential file sources, it needs to import the data and convert it into partioned data in the native format. If CoSort can read, sort, filter and aggregate this data before DataStage processes it you may save a lot of time.