Anyone use CoSORT products in conjunction with DataStage Enterprise Edition?
Thanks,
Jason
CoSORT
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There is a CoSORT plug-in stage that gives access to CoSORT functionality.
"Plug-in", of course, simply means "optionally installed". In this case - due to licensing issues - it also means "separately chargeable".
I can not comment on its use, as I have not used it. However, if you visit the sister site of DSXchange, you can learn about the plug-in stage there.
"Plug-in", of course, simply means "optionally installed". In this case - due to licensing issues - it also means "separately chargeable".
I can not comment on its use, as I have not used it. However, if you visit the sister site of DSXchange, you can learn about the plug-in stage there.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
So far CoSort have worked mainly with the server edition for developing the plugin and determining compatibility and measuring performance improvements. There is no doubt the CoSort sorting, filtering and aggregation is many times faster then a server job. As much as 100 times faster. What I am dubious about is whether it is an advantage to Enterprise Edition customers. If I had the time I would run benchmark comparisons.
Since they don't have a parallel plugin you would need to implement it as a server shared container in a parallel job or as an external job run from a Sequence job stage or routine. My investigation of DataStage server v enterprise: some performance stats indicates the parallel sort, aggregate and database stages are many times faster then the server edition equivelent. So the ROI of choosing CoSort is not as strong and the lack of a native parallel stage is a major drawback.
Probably worth evaluating if you have very large amounts of source data in text files. Enterprise Edition isn't great at sequential file sources, it needs to import the data and convert it into partioned data in the native format. If CoSort can read, sort, filter and aggregate this data before DataStage processes it you may save a lot of time.
Since they don't have a parallel plugin you would need to implement it as a server shared container in a parallel job or as an external job run from a Sequence job stage or routine. My investigation of DataStage server v enterprise: some performance stats indicates the parallel sort, aggregate and database stages are many times faster then the server edition equivelent. So the ROI of choosing CoSort is not as strong and the lack of a native parallel stage is a major drawback.
Probably worth evaluating if you have very large amounts of source data in text files. Enterprise Edition isn't great at sequential file sources, it needs to import the data and convert it into partioned data in the native format. If CoSort can read, sort, filter and aggregate this data before DataStage processes it you may save a lot of time.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn