poor performance
Moderators: chulett, rschirm, roy
poor performance
Hi ,
My job is like db--->AggStage---->db. I am aggregating the sal group by geography. i am processing 1 million records. my job is running very slow. I used Link Partitioner though its very slow. How can I improve my job performance?
any help would be appreciated
My job is like db--->AggStage---->db. I am aggregating the sal group by geography. i am processing 1 million records. my job is running very slow. I used Link Partitioner though its very slow. How can I improve my job performance?
any help would be appreciated
Ravi
Sort the data before passing it to the aggregator stage. You can sort the incoming data by a mere order by clause in your sql select, and let the aggregator know that the sort order. That will help speed up things.
Also, for that amount of data, do the aggregation at the database level. I mean, in you sql itself.
Regards,
Also, for that amount of data, do the aggregation at the database level. I mean, in you sql itself.
Regards,
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
poor performance
Hi thanks for quick response.
Do u mean that should i write the user defined sql at source(database) level to sort the records?
thanks in advance
Do u mean that should i write the user defined sql at source(database) level to sort the records?
thanks in advance
Ravi
Re: poor performance
You can use a sort stage to sort the recordsravij wrote:Hi thanks for quick response.
Do u mean that should i write the user defined sql at source(database) level to sort the records?
thanks in advance
Then put your ORDER BY into your Oracle SELECT statement.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
In the OCI stage, in the sql tab, click on column generated sql query and click the build button.. There you can see theDSguru2B wrote:A better option would be a db order by rather than a sort stage for that amount of records. Do as ArndW suggested. Put the orderby clause in the sql select. You might have to go for user defined sql. Look into it.
FROM , WHERE and OTHER clause
You can specify your ORDER BY in the OTHER Clause. This way you don't need to use a user defined sql.