SQL query or explicit stage

sharmabhavesh · Post by **sharmabhavesh** » Tue Aug 07, 2012 1:24 pm

Hi,
I have data in a source table. I want to group the data based on a field.
Should I use SQL query inside my source table stage or shall I use an agreegator stage. Which one is a better option and why?

Thanks in advance

.

Kryt0n · Post by **Kryt0n** » Tue Aug 07, 2012 4:42 pm

Try both ways and then you know.

All down to complexity of query, loads on each server, size of each server etc etc

ray.wurlod · Post by **ray.wurlod** » Tue Aug 07, 2012 5:34 pm

Is the field indexed in the database?

sharmabhavesh · Post by **sharmabhavesh** » Tue Aug 07, 2012 9:38 pm

Hi Ray,
There's no indexing in the table.
Right now, sample data I have has very low number of records so I am not able to find out any difference in time by both the methods.
But in production, there would be huge amount of data and that's why I want to know which one would work better?

BI-RMA · Post by **BI-RMA** » Wed Aug 08, 2012 4:00 am

It is almost impossible to answer this question not knowing the particular task and data.

If SQL-grouping will reduce the number of records selected drastically and the database is not too weak on resources, I would probably opt for that. If not I would expect DataStage to be faster in many circumstances.

But as KrytOn said: the answer depends on many factors...

DSXchange

SQL query or explicit stage

SQL query or explicit stage

In stage query or explicit stage