Sorting In teradata Enterprise Stage

makreddy · Post by **makreddy** » Fri Mar 21, 2008 1:31 pm

Hi,
My tera data Enterprise stage produces 140 Million records. I need these records to be sorted while the records are processed by the next stage. If I sort in datastage, it is not a good option to sort 140 Million records. So, I am planning to sort in the database stage.
But the sorting oredr will be gone, when it comes out. So i am forced to run the tera data enterprise in sequential mode.

Are there anu options or solutions , where I can get the data in sorted fshion out from tera data enterprise stage.

Thanks

ray.wurlod · Post by **ray.wurlod** » Fri Mar 21, 2008 5:17 pm

Sort in DataStage, using a key-based partitioning algorithm (Hash or Modulus) on the sort key(s). You may be surprised how efficient it is. Make sure you have plenty of scratch disk space.

kumar_s · Post by **kumar_s** » Fri Mar 21, 2008 7:12 pm

Even the Partitioning Sorted data would loose the sort oder, so either it should run in Sequential more or to sort in Datastage.

ray.wurlod · Post by **ray.wurlod** » Sat Mar 22, 2008 3:05 am

There's no reason to argue that partitioning loses sorted order. The data on each partition continue to flow in the same order in which they occurred prior to the partitioner.

kumar_s · Post by **kumar_s** » Sat Mar 22, 2008 8:02 am

That case, Hash partition based on Key should still maintain the sorting order from the Database in each partition.