Hi,
My tera data Enterprise stage produces 140 Million records. I need these records to be sorted while the records are processed by the next stage. If I sort in datastage, it is not a good option to sort 140 Million records. So, I am planning to sort in the database stage.
But the sorting oredr will be gone, when it comes out. So i am forced to run the tera data enterprise in sequential mode.
Are there anu options or solutions , where I can get the data in sorted fshion out from tera data enterprise stage.
Thanks
Sorting In teradata Enterprise Stage
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Sort in DataStage, using a key-based partitioning algorithm (Hash or Modulus) on the sort key(s). You may be surprised how efficient it is. Make sure you have plenty of scratch disk space.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There's no reason to argue that partitioning loses sorted order. The data on each partition continue to flow in the same order in which they occurred prior to the partitioner.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.