Sort in Ascending or Descending based on parameter
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 147
- Joined: Sat Apr 30, 2005 1:23 am
- Location: Bangalore,India
Sort in Ascending or Descending based on parameter
I have to sort the data either in ascending order or descending order based on parameter value. I tried to implement it with the following approach:
In the transformer create two output links. If sorting to be done in Ascending enable the first link and pass the data to a sort stage and set sorting as Ascending in its properties. If descending, enable the second link and pass the data to a sort stage and set sorting as descending in its properties. Funnel the data and write it to output. Make the operation as sequential from sort stages.
Transformer ------------>Ascending
| Sort Stage --------------->
| Funnel --->Output
|---------------->Descending -------------->
Sort Stage
I believe there will be a better approach than this. Can anybody give me some inputs towards the optimal solution?
In the transformer create two output links. If sorting to be done in Ascending enable the first link and pass the data to a sort stage and set sorting as Ascending in its properties. If descending, enable the second link and pass the data to a sort stage and set sorting as descending in its properties. Funnel the data and write it to output. Make the operation as sequential from sort stages.
Transformer ------------>Ascending
| Sort Stage --------------->
| Funnel --->Output
|---------------->Descending -------------->
Sort Stage
I believe there will be a better approach than this. Can anybody give me some inputs towards the optimal solution?
Raghavendra
Dare to dream and care to achieve ...
Dare to dream and care to achieve ...
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Please encase your design in Code tags - what you have posted is incomprehensible. Does your design work? The direction of sort, wherever it is specified, is a sub-property that does not accept a job parameter reference.
There's no need for sequential operation; why do you believe there is?
There's no need for sequential operation; why do you believe there is?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 147
- Joined: Sat Apr 30, 2005 1:23 am
- Location: Bangalore,India
The design is as follows:
ASC_IND_PARM specifies whether to sort in Ascending or Descending order. If ASC_IND_PARM =1 then the data will be sent to Ascending sort stage (Sort Order set to Ascending) and If ASC_IND_PARM <>1 the data will be sent to Descending sort stage (Sort Order set to Descending). Then both of them will be joined by a funnel stage. Here only one link will be active at a time.
Limitation in this design is that we need to set the sort stages to sequential mode and use sequence funnel which will not disturb the sort order.
ASC_IND_PARM specifies whether to sort in Ascending or Descending order. If ASC_IND_PARM =1 then the data will be sent to Ascending sort stage (Sort Order set to Ascending) and If ASC_IND_PARM <>1 the data will be sent to Descending sort stage (Sort Order set to Descending). Then both of them will be joined by a funnel stage. Here only one link will be active at a time.
Limitation in this design is that we need to set the sort stages to sequential mode and use sequence funnel which will not disturb the sort order.
Code: Select all
Transformer ---------------->Sort Stage ( Properties set to Ascending )
| |
| |
| |
\/ \/
Sort Stage ------------------>Funnel Stage -------> Outputfile
( Properties
set to Descending)
Raghavendra
Dare to dream and care to achieve ...
Dare to dream and care to achieve ...
-
- Participant
- Posts: 147
- Joined: Sat Apr 30, 2005 1:23 am
- Location: Bangalore,India
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 147
- Joined: Sat Apr 30, 2005 1:23 am
- Location: Bangalore,India
Continuous Funnel combines the records of the input data in no guaranteed order.In Sort funnel we can either sort in ascending or descending but not both ( Sort order accepts only one fixed value). And so I have used Sequence Funnel to collect the data.
If you use parallel sort stages before sequence funnel, the data will be sorted within a node but not across the nodes.
Sequence funnel will just collect the data from sorts and pass the data to output and the data will not be in sorted order.And so the sort stages execution is set sequential mode in the implementation.
If you use parallel sort stages before sequence funnel, the data will be sorted within a node but not across the nodes.
Sequence funnel will just collect the data from sorts and pass the data to output and the data will not be in sorted order.And so the sort stages execution is set sequential mode in the implementation.
Raghavendra
Dare to dream and care to achieve ...
Dare to dream and care to achieve ...
-
- Participant
- Posts: 147
- Joined: Sat Apr 30, 2005 1:23 am
- Location: Bangalore,India
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If your data are key-partitioned on the sort keys (hash or modulus algorithm) every distinct value will occur on one and only on one partition, so that your entire data set is sorted no matter how many degrees of parallelism you use.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.