How to Sort the data in the Sequential Stage
Moderators: chulett, rschirm, roy
How to Sort the data in the Sequential Stage
I know there is a filter property in the sequential stage but i dont know how to use that can someone give me the example.
Search on this forum will give your more insight.
The existing filter option enables you to use Unix command in it.
Say for example,
if you want to read a file in a sorted order use the filter option
And make the file name to dummy by pointing to /dev/null
The existing filter option enables you to use Unix command in it.
Say for example,
if you want to read a file in a sorted order use the filter option
Code: Select all
sort <filename>| [options]
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
Hi Kumarkumar_s wrote:Search on this forum will give your more insight.
The existing filter option enables you to use Unix command in it.
Say for example,
if you want to read a file in a sorted order use the filter optionAnd make the file name to dummy by pointing to /dev/nullCode: Select all
sort <filename>| [options]
Where do we have filter option?
Sam
Re: How to Sort the data in the Sequential Stage
You can use sort stage which is supplied by DS (remember this will be very slow if you are dealing with big file) or you can use CoSort (this is tool by which you can do sort, aggregate and many more things. If your company has this I can help you to setup script).Ashish wrote:I know there is a filter property in the sequential stage but i dont know how to use that can someone give me the example.
Let me know this helps you or not Ashish
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
If we are talking PX (as that's the forum we're in) then the filter is an option that can be selected in the bottom right hand pane.
As its name suggests it is used to pass the data through a filter program.
If you want to sort the data coming from a sequential file use a sort stage, if it is being written to the sequential file use the appropriate collection method on the partitioning tab.
As its name suggests it is used to pass the data through a filter program.
If you want to sort the data coming from a sequential file use a sort stage, if it is being written to the sequential file use the appropriate collection method on the partitioning tab.