Hi,
I am struck while using a funnel stage.
I am writing the output of a transformer to a sequential file say f1.
I have another file f2 which I need to append to f1 at the begining.
for this I tried using a funnel stage.It didn't work because sequential file can have a single inputlink or a single o/p link.
ofcourse we can do the funneling operator in another job.
but is there any way that this can be done in a single job???
Any inputs from u guys are appreciated..
problem while using seq file and funnel stages.
Moderators: chulett, rschirm, roy
Hi,
First you send the contents of file f2 to funnel , then send the transformer output to the funnel. Set Funnel Type to "Sequence" in stage properties page. Ensure the link from file f2 is the link 0 in "Link Ordering" tab of the funnel properties.
Then direct the output of funnel to a sequential file.
First you send the contents of file f2 to funnel , then send the transformer output to the funnel. Set Funnel Type to "Sequence" in stage properties page. Ensure the link from file f2 is the link 0 in "Link Ordering" tab of the funnel properties.
Then direct the output of funnel to a sequential file.
Thanks mandyli..Your suggestion worked for me..mandyli wrote:Hi,
First you send the contents of file f2 to funnel , then send the transformer output to the funnel. Set Funnel Type to "Sequence" in stage properties page. Ensure the link from file f2 is the link 0 in "Link Ordering" tab of the funnel properties.
Then direct the output of funnel to a sequential file.
sun
I wonder this may not work
I did a simple test, it did not work. records from source A and source B are writen to fileC though Funnal stage, which I set funnel type to Sequence. I checked the fileC, the order of the data are not as defined. records are random fixed SourceA and SourceB. I don't know where I am wrong. I think better to use unix comand to cat two files.
changming - if you specify link ordering you will get the order that you specify.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
Hi, ArndW, that is also what I thought it should be. But I do see the records are all mixed from different source. BTW, the link order are set automatically by datastage in a default order. But if I set the excution mode to Sequential, records order in the ouput file is same as I specified.ArndW wrote:changming - if you specify link ordering you will get the order that you specify. ...
How many nodes are in your configuration file? If more than 1 and you are not using sequential mode then of course the order in your output will be different, depending on the hashing algorithm used with parallel processing.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>