Folder Stage to filter Record
Moderators: chulett, rschirm, roy
Folder Stage to filter Record
Hi All,
Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?
What is the use of derivation field here, by the way?
Thanks
Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?
What is the use of derivation field here, by the way?
Thanks
Re: Folder Stage to filter Record
Hi,Ocean wrote:Hi All,
Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?
I would see two solutions here:
1. Read the file name and contents to the transformer and set the derivation to read the data until a Char(254) is found. If files are big it may cause problems.
2. Other way of working that out may be to use the folder stage to read filenames only and write a simple DS routine to read a first line of a sequential file and return it to the data flow in a tranformer.
Hope this helps.
Regards
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact:
Anupam,
interesting - I tried it as well since, like Ray, I was sure it only used one character.
Unfortunately, it did not work for me; it only uses the first character and ignores any subsequent ones for the delimiter.
I suggest you check again.
interesting - I tried it as well since, like Ray, I was sure it only used one character.
Unfortunately, it did not work for me; it only uses the first character and ignores any subsequent ones for the delimiter.
I suggest you check again.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact:
Yes i tested, It is working for me ...ArndW wrote:Anupam,
interesting - I tried it as well since, like Ray, I was sure it only used one character.
Unfortunately, it did not work for me; it only uses the first character and ignores any subsequent ones for the delimiter.
I suggest you check again.
I created sample source file as
10,Harmani
20,Rocky
30,Samaron
40,Electra
and i am getting result as
"10,Harmani"
Thank You,
Anupam
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Please inspect your result file with a hex editor, or Oconv(String,"MX0C"), and show us that the Char(10) characters have gone. After all, it's not a printable character.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact:
Yes, It is also working without Char(10) also...ray.wurlod wrote:Please inspect your result file with a hex editor, or Oconv(String,"MX0C"), and show us that the Char(10) characters have gone. After all, it's not a printable character. ...
Field(Column2,Char(13), 1)
But Field function allows more than single character as the delimiter.
Thanks,
Anupam
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
From the DataStage BASIC manual:
I tried it with two, where only the second delimiter character existed in the string, and Field() failed to extract the delimited string. So I believe the manual.delimiter evaluates to any character, including field mark, value mark, and subvalue marks. It delimits the start and end of the substring. If delimiter evaluates to more than one character, only the first character is used.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 232
- Joined: Fri Sep 30, 2005 4:52 am
- Contact:
Where I have to use that Field Function? In transformer? What I'm looking for is to get file name and first record of source file from folder stage itself without passing into transformer since data volumn is quite big.sb_akarmarkar wrote:Hi,
Yes you can extract file name and first record from file name too using field function.
For file name map key column from folder
For first record : Field(Column2,Char(13) : Char(10),1)
Thank you,
Anupam
Thanks.
Re: Folder Stage to filter Record
How can I use derivation to read data until Char(254)? Look like derivation in Folder stage is not working at all. Please advice.g_goli wrote:Hi,Ocean wrote:Hi All,
Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?
I would see two solutions here:
1. Read the file name and contents to the transformer and set the derivation to read the data until a Char(254) is found. If files are big it may cause problems.
2. Other way of working that out may be to use the folder stage to read filenames only and write a simple DS routine to read a first line of a sequential file and return it to the data flow in a tranformer.
Hope this helps.
Regards
You can't really stop 'reading' at the first record delimiter. What I supposed the suggestion is would be to attempt to pull off the first 'record' in the Derivation after the entire file has been read in. May work... but I don't think that's the right choice of character to look for. Is this a UNIX file or a DOS file?
Try with the FIELD function, telling it the field delimiter is a CHAR(254) and that you want the first field. If that doesn't work, try again with CHAR(10) or CHAR(13).
And no, the derivation field in the Folder stage is not active. All objects of that nature use the same GUI 'widget' but not every stage actually uses all of the components.
Try with the FIELD function, telling it the field delimiter is a CHAR(254) and that you want the first field. If that doesn't work, try again with CHAR(10) or CHAR(13).
And no, the derivation field in the Folder stage is not active. All objects of that nature use the same GUI 'widget' but not every stage actually uses all of the components.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers