Folder Stage to filter Record

[Ocean]

Hi All,

Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?

What is the use of derivation field here, by the way?

Thanks

[chulett]

No... as it doesn't read 'records' but the full file contents all at once. And the derivation isn't used in this stage AFAIK, one of the joys of a standard 'widget' - not all fields are applicable everywhere it is used.

[g_goli]

Hi All,

Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?

Hi,

I would see two solutions here:
1. Read the file name and contents to the transformer and set the derivation to read the data until a Char(254) is found. If files are big it may cause problems.
2. Other way of working that out may be to use the folder stage to read filenames only and write a simple DS routine to read a first line of a sequential file and return it to the data flow in a tranformer.

Hope this helps.

Regards

[sb_akarmarkar]

Hi,

Yes you can extract file name and first record from file name too using field function.


For file name map key column from folder

For first record : Field(Column2,Char(13) : Char(10),1)


Thank you,
Anupam

[ray.wurlod]

Are you sure?

It's my understanding that Field() only allows a single character as the delimiter.

But I'm always happy to learn.

[sb_akarmarkar]

Are you sure?

It's my understanding that Field() only allows a single character as the delimiter.

But I'm always happy to learn. ...

Yes, It works...


Thanks,
Anupam

[ArndW]

Anupam,

interesting - I tried it as well since, like Ray, I was sure it only used one character.

Unfortunately, it did not work for me; it only uses the first character and ignores any subsequent ones for the delimiter.

I suggest you check again.

[sb_akarmarkar]

Anupam,

interesting - I tried it as well since, like Ray, I was sure it only used one character.

Unfortunately, it did not work for me; it only uses the first character and ignores any subsequent ones for the delimiter.

I suggest you check again.

Yes i tested, It is working for me ...

I created sample source file as

10,Harmani
20,Rocky
30,Samaron
40,Electra

and i am getting result as

"10,Harmani"

Thank You,
Anupam

[ray.wurlod]

Please inspect your result file with a hex editor, or Oconv(String,"MX0C"), and show us that the Char(10) characters have gone. After all, it's not a printable character.

[sb_akarmarkar]

Please inspect your result file with a hex editor, or Oconv(String,"MX0C"), and show us that the Char(10) characters have gone. After all, it's not a printable character. ...

Yes, It is also working without Char(10) also...

Field(Column2,Char(13), 1)

But Field function allows more than single character as the delimiter.

Thanks,
Anupam

[ray.wurlod]

From the DataStage BASIC manual:

delimiter evaluates to any character, including field mark, value mark, and subvalue marks. It delimits the start and end of the substring. If delimiter evaluates to more than one character, only the first character is used.

I tried it with two, where only the second delimiter character existed in the string, and Field() failed to extract the delimited string. So I believe the manual.

[sb_akarmarkar]

Ray,

Yes it takes only single charcter.....

Thanks,
Anupam

[Ocean]

Hi,

Yes you can extract file name and first record from file name too using field function.


For file name map key column from folder

For first record : Field(Column2,Char(13) : Char(10),1)


Thank you,
Anupam

Where I have to use that Field Function? In transformer? What I'm looking for is to get file name and first record of source file from folder stage itself without passing into transformer since data volumn is quite big.

Thanks.

[Ocean]

Hi All,

Is there any option in Folder stage to extract file name and first record of source file instead of all data? Can I make use of Derivation field in output column page?

Hi,

I would see two solutions here:
1. Read the file name and contents to the transformer and set the derivation to read the data until a Char(254) is found. If files are big it may cause problems.
2. Other way of working that out may be to use the folder stage to read filenames only and write a simple DS routine to read a first line of a sequential file and return it to the data flow in a tranformer.

Hope this helps.

Regards

How can I use derivation to read data until Char(254)? Look like derivation in Folder stage is not working at all. Please advice.

[chulett]

You can't really stop 'reading' at the first record delimiter. What I supposed the suggestion is would be to attempt to pull off the first 'record' in the Derivation after the entire file has been read in. May work... but I don't think that's the right choice of character to look for. Is this a UNIX file or a DOS file?

Try with the FIELD function, telling it the field delimiter is a CHAR(254) and that you want the first field. If that doesn't work, try again with CHAR(10) or CHAR(13).

And no, the derivation field in the Folder stage is not active. All objects of that nature use the same GUI 'widget' but not every stage actually uses all of the components.