DSXchange

I recieve several source files as input. Problem is I don't know which ones are UTF8 encoded. What I want to do is the following:

Folder---Transformer---ODBC

NLS set toUTF8 in the folder stage.
The filename from the folder stage is written to the odbc.

When I run the job, any source file that does not conform to UTF8 generates a warning (ds_uvgetnext() : record 'Customers.txt' contains characters which are not defined in the file's NLS map).

I want to write only those files out where the warning is generated. I have looked at link variables in the constraint, but the 'Input' link variables are greyed out.

Any help on this would be greatly appreciated. :D

I think that DataStage might not be the most efficient tool to do this, but if I were to do it in DS I would create a job that reads the file and writes it to /dev/null or something of the like. I'd call the job from a sequence with the filename to check and then use the job's completion status as the condition to branch to the "real" extraction job. It will waste lot's of {speedy} sequential I/O on files that generate warnings on row 1 but is pretty easy to implement.

I still think I'd use a UNIX level command - perhaps searchign for a known string that will show up in one type of file and never in the other.

Thanks for the reply. However, my OS is Windows, not Unix.

All I want to do is somehow identify those source files I recieve who's content is UTF8 encoded. I currently run this process manually and want to automate it.