I recieve several source files as input. Problem is I don't know which ones are UTF8 encoded. What I want to do is the following:
Folder---Transformer---ODBC
NLS set toUTF8 in the folder stage.
The filename from the folder stage is written to the odbc.
When I run the job, any source file that does not conform to UTF8 generates a warning (ds_uvgetnext() : record 'Customers.txt' contains characters which are not defined in the file's NLS map).
I want to write only those files out where the warning is generated. I have looked at link variables in the constraint, but the 'Input' link variables are greyed out.
Any help on this would be greatly appreciated. :D
Redirect data on a warning
Moderators: chulett, rschirm, roy
I think that DataStage might not be the most efficient tool to do this, but if I were to do it in DS I would create a job that reads the file and writes it to /dev/null or something of the like. I'd call the job from a sequence with the filename to check and then use the job's completion status as the condition to branch to the "real" extraction job. It will waste lot's of {speedy} sequential I/O on files that generate warnings on row 1 but is pretty easy to implement.
I still think I'd use a UNIX level command - perhaps searchign for a known string that will show up in one type of file and never in the other.
I still think I'd use a UNIX level command - perhaps searchign for a known string that will show up in one type of file and never in the other.