Sequential File Stage and file name variable

harborboy76 · Post by **harborboy76** » Wed Apr 21, 2010 2:27 pm

What's the best approach to parse out the sequential file name to be used
as a variable inside Transformer Stage ? For our project, file name is a
unique ID that will be used within jobs to populate our target table.

Am I correct in thinking that I may have to use Job Sequence to execute
linux command (Execute Command Stage) to parse out the file name and
pass in as a variable ? Or is there a simpler alternative to accomplish the
same thing.

Thanks!

anbu · Post by **anbu** » Wed Apr 21, 2010 2:46 pm

Keep the filename as job parameter and use it in the transformer

harborboy76 · Post by **harborboy76** » Wed Apr 21, 2010 3:44 pm

Would it mean that I have to know the file name ahead of time to be able
to pass it in at runtime ? I need to automate this since I may not always
know the file name ahead of time or want to look for file name to be
passed in as parameter manually. Sorry if I misunderstood your response.

anbu · Post by **anbu** » Wed Apr 21, 2010 4:00 pm

Are you using sequential file stage and want to use this file name in transformer?

harborboy76 · Post by **harborboy76** » Wed Apr 21, 2010 4:44 pm

That is correct -

chulett · Post by **chulett** » Wed Apr 21, 2010 6:06 pm

Not enough information for a cogent answer. How are you reading the files now - with a file pattern? Is that why you don't know the actual filename run to run? I'm assuming that, otherwise as others have noted you'd just use the job parameter that was passing it in.

There's an option in the Sequential File stage to add a column populated with the actual filename found. Don't recall exactly and can't check but should be easy enough to find. That would give you the filename in a new column as part of each record passing through the job.

And if it doesn't work as expected, check posts like this.

harborboy76 · Post by **harborboy76** » Wed Apr 21, 2010 7:29 pm

It works!

Thank you very much for the pointer. Under Sequential File Stage, I
found File Name Column under Options. When I add that column name to
the output, I now have the file name (and the path). All I need to do now
is parsing out the file name.

This is too cool~!

ray.wurlod · Post by **ray.wurlod** » Wed Apr 21, 2010 9:02 pm

Code: Select all

Field(InLink.FileNameColumn, "/", Count(InLink.FileNameColumn, "/") + 1, 1)

Maybe you can mark this thread as Resolved?

harborboy76 · Post by **harborboy76** » Wed Apr 21, 2010 11:11 pm

Oh, I will do that now. Thank you for the tip as well! :D