script to process a file

India2000 · Post by **India2000** » Sat Mar 10, 2012 12:11 pm

I need an idea as how to process files in the directory file by file method using pattern after listing file names in the list and retrieving each file name in the list for process.I'm stuck at this point..Once read file names how should I proceed to put the file in the specific path and how do I come to know the file processing is completed to start with the next file? Pls direct me

Thanks .

chulett · Post by **chulett** » Sat Mar 10, 2012 1:26 pm

What?

So... you've put a list of filenames in a file and want to know how to leverage that file? If that's the case, then use File Pattern in the Sequential File stage and give it your filename.

India2000 · Post by **India2000** » Sat Mar 10, 2012 2:04 pm

Thanks..I don;t want to use the pattern method reading from seq file..In the event of failure/issues it would be difficult to trace files if I use that method. I have already used that method..But I want to know if it can be done efficiently using scripting like as each file gets processed log will show that file processes successfully.

qt_ky · Post by **qt_ky** » Sat Mar 10, 2012 2:26 pm

If you don't want to use Sequential File stage file pattern then you can parameterize a multi-instance job and still process all the files at once.

chulett · Post by **chulett** » Sat Mar 10, 2012 3:31 pm

... or a looping Sequence job.

ray.wurlod · Post by **ray.wurlod** » Sat Mar 10, 2012 8:16 pm

... or, indeed, a looping sequence job that uses the contents of your file of file names. The technique has been explained on DSXchange in the past.

India2000 · Post by **India2000** » Sun Mar 11, 2012 11:31 pm

for a start of loop activity list option how do I need to use list values as I have list of filenames in a column...like

File Name 1
File Name 2
...
what would be my delimiter value?

ray.wurlod · Post by **ray.wurlod** » Sun Mar 11, 2012 11:37 pm

Whatever you want it to be. When you read your file it's the line terminator character, but you can convert that you anything you like in a User Variables activity.

India2000 · Post by **India2000** » Mon Mar 12, 2012 1:00 am

thanks Ray.But your answer is not clear to me.. I meant how should I give a input to the start loop activity for a list of file names.. Should I give .list as an input? or the file pattern?

kandyshandy · Post by **kandyshandy** » Mon Mar 12, 2012 2:36 am

This is not the answer for your question. But it would be great if you locate 'Help' button on the start loop activity

That will give you enough information

India2000 · Post by **India2000** » Mon Mar 12, 2012 7:45 am

I do want to provide list file names explicitly coz..Instead is there a way that I can do this coz I'm not sure about the number of files..So need to use file pattern

chulett · Post by **chulett** » Mon Mar 12, 2012 7:50 am

Why not simply use the File Pattern capability of the Sequential File stage and be done with this? Is there something in what you are doing that requires you to process each file one at a time?

India2000 · Post by **India2000** » Mon Mar 12, 2012 11:11 am

it would be easier track issues with files if I use this method

battaliou · Post by **battaliou** » Mon Mar 12, 2012 11:58 am

Take a look at the UNIX tr -s '\n\' ',' command.

chulett · Post by **chulett** » Mon Mar 12, 2012 12:44 pm

India2000 wrote:it would be easier track issues with files if I use this method

Why? What would be easier about it? The stage can import the current filename along with the data, if that is your concern. To me, this seems like more work for zero gain. [shrug]