DSXchange

[quote="dsedi"]

The possible issue could be the columns metadata datatype/length may not be matching.

You can concatenate all your files /inputfiledir/Datafile* to a single file in your before job subroutine. Use concatenated file in your sequential stage. cat /inputfiledir/Datafile* > /inputfiledir/NewFile Anu, Does it cause processing time to merge files into one? Craig, The unix OS is: AIX 5.3, ...

Any suggestions on this please?

Thank you!

Try this UNIX command and see it you run into a problem. for file in `ls -1 Datafile*` do echo $file done There may be an operating system limit on the size of a list. ... I am able to see all files in OS (unix) using ls -l Datafile*. But when do that in Datastage using file pattern in sequential f...

One obeservation though: in the file pattern: /inputfiledir/Datafile* 8 files are: Datafile_1_1 format remaining 492 files are as Datafile_1234_1234 format If I keep these 492 files or 8 file only in the /inputfiledir, then I am able to read. But when I have all the 500 files, getting 'file not foun...

I have job with a sequential file stage that reads files using a file pattern and loads in to a table. The job design: seq stage --> transformer --> Target Table This job is running file with few files (I tested with 8 files) qualified for the file pattern. But, when I have 500 files, the job log sa...

So, I am using following code: mv #SrcDir#/Filename* #TgtDir# || echo Move Failed and it is working for me.. I'm always curious in situations like this - did you look up what the "pipe pipe" means in UNIX or since it works for you are you just using it? Serious question, btw. It has a sis...

Thank you for all your replies.

I do not know my exact file names as these are suffixed with timestamp.
So, I am using following code:
mv #SrcDir#/Filename* #TgtDir# || echo Move Failed
and it is working for me..

Thanks again..

anbu wrote:

Code: Select all

if [ -f file1 ]; then mv file1 file2 ;fi

Anbu,

One question, if I have multiple files to handle, how am I going to do this?
If I have multiple files, I want to move them all to a different directory in before routine. But, I do not want to abort my job if no files exist.

Thanks!

anbu wrote:

Code: Select all

if [ -f file1 ]; then mv file1 file2 ;fi

Thank you! I appreciate all your help! DSXchange rocks!!!

I am using a before job subroutine to move a file. But, if file does not exist I do not want to abort my job. Can anyone suggest me how to do this? Before-job subrotine (ExecSH), input Value : mv file1 file2 I am looking for any command that I can use in input value to move the file only if it exist...

Thanks again. I will experiment on the solution and will reply on how it went. Craig, The solution works! Thanks for all your help. Ganesh, I am marking the topic as resolved. But, feel free to eloborate if you have ideas about my comments on Multi-Instances. I am thinking to enable this to speed-u...

Unless your Database is partitioned, going for more nodes in your configuration file is not going to make a positive difference. ArndW, So far the only size you've mentioned is the 35Gb of scratch. Since we have no idea of the sizes of the two data sources it is impossible to make any suggestions o...

for large volume of data to join, more number of nodes. can you post the content of configuration your are using for this job? Here is my max 4x4 config (total 8 nodes): but pls note that we have only one scratchdisk with 35GB. { node "node1" { fastname "ds01lp01" pools "&q...

Thank you.
I unchecked 'Log warnings after activities that finish with status other than OK' to get rid of warning.

Thanks again