what is best way to do ?

thumati.praveen · Post by **thumati.praveen** » Wed Jan 30, 2008 7:05 am

Hi,

I need to read 10 files from the single directory.These files are different file name and different data for different table loads.

The process will run in two phases one is validation if it is success then start the second phase transformation otherwise the whole process needs to be stoped.

we want to run all files at same time in single run.

How i need to design job can any give me a push up please.

Thanks&Regards,
Praveen.

Maveric · Post by **Maveric** » Wed Jan 30, 2008 7:09 am

Need more information. What kind of validations? What kind of transformations? Is meta data same across all the 10 files? If one file fails a validation, then should all the loads be stopped?

thumati.praveen · Post by **thumati.praveen** » Wed Jan 30, 2008 7:26 am

All validations and transformations done.we need to arrange the jobs in a sequence.

In one sequence we need to process all file validations.For that i need to read files one by one but I dont want to read one by one I want to read all 10 files at one time. By using which stages and which job design i can achive this?

Thanks&Regards,
praveen

thumati.praveen · Post by **thumati.praveen** » Wed Jan 30, 2008 7:36 am

The meta data is different for all.If one file fail all should stop the process.

Thanks,
Praveen

ray.wurlod · Post by **ray.wurlod** » Wed Jan 30, 2008 4:19 pm

If the metadata are different for each file create ten separate jobs to do the validation. Take OK triggers from each job activity into a Sequencer set to fire only if All of its inputs have fired. Make sure there is an alternative exit from the job sequence, for example non-OK triggers or automatic handling of exceptions.

thumati.praveen · Post by **thumati.praveen** » Wed Jan 30, 2008 11:57 pm

ray.wurlod wrote:If the metadata are different for each file create ten separate jobs to do the validation. Take OK triggers from each job activity into a Sequencer set to fire only if All of its inputs have fired. ...

Hi,

The validation jobs all were done.The problem is i need to run all jobs one time for that i need to send the file names for all jobs at run time.

Here one more problem also that is when the file was not run in the previous run that file we need to run first in the second run.

can you give me any ideas about imlementaion?

Thanks,
Praveen

ray.wurlod · Post by **ray.wurlod** » Thu Jan 31, 2008 1:11 am

If the metadata are different for each file create ten separate jobs to do the validation. Take OK triggers from each job activity into a Sequencer set to fire only if All of its inputs have fired. Make sure there is an alternative exit from the job sequence, for example non-OK triggers or automatic handling of exceptions. Use job parameter to supply the file name to each of the ten jobs.

thumati.praveen · Post by **thumati.praveen** » Thu Jan 31, 2008 8:59 am

ray.wurlod wrote:If the metadata are different for each file create ten separate jobs to do the validation. Take OK triggers from each job activity into a Sequencer set to fire only if All of its inputs have fired. Ma ...

Hi Ray,

All things was done properly.

The exact problem is now i am getting here when i am trying to run the executecommand activity through the sequence the command is executing in the /home/dsadm location.

But i want to execute the unix command in the some other location like /etl/source/data here my files are located i want to execute here.
Is their any way to execute the unix command through Execute command activity?

Thanks,
Praveen.

chulett · Post by **chulett** » Thu Jan 31, 2008 9:01 am

What command? Either supply a full path or explicitly 'cd' there first.

thumati.praveen · Post by **thumati.praveen** » Sat Feb 02, 2008 3:04 am

chulett wrote: What command? Either supply a full path or explicitly 'cd' there first. ...

command: ls -1|grep "xyz"|sort -u -r -t"." +5|head -1
parameter:etl/source/data

ray.wurlod · Post by **ray.wurlod** » Sat Feb 02, 2008 4:22 am

command: cd pathname && ls -1|grep "xyz"|sort -u -r -t"." +5|head -1
parameter:etl/source/data

ray.wurlod · Post by **ray.wurlod** » Sat Feb 02, 2008 4:25 am

command: source
parameter: cd #pathname# && ls -1|grep "xyz"|sort -u -r -t"." +5|head -1

chulett · Post by **chulett** » Sat Feb 02, 2008 8:32 am

Not quite sure what 'source' is, so...

command: sh
parameter: cd #pathname# && ls -1|grep "xyz"|sort -u -r -t"." +5|head -1

Where 'pathname' is a job parameter for the directory to execute the command in - in your case that would be '/etl/source/data'.

ray.wurlod · Post by **ray.wurlod** » Sat Feb 02, 2008 2:21 pm

source is the long form of . for executing shell scripts.

See man source for more information.

chulett · Post by **chulett** » Sat Feb 02, 2008 5:12 pm

First thing I tried when I saw your post was 'man source' but on our HP-UX systems I got nada. Guess I'll stick with the short form.