Joining Sequential Files

[donlank]

I have 3 files with the same column definitions coming from 3 different jobs. I want to put all three files into one file in order to perform one lookup on a hashed file instead of three separate identical lookups.

Is there a way to put these files together?

Thanks,
Kevin

[crouse]

You could "cat" the files together at the command line in a Before Job routine, then read the resultant file and create the hash file.

-Craig

[crouse]

Or, read all three files with three SeqFile stages and into a Link Collector stage, and then into the hash file.

I like the "cat" option better, though.

-Craig

[donlank]

I tried the link collector already. It says, "Link Collector Stage does not support in-process active-to-active inputs or outputs"

So the only option to put three files into one in a Server Job is to write a Before Job Routine? There isnt a stage to accomplish this?

Thanks,
Kevin

[crouse]

Nope, no stage to do it (the Before Routine).

You can make the Link Collector work with adding in IPC stages and such. That's why I favor the "cat" method.

-Craig

[changming]

I tried the link collector already. It says, "Link Collector Stage does not support in-process active-to-active inputs or outputs"

So the only option to put three files into one in a Server Job is to write a Before Job Routine? There isnt a stage to accomplish this?

Thanks,
Kevin

open job peoperty and click performence, then select inter process.
anoter suggestion to your job is using multiple instanc, I believe that your job is a typical multi-instance job.

[tonystark622]

Why not three sequential file stages into a single hash file stage, only all of them feeding the same hash file?

Tony

[crouse]

The nice thing about DataStage is that there are several ways to do the same thing.

The bad thing about DataStage is that there are several ways to do the same thing.

:D

[jreddy]

If you need just the hash file, and not the 3 individual sequential files, could you modify your 3 initial jobs to write to a hash file, rather than a sequential file ??

this way, all 3 jobs are writing to same hash file and when you are done with those, the next set of jobs can lookup on just this single hash file.

[rasi]

Hi jreddy

when doing that make sure that the first file writes to the hash will delete the content before writing the records. Otherwise you will have the old records stacked into it.

Thanks

[jreddy]

kevin,

Actually, you might want to clear previous contents only on the first job that creates this hash file. The other two jobs that write to this hash file should insert data in append mode.

[donlank]

Thanks for all your input.

I decided to do an after job routine after the 3rd job finishes and cat all three files together, then do the hash file lookup.