Passing Filenames between Jobs in Job Sequence

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Passing Filenames between Jobs in Job Sequence

Post by vijayrc »

Hi,
I have a stream of 3 jobs and I'm sequencing it. The Output from Job#1 is Input to Job#2 and Job#3 and I want the Output of Job#1 to end with a type based on the run. How do I establish this. I included filename#XX# as the filename, and XX is prompt in Job Sequencer, but in the following jobs, the value entered for XX isn't carried forward. How do I go about doing this one. Any help is appreciated.
Also I'm encountering SIGKILL and SIGSEGV errors quite often. Any guidance on this is appreciated. I just run again, and it goes thru.
Thx, -V
roy
Participant
Posts: 2598
Joined: Wed Jul 30, 2003 2:05 am
Location: Israel

Post by roy »

Hi,
Regarding the sequence issue you can have an additional job parameter or use the invocation id if you use a multi instance job to diferentiate between file names.

Regarding the other issues it seems sometihng is killing your jobs and or the system is overloaded at times.
Sometimes there are processes in the system that issues a kill all processes that are connected to a DB when critical DB operations take place, in those cases you have little to do other then run at diferent times.

Another culprit might be a fire-wall.

IHTH,
Roy R.
Time is money but when you don't have money time is all you can afford.

Search before posting:)

Join the DataStagers team effort at:
http://www.worldcommunitygrid.org
Image
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Post by vijayrc »

roy wrote:Hi,
Regarding the sequence issue you can have an additional job parameter or use the invocation id if you use a multi instance job to diferentiate between file names.

Regarding the other issues it seems sometihng is killing your jobs and or the system is overloaded at times.
Sometimes there are processes in the system that issues a kill all processes that are connected to a DB when critical DB operations take place, in those cases you have little to do other then run at diferent times.

Another culprit might be a fire-wall.

IHTH,
Well thanks..I did try Batch Sequencer as an alternative, and helped to certain extend. But I guess it has its limitations that all jobs can only be sequentially run one after another, whereas in Sequencer, few jobs can be run in parallel...!! :(
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

That depends on how you trigger you job. If you want to execute jobs parallaly, dont wait for it to finish before other job starts.
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
vijayrc
Participant
Posts: 197
Joined: Sun Apr 02, 2006 10:31 am
Location: NJ

Post by vijayrc »

kumar_s wrote:That depends on how you trigger you job. If you want to execute jobs parallaly, dont wait for it to finish before other job starts.
That's my intent., to run few jobs parallel[y] but with Batch Sequencer, it doesn't allow that way. A Job sequencer, does help though, but I'm having difficulties passing parameters from one job to another
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That's basically because there is no mechanism for doing so. Essentially you will need to touch down a list of file names to disk, which the subsequent job can read. Of course, you could pass the XX parameter to each of the jobs run from the job sequence, and allow it to be used to construct the file name in each job. While you can not get at the value used within the upstream job (except via a Routine activity), you know that the parameter value passed to it can not be changed within that job.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply