Page 1 of 1
How to start a Sequencer from Mainframe
Posted: Mon Oct 02, 2006 12:12 pm
by shamshad
Our ETL load 70 tables in our warehouse. All the sources are text file (.TXT).
A mainframe job ftps all the source file to the Data Stage server. What we want to achieve is when the LAST SOURCE FILE is COPIED successfully to the Data Stage server, we want to trigger the MASTER JOB in Data Stage.
How can this be done? Please share your views. Any help will he greatly appreciated.
Posted: Mon Oct 02, 2006 1:07 pm
by lstsaur
The way I have done before is that after your mainframe has FTP'ed all the files, it also sends out a flag to the specific directory of the DS server. Then your sequnece job has a Wait_For_File_Activity first to pick up this flag and subsequently to process other jobs in the sequence.
Hope this will give you an idea.
Posted: Mon Oct 02, 2006 2:49 pm
by kcbland
If you absolutely wish the mainframe processes to initiate the load, and avoid a scheduler, then you simply need to start the jobstream using the command line program dsjob on the unix server. You can invoke this using any means your mainframe has for running remote programs on other machines. In unix we'd suggest ssh, remsh, rsh, rlogin, etc.
Posted: Mon Oct 02, 2006 3:11 pm
by ray.wurlod
If the FTP transfers are under control of an enterprise scheduler, such as Control-M, SeeBeyond, etc., then the same scheduler can be used to initiate the DataStage job sequence.
Re: How to start a Sequencer from Mainframe
Posted: Mon Oct 02, 2006 3:50 pm
by avi21st
shamshad wrote:Our ETL load 70 tables in our warehouse. All the sources are text file (.TXT).
A mainframe job ftps all the source file to the Data Stage server. What we want to achieve is when the LAST SOURCE FILE is COPIED successfully to the Data Stage server, we want to trigger the MASTER JOB in Data Stage.
How can this be done? Please share your views. Any help will he greatly appreciated.
Hi
We have done somthing like that -
One way is ---
Code: Select all
you can pass the Unix shell script name in the PARAM part of your JCL
Now once the JCL triggers the process it triggers the datastage job from unix by a dsjob -run
Another way which I have implemented is by file watcher concept using two unix daemon scripts.
Code: Select all
A Unix daemon checks for new files say in a Inbox directory. It compares file size to check if transfer is done. then move the file to SrcFiles folder with timestamp appended. Then a daemon process running in Unix wakes up in a certain interval- gets all the file available. I use a FILE_LIST table and have a column called process_flg. So the second daemon checks the file with process_flg as Y and kick of that job via dsjob run
These are very standard process-let me know if you have questions
Re: How to start a Sequencer from Mainframe
Posted: Tue Oct 03, 2006 9:51 am
by venkat_kp
Avishek Mukherjee,
Appreciate if you can share the unix code in a generic way.
Thanks.
avi21st wrote:
Another way which I have implemented is by file watcher concept using two unix daemon scripts.
Code: Select all
A Unix daemon checks for new files say in a Inbox directory. It compares file size to check if transfer is done. then move the file to SrcFiles folder with timestamp appended. Then a daemon process running in Unix wakes up in a certain interval- gets all the file available. I use a FILE_LIST table and have a column called process_flg. So the second daemon checks the file with process_flg as Y and kick of that job via dsjob run
These are very standard process-let me know if you have questions