Our ETL load 70 tables in our warehouse. All the sources are text file (.TXT).
A mainframe job ftps all the source file to the Data Stage server. What we want to achieve is when the LAST SOURCE FILE is COPIED successfully to the Data Stage server, we want to trigger the MASTER JOB in Data Stage.
How can this be done? Please share your views. Any help will he greatly appreciated.
How to start a Sequencer from Mainframe
Moderators: chulett, rschirm, roy
The way I have done before is that after your mainframe has FTP'ed all the files, it also sends out a flag to the specific directory of the DS server. Then your sequnece job has a Wait_For_File_Activity first to pick up this flag and subsequently to process other jobs in the sequence.
Hope this will give you an idea.
Hope this will give you an idea.
If you absolutely wish the mainframe processes to initiate the load, and avoid a scheduler, then you simply need to start the jobstream using the command line program dsjob on the unix server. You can invoke this using any means your mainframe has for running remote programs on other machines. In unix we'd suggest ssh, remsh, rsh, rlogin, etc.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
If the FTP transfers are under control of an enterprise scheduler, such as Control-M, SeeBeyond, etc., then the same scheduler can be used to initiate the DataStage job sequence.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Re: How to start a Sequencer from Mainframe
Hishamshad wrote:Our ETL load 70 tables in our warehouse. All the sources are text file (.TXT).
A mainframe job ftps all the source file to the Data Stage server. What we want to achieve is when the LAST SOURCE FILE is COPIED successfully to the Data Stage server, we want to trigger the MASTER JOB in Data Stage.
How can this be done? Please share your views. Any help will he greatly appreciated.
We have done somthing like that -
One way is ---
Code: Select all
you can pass the Unix shell script name in the PARAM part of your JCL
Now once the JCL triggers the process it triggers the datastage job from unix by a dsjob -run
Code: Select all
A Unix daemon checks for new files say in a Inbox directory. It compares file size to check if transfer is done. then move the file to SrcFiles folder with timestamp appended. Then a daemon process running in Unix wakes up in a certain interval- gets all the file available. I use a FILE_LIST table and have a column called process_flg. So the second daemon checks the file with process_flg as Y and kick of that job via dsjob run
Avishek Mukherjee
Data Integration Architect
Chicago, IL, USA.
Data Integration Architect
Chicago, IL, USA.
Re: How to start a Sequencer from Mainframe
Avishek Mukherjee,
Appreciate if you can share the unix code in a generic way.
Thanks.
Appreciate if you can share the unix code in a generic way.
Thanks.
avi21st wrote:
Another way which I have implemented is by file watcher concept using two unix daemon scripts.
These are very standard process-let me know if you have questionsCode: Select all
A Unix daemon checks for new files say in a Inbox directory. It compares file size to check if transfer is done. then move the file to SrcFiles folder with timestamp appended. Then a daemon process running in Unix wakes up in a certain interval- gets all the file available. I use a FILE_LIST table and have a column called process_flg. So the second daemon checks the file with process_flg as Y and kick of that job via dsjob run