How to generate a unique Multiple Instance indentifier

[rodre]

Hello!
I have Server Job that picks up files as they come into a folder and loads them into a SYBASE Database. The job needs to run each time a file comes in, however, at times there might be multiple files coming in at the same time, of which should run simultaneously using the "Allow Multiple Instance" for each file (each file has 1 record).

I have two questions:
1) How can I generate (through code) a unique Multiple Instance Identifier to use each time the job is run?

2) How can select one file at a time from the folder to pass through, without picking up a file name again when is already running through an Instance (when files arrive simultaneously)? I will never know what the file name is but they would look something like this: RX.MIRSR.20061031.105227.C78

Thank you in advanced; your help is much appreciated!!
Rodre

[talk2shaanc]

I dont know about sybase. But first question, before discussing on logic: is multiple write to a database possible ? at least for DB2 and Oracle its not. Basically the table is locked first and then data is written, so at a time only one process(DS Job) will be writing to your Database.
I assume it would be same for Sybase.

[ray.wurlod]

(1) Through a call to the SDK Key Management functions (which effectively implement sequences for DataStage).

(2) You could only achieve this using your own job control routine. It would need to check all currently-running instances (DSGetJobInfo(hJob, DSJ.JOBINVOCATIONS)) and the file name/path associated with each (or keep track of them as the run requests are issued).

[rodre]

Shantanu,
Thank you for the information. I found out and it does not work in SYBASE either. However, I did try in a Hashed file and it does allow you to do multiple write (unless I missed something and I am proving wrong).
Can this be possible doing bulk updates into a database?

Ray,
Thank you for the info. I will look into this...

[dcguuenther]

Don't do this.

It fills up your logs with a new instance for every file and its a real pain. you could feasible go in and delete these instances then to clean the log but then you lose a clean visibility into your log.

For example, say you start and then leave. You come back and you have run 100 files. Now you have 100 instances of that job in your directory folder and it takes FOREVER to come up.

Why can you run the files sequentially? Process one, then process the next. If that is not feasible, I would come up with a set of instance ID's (say 1-10) so that you resuse the same ones over and over without having to clean up your logs)

Just my opinion

[ray.wurlod]

You can do that with my approach too. Just re-use the invocation IDs. Keep a list of them in memory (that is, in a variable or an array), grab the first unused slot.

[rodre]

I am using a routine that takes all the files in a folder, one at a time (this parts works fine)...., and then run a "Multiple Instance" Job in a loop (code bellow). It works if I let the job finish, using the DSWaitForJob to let it finish, however, how can I run the same job using the "Multiple Instance" feature without having to let the job finish in a loop? Is it possible? Is there a better way?

Here is the code:

Code: Select all

         While ReadNext FileName From 5
         Instance = MyInstace + 1
            hJob = DSAttachJob("MirsLoader_Test2":'.':Instance, DSJ.ERRNONE)
            ErrCode = DSSetParam(hJob, "FileName", FileName)
            ErrCode = DSRunJob(hJob, DSJ.RUNNORMAL)
            ErrCode = DSWaitForJob(hJob)
            ErrCode = DSDetachJob(hJob)
         Repeat

[talk2shaanc]

Remove
ErrCode = DSWaitForJob(hJob)
ErrCode = DSDetachJob(hJob)
from this loop and add it to another loop.

[Kirtikumar]

If you remove the DSDetachJob line from the loop then also remove the following line:
hJob = DSAttachJob("MirsLoader_Test2":'.':Instance, DSJ.ERRNONE)