File Processing

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
mimran
Participant
Posts: 7
Joined: Wed Nov 05, 2003 2:56 pm

File Processing

Post by mimran »

Hi,

I've couple of files at a particular location. Using Job Sequence i need to pick each of these files and process one after another. How can i do this??

To add, processing done on each file is also the same.

Thanks in advance.
anupam
Participant
Posts: 172
Joined: Fri Apr 04, 2003 10:51 pm
Location: India

Post by anupam »

do u have some parameter attached to the file Name, say sequence number? If yes, then u can design a job which will have a parameter called Seuence Number. On the basis of this sequence number the files will be picked and the processing will be done. U should also add some parameter in the output file oherwise the same output file will be overwritten.

Now u can call this job from a sequencer passing different values of sequence number for picking different files and the processing can be done.

I hope this will solve ur problem
----------------
Rgds,
Anupam
----------------
The future is not something we enter. The future is something we create.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Create a server job.
Create job control code to select the files one at a time.
Pass the file's pathname as a paramter to your actual job.

The server job is now a custom job sequence, since it contains only code. I recommend placing an Annotation on the canvas to warn future developers that this is the case, that they should not delete the job merely because it contains no stages!
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

Alternative suggestion:

Create an UNIX script using 'dsjob' to run the job with parameters. Ray's suggestion allows for specific definitions of filenames. My suggestion allows for dynamic filenames passed as a Job Parameter based on whatever you want to define your script.

Of course, Ray's suggestion can be expanded to pull filenames from a text file somewhere that you created in advance. Also could pass a long comma-delimited string of files.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It can also pick the file names.
Open the directory as a table, with an Open or OpenPath statement.
Generate a Select List of file names matching your criteria.
Use ReadNext statement in a loop to process the file names.

Code: Select all

* Error checking omitted for brevity
OpenPath dirpath To filevariable
Then
   Select filevariable To 9  ; * selects all files in directory
   Loop
   While ReadNext FileName From 9
      GoSub RunJobForThisFile
   Repeat
End
If you only want to process a subset of the files in the directory, you have two choices:
(a) select all files as above and include an If test on the file name
(b) generate Select List using a SELECT statement executed via DSExecute("UV",...)
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
mimran
Participant
Posts: 7
Joined: Wed Nov 05, 2003 2:56 pm

Post by mimran »

Hi Ray:

The solution which you gave worked perfectly fine. Now, I'm able to pick the files through the server job which just has the job control script and no stages.

I've been busy with other work and wasn't able to test it out until today though i got the solution from you guys long back.

Thanks a lot for your help.

Zafar Imran
Post Reply