Hello,
I have a job designed for reading data using sequential stage where I do some transformations and load to new file.
I have around 2000 files to follow the same process, they all have the same layout and they are dated depending on their arrival.
Is there any processto automate this in a parallel job? Please suggest
Thank you
Reading no of files
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 248
- Joined: Fri Jul 30, 2010 9:04 am
Reading no of files
Regards,
Kenny
Kenny
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
-
- Participant
- Posts: 248
- Joined: Fri Jul 30, 2010 9:04 am
@Priyadarshikunal
Order does not matter
@Chulett
ok, I will use wild card like filename*.txt so that it pulls all the files.
When loading it does not need to be loaded one at a time, new files can be created all at a time.
I will use looping if they need to be loaded one at a time in sequence, if that is the case shall the use the same parallel job in the sequence job and execute it at a time?
Thank you
Order does not matter
@Chulett
ok, I will use wild card like filename*.txt so that it pulls all the files.
When loading it does not need to be loaded one at a time, new files can be created all at a time.
I will use looping if they need to be loaded one at a time in sequence, if that is the case shall the use the same parallel job in the sequence job and execute it at a time?
Thank you
Regards,
Kenny
Kenny