Commit in Sequential file
Posted: Tue Mar 01, 2005 8:16 pm
Hi,
I have a job that loads data into a sequential file. The jobs takes close to 30 minutes to finish.
If I try to view the data in the sequential file from DataStage designer when the job is running I was able to see the records in the file until a few days ago. But I have noticed that from past few days if I try to view the sequential file data when the job processing records into it it says "data source is empty" even if the performance statistics for the link shows a few number of records went into the file. Another interesting thing is if I wait for some more records to be processed into the file and then try to view the data the data shows up.
I think the data does not physically go into the sequential file until the job is either finished or has processed atleast a certain number of records.
Is there any commit size (like transaction size for RDBMS) for sequential files also? If yes then where do we define it?
Thanks and Regards,
-Sumit
I have a job that loads data into a sequential file. The jobs takes close to 30 minutes to finish.
If I try to view the data in the sequential file from DataStage designer when the job is running I was able to see the records in the file until a few days ago. But I have noticed that from past few days if I try to view the sequential file data when the job processing records into it it says "data source is empty" even if the performance statistics for the link shows a few number of records went into the file. Another interesting thing is if I wait for some more records to be processed into the file and then try to view the data the data shows up.
I think the data does not physically go into the sequential file until the job is either finished or has processed atleast a certain number of records.
Is there any commit size (like transaction size for RDBMS) for sequential files also? If yes then where do we define it?
Thanks and Regards,
-Sumit