Hello to all again.
Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
Inserting into a sequential file
Pedro.. unfortunately it is not. The NT operating system (which I assume you are running on) only supports one process having write access to a file at a time. I havent tried to reproduce your problem, but I am assuming that the job allows one of the processes to run first and then the other (in essence, overlyaing the data from the first). You may be able to get around this be deleting the file in the job control section of the job. Then in the sequential file stage, specify "append" instead of "clear file". This should allow one of the processes to write all of its data and then the other will add to it.
pedro santos wrote: Hello to all again. Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
---------------------------------
Do You Yahoo!?
Find the one for you at Yahoo! Personals.
pedro santos wrote: Hello to all again. Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
---------------------------------
Do You Yahoo!?
Find the one for you at Yahoo! Personals.
Hi Pedro... another method to use is a hash table. Multiple streams from a
single transformer stage, or multiple transformer stages, can be brought into the same hash table, and then, as needed, taken from there via another output link, and into a sequential stage. Its an extra pass of the data, but will accomplish what you need.
Just check carefully the "order" of the rows in your final output. If they are coming from completely separate transformers (with, presumably, entirely separate origins in the job, which I believe is what your example sounds like), then they are likely running in their own process, so row order
between your two target links will be unpredictable. If they are both
coming from the "same" transformer (two output links from the same icon), then you can more accurately control their ordering with the link order function.
Ernie
-----Original Message-----
From: Tony Curcio [mailto:tonycurcio@yahoo.com]
Sent: Friday, November 16, 2001 11:37 AM
To: datastage-users@oliver.com
Subject: Re: Inserting into a sequential file
Pedro.. unfortunately it is not. The NT operating system (which I assume you are running on) only supports one process having write access to a file at a time. I havent tried to reproduce your problem, but I am assuming that the job allows one of the processes to run first and then the other (in essence, overlyaing the data from the first). You may be able to get around this be deleting the file in the job control section of the job. Then in the sequential file stage, specify "append" instead of "clear file". This should allow one of the processes to write all of its data and then the other will add to it.
pedro santos wrote: Hello to all again. Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
---------------------------------
Do You Yahoo!?
Find the one for you at Yahoo! Personals.
single transformer stage, or multiple transformer stages, can be brought into the same hash table, and then, as needed, taken from there via another output link, and into a sequential stage. Its an extra pass of the data, but will accomplish what you need.
Just check carefully the "order" of the rows in your final output. If they are coming from completely separate transformers (with, presumably, entirely separate origins in the job, which I believe is what your example sounds like), then they are likely running in their own process, so row order
between your two target links will be unpredictable. If they are both
coming from the "same" transformer (two output links from the same icon), then you can more accurately control their ordering with the link order function.
Ernie
-----Original Message-----
From: Tony Curcio [mailto:tonycurcio@yahoo.com]
Sent: Friday, November 16, 2001 11:37 AM
To: datastage-users@oliver.com
Subject: Re: Inserting into a sequential file
Pedro.. unfortunately it is not. The NT operating system (which I assume you are running on) only supports one process having write access to a file at a time. I havent tried to reproduce your problem, but I am assuming that the job allows one of the processes to run first and then the other (in essence, overlyaing the data from the first). You may be able to get around this be deleting the file in the job control section of the job. Then in the sequential file stage, specify "append" instead of "clear file". This should allow one of the processes to write all of its data and then the other will add to it.
pedro santos wrote: Hello to all again. Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
---------------------------------
Do You Yahoo!?
Find the one for you at Yahoo! Personals.
Pedro -
I have noticed from working with the "append" feature that it is unreliable, especially if it is used in the middle of a long, complicated job. It is common to run the same program a number of times in succession with the same input files and get totally different results in output. if youre running Datastage on a UNIX platform, the easiest and most dependable way to handle the problem is to break the job into two separate jobs which are run one after another in a batch. The first job will create File1 and File2. The second job in the batch will execute a before-job script which will concatenate File1 and File2 to create the sequential file you need,
"cat /ds/proj1/data/File1 /ds/proj1/data/File2 > /ds/proj1/data/File3"
File3 can then be brought into the first stage of job 2 of the batch as the sequential file containing all of the records in both File1 and File2. If File3 already has data in it, it will have to be added to the first half of the "cat" command, and the name of the file youre creating will have to be changed.
Good luck, Lindi
>>> pedro_f_santos@hotmail.com 11/16/01 11:27AM >>>
Hello to all again.
Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp
I have noticed from working with the "append" feature that it is unreliable, especially if it is used in the middle of a long, complicated job. It is common to run the same program a number of times in succession with the same input files and get totally different results in output. if youre running Datastage on a UNIX platform, the easiest and most dependable way to handle the problem is to break the job into two separate jobs which are run one after another in a batch. The first job will create File1 and File2. The second job in the batch will execute a before-job script which will concatenate File1 and File2 to create the sequential file you need,
"cat /ds/proj1/data/File1 /ds/proj1/data/File2 > /ds/proj1/data/File3"
File3 can then be brought into the first stage of job 2 of the batch as the sequential file containing all of the records in both File1 and File2. If File3 already has data in it, it will have to be added to the first half of the "cat" command, and the name of the file youre creating will have to be changed.
Good luck, Lindi
>>> pedro_f_santos@hotmail.com 11/16/01 11:27AM >>>
Hello to all again.
Yes its me again with the same basic questions, well im just starting... I wonder if any of you nice people can help-me? Im using two different
transformer stages to input data into an sequential file, the data exiting
in the two transformer stages is equal, the columns are the same, and the
two input stages of the sequential file are appending data to the same file,
yet when i check to see if all data was loaded in that sequential file i
find that only the data coming out of transformer stage one is appended to
the file the data from the transformer two is discarded. Is it possible to
directly load a sequential file from several transformer stages? If so what
can i be doing wrong?
Thank you for your kindness:
Pedro
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp