i get a CSV file i am reading it and inserting/updating a table based on conditions
I need to get the values of how many rows are in the file: Is there any function that does this
how can i know which row number from the file is being processed ...so that if that row fails i can insert into log file that this rownumber failed to load
i need to know how many rows were inserted into the table as well ... is there nyother way than using sql to do a count(*) from the table and get the number of rows inserted ....
file -- > tranformer -- link1 --> odbc stage1 (insert table)
-- link 2 --> odbc stage2 (update table)
is there any function/macro that will tell me how many rows were sent over from transformer to link1
I'm tryin to get my 1st datastage ETL map done any help would be appreciated
reading file validations and log file
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 457
- Joined: Tue Sep 25, 2007 4:05 pm
Re: reading file validations and log file
Try looking up DSGetLinkInfo()...!dstage443 wrote: is there any function/macro that will tell me how many rows were sent over from transformer to link1
Vivek Gadwal
Experience is what you get when you didn't get what you wanted
Experience is what you get when you didn't get what you wanted
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The row number that you are reading from the file is reported in the Transformer stage via the @INROWNUM system variable.
The number of rows that were sent to the target (whether successfully or not) is reported in the @OUTROWNUM system variable on each output link from the Transformer stage. You can add a reject link to the Transformer stage to capture rows that are rejected by the database or ODBC driver.
Post process the link row counts (including the reject link) to determine the results you seek.
Another possibilty is to use ETLStats, which you can download (free) from Kim Duke's website which will also gather these statistics for you.
The number of rows that were sent to the target (whether successfully or not) is reported in the @OUTROWNUM system variable on each output link from the Transformer stage. You can add a reject link to the Transformer stage to capture rows that are rejected by the database or ODBC driver.
Post process the link row counts (including the reject link) to determine the results you seek.
Another possibilty is to use ETLStats, which you can download (free) from Kim Duke's website which will also gather these statistics for you.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
You can use DS functions in the transformer to get the number of rows from a input link. GetLinkInfo().
Using system variables like @INROWNUM , you will get the number of row that you are inserting/updating or the row that is rejected. If you want to see the total number of rows inserted , then check the log from the DS director.
Sean
Using system variables like @INROWNUM , you will get the number of row that you are inserting/updating or the row that is rejected. If you want to see the total number of rows inserted , then check the log from the DS director.
Sean
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Not quite. @INROWNUM is the number of rows into the Transformer stage, not the number of rows sent into the target. @OUTROWNUM is the number of rows sent to the target, each of these may or may not be successful.
You need reject handling in the job to be confident about how many of the rows were successfully processed into the target.
You need reject handling in the job to be confident about how many of the rows were successfully processed into the target.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Shall i recommend you to read this - Job reads its own link countsdsean wrote:You can use DS functions in the transformer to get the number of rows from a input link. GetLinkInfo().