Page 1 of 1

reading file validations and log file

Posted: Sun Jun 01, 2008 9:27 am
by dstage443
i get a CSV file i am reading it and inserting/updating a table based on conditions

I need to get the values of how many rows are in the file: Is there any function that does this

how can i know which row number from the file is being processed ...so that if that row fails i can insert into log file that this rownumber failed to load

i need to know how many rows were inserted into the table as well ... is there nyother way than using sql to do a count(*) from the table and get the number of rows inserted ....

file -- > tranformer -- link1 --> odbc stage1 (insert table)
-- link 2 --> odbc stage2 (update table)


is there any function/macro that will tell me how many rows were sent over from transformer to link1


I'm tryin to get my 1st datastage ETL map done any help would be appreciated

Re: reading file validations and log file

Posted: Sun Jun 01, 2008 1:18 pm
by vivekgadwal
dstage443 wrote: is there any function/macro that will tell me how many rows were sent over from transformer to link1
Try looking up DSGetLinkInfo()...!

Posted: Sun Jun 01, 2008 2:25 pm
by ray.wurlod
The row number that you are reading from the file is reported in the Transformer stage via the @INROWNUM system variable.

The number of rows that were sent to the target (whether successfully or not) is reported in the @OUTROWNUM system variable on each output link from the Transformer stage. You can add a reject link to the Transformer stage to capture rows that are rejected by the database or ODBC driver.

Post process the link row counts (including the reject link) to determine the results you seek.

Another possibilty is to use ETLStats, which you can download (free) from Kim Duke's website which will also gather these statistics for you.

Posted: Sun Jun 01, 2008 2:36 pm
by dsean
You can use DS functions in the transformer to get the number of rows from a input link. GetLinkInfo().

Using system variables like @INROWNUM , you will get the number of row that you are inserting/updating or the row that is rejected. If you want to see the total number of rows inserted , then check the log from the DS director.


Sean

Posted: Sun Jun 01, 2008 4:04 pm
by ray.wurlod
Not quite. @INROWNUM is the number of rows into the Transformer stage, not the number of rows sent into the target. @OUTROWNUM is the number of rows sent to the target, each of these may or may not be successful.
You need reject handling in the job to be confident about how many of the rows were successfully processed into the target.

Posted: Mon Jun 02, 2008 3:01 am
by ag_ram
dsean wrote:You can use DS functions in the transformer to get the number of rows from a input link. GetLinkInfo().
Shall i recommend you to read this - Job reads its own link counts