view

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
laxmi_etl
Charter Member
Charter Member
Posts: 117
Joined: Thu Sep 28, 2006 9:10 am

view

Post by laxmi_etl »

Hi,

My problem is like this

Actually iam loading view from one database to another database table.
One of the views has more than a billion records.
my source stage is ODBC and target is ORACLE stage.

Though it has more than 1 billion records its reading only 138 million records and righting the same number of records to the target.
The job is not erroring out.I mean status is successful.

But my question is what happend to the rest of the count.
Why it is unable to read the all the records.


Thanks.
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Are you sure you are looking at the right View? Maybe it got updated between when you took count and ran your job :?
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
laxmi_etl
Charter Member
Charter Member
Posts: 117
Joined: Thu Sep 28, 2006 9:10 am

Post by laxmi_etl »

I don't think your guess is right because i am running this view from production.The batch cycle in the production for the dims and facts will be done in the morning only.
narasimha
Charter Member
Charter Member
Posts: 1236
Joined: Fri Oct 22, 2004 8:59 am
Location: Staten Island, NY

Post by narasimha »

I would suggest you to use bulk load rather than using the Oracle Stage.
Loading more than 1 billion records would take you for ever with the method you have choosen.
Narasimha Kade

Finding answers is simple, all you need to do is come up with the correct questions.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

You need to verify the actual number of records available from the view and not perhaps the base tables that make up the view.

Views are typically used to constrain output based on whom the view is intended to be used by and to hide any (what could be complex) joins from other processes. Either of those two conditions could legitimately restrict your output to the 138M records you are getting now from the alleged billion you are expecting.
-craig

"You can never have too many knives" -- Logan Nine Fingers
laxmi_etl
Charter Member
Charter Member
Posts: 117
Joined: Thu Sep 28, 2006 9:10 am

Post by laxmi_etl »

Yes, That is what i am doing. I am taking the table count against view count
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

138 Million will has 8 digits and a billion will have 9 digits. Is it possible that you might be missing out on a single digit.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

That's an American billion. An English billion is a million million (from "bi"- and "million") and thus has thirteen digits: 1,000,000,000,000.
:!:
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply