performance variance
Moderators: chulett, rschirm, roy
performance variance
Hi,
I want to know the difference between the below designs:
The job design is simple, as extracting from a table and loading into datset
The source table contains 4 million records and we are using 2 node configuration.
1) Oracle stage---->Transformer------>Dataset
2)Oraclestage------->Copy-------->Dataset
3)Oracle stage------->Dataset.
Among those three designs which one will be effective one?
How will be the performance?
Thanks
I want to know the difference between the below designs:
The job design is simple, as extracting from a table and loading into datset
The source table contains 4 million records and we are using 2 node configuration.
1) Oracle stage---->Transformer------>Dataset
2)Oraclestage------->Copy-------->Dataset
3)Oracle stage------->Dataset.
Among those three designs which one will be effective one?
How will be the performance?
Thanks
pandeeswaran
Re: performance variance
As far as i know, you wont find much difference in result, depend the data volume.
Hence you are writing it in a file, i guess you wont find much.
In the coming days, if you want to do something in the data, that time TFM will help.
DS User
Hence you are writing it in a file, i guess you wont find much.
In the coming days, if you want to do something in the data, that time TFM will help.
DS User
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
(2) and (3) are identical, assuming the Force option is not used in the Copy stage. Adding a Transformer stage, even one that transfers data only, will add a small demand for resources.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Maybe, maybe not. For 0 rows, definitely not. Times are only reported in whole seconds, so there may be no measurable difference for a moderate number of rows either. How many will depend upon how wide the rows are; you did not offer that information.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Premium Member
- Posts: 353
- Joined: Mon Jan 17, 2011 5:03 am
- Location: Mumbai, India
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
It's the cheapest stage for renaming columns, dropping columns, re-ordering columns on the link and executing implicit data type conversions.
It's particularly useful for making copies of its input when you need more than one copy.
It's particularly useful for making copies of its input when you need more than one copy.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
You can
take more than one copy of the input data.
shuffle the metadata order
Rename the column
Drop metadata etc.
All depends what you need to do? where you need to use!
Example Scenario: Input date will pass into AGGR stage, as well as to JOIN stage from a COPY stage and then do inner join to combine data....
DS User
take more than one copy of the input data.
shuffle the metadata order
Rename the column
Drop metadata etc.
All depends what you need to do? where you need to use!
Example Scenario: Input date will pass into AGGR stage, as well as to JOIN stage from a COPY stage and then do inner join to combine data....
DS User