The target is a flat file which is an input file to another datastage job for some transformations.
thanks
ORACLE9i issue
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 3
- Joined: Thu Sep 09, 2004 8:19 am
- Location: Stockholm / Sweden
- Contact:
More questions:
The target is a flat file, which is processed in the same Datastage server/environment - correct ?
In the transformation stage mentioned, how many more sources is there as input ?
Reason for the questions: It may be possible to solve it within a SP ?
Oh, another question: Is the source table a partioned table (yes I would assume so) ?
And the target after the transformation is Oracle, inside the same server as the sourcetable ?
The target is a flat file, which is processed in the same Datastage server/environment - correct ?
In the transformation stage mentioned, how many more sources is there as input ?
Reason for the questions: It may be possible to solve it within a SP ?
Oh, another question: Is the source table a partioned table (yes I would assume so) ?
And the target after the transformation is Oracle, inside the same server as the sourcetable ?
Best Regards
Lars Hedberg
I'm a Rock Climber on my free time
Lars Hedberg
I'm a Rock Climber on my free time
Folks, the problem here is simple. There is a single Oracle process spooling data once a query has resolved and produces output. There is a finite rate at which a single process can spool data out of Oracle. Once you've achieved that top rate, NOTHING will speed it up.
The ONLY SOLUTION is to have multiple processes spooling data. This fundamental concept is being missed here. The parallel product works on this concept, that you have to use multiple processes to move data.
My earlier recommendation is to segment (partition) your query into logical groups (ranges, mod, etc) and use multiple job instances to each grab a subset of the source data and spool out to a file.
Stored procedures, parallel queries, hints, indexes, NOTHING will help once you have reached the point of a spooling bottleneck.
The ONLY SOLUTION is to have multiple processes spooling data. This fundamental concept is being missed here. The parallel product works on this concept, that you have to use multiple processes to move data.
My earlier recommendation is to segment (partition) your query into logical groups (ranges, mod, etc) and use multiple job instances to each grab a subset of the source data and spool out to a file.
Stored procedures, parallel queries, hints, indexes, NOTHING will help once you have reached the point of a spooling bottleneck.
Kenneth Bland
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
-
- Participant
- Posts: 19
- Joined: Fri Apr 02, 2004 10:13 am
-
- Participant
- Posts: 3
- Joined: Thu Sep 09, 2004 8:19 am
- Location: Stockholm / Sweden
- Contact:
The question is more likly if this is the optimum complete solution - why just look at the <select to flat file> option, when it may very well be more alternatives to the problem if one looks outside the box.
So why I ask is: If the target is within the same Oracle installtion, there might not be a need for a flat file.
So why I ask is: If the target is within the same Oracle installtion, there might not be a need for a flat file.
Best Regards
Lars Hedberg
I'm a Rock Climber on my free time
Lars Hedberg
I'm a Rock Climber on my free time