Oracle Tables with many columns

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
yugee
Participant
Posts: 34
Joined: Fri Feb 04, 2011 5:54 pm

Oracle Tables with many columns

Post by yugee »

Hi,

Sorry if this is a repetitive post, I have searched and couldn't find any thing.

I have a situation where I am extracting data from Oracle table with more than 200 columns and loading into an another Oracle table with same structure. I need to select all the columns. I have to do some transformation before loaind the data. I am facing an issue in defining more than 200 columns in SELECT Quey as well as INSERT Query (the column definition in Oracle Conector Stage). Is there any way:

> to define the columns in a better and faster way
> Can I use SELECT * function to select all the columns and dynamically define the columns in the table?

Thanks
Yugee.
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Look into Run-time Column Propagation (RCP) in the Parallel Job Developer's Guide.

It allows you to read a table using SELECT * without defining the column names in advance. You can reference the columns you DO want to modify at any time. When you output the table it will write out ALL the columns even if you have not explicitly defined them.

It is VERY useful for tables with numerous columns. RCP must be enabled by the Administrator for the project before it will work. I do NOT recommend making it the default for all links (another option) because that can cause issues.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Or... if you are crafting your own SQL that selects the columns, let the stage generate that for you at runtime. No need for any 'custom / user defined' SQL for anything of that nature.

Note that this is not the same as RCP.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply