DSXchange

samsuf2002

Thanks for your reply DSguru2B.

I will try the steps you showed me and my job design is just an oracle stage followed by data set. I am using auto partitioning on data set.

samsuf2002

Hi, My job loads data from Oracle stage into a data set, there are around 6 columns and 200K records. We are doing some inner joins in Oracle stage itself, job takes around 2 hrs to finish on 8 nodes. My question is what will be the best way to increase the performance of the job ? Is writing a join...

samsuf2002

Make sure the format you specified in sequential file stage is same as actual file, example the format , delimeter etc.

samsuf2002

Do I need a patch to fix this ?

samsuf2002

Hi , I am using a stored procedure stage in my PX job for oracle db, i want to specify environmental variable for the data source and data base fields. I gave $dbname for database field its working fine but when I use $dbsource for data source field its aborting with this error "Error: ORA-1215...

samsuf2002

What is your target ?, if it is a file then you can pass it through another transformer and put a constraint to capture ?s for that column and reject it.

samsuf2002

Hi , I have a requirement to split my data into multiple files each containing 100K records, now I have my input row count 4 million which need to be split into 40 files. What could be the easy way to do it ,right now I am just using link partitioner....is it efficient for just one time load. thanks...

samsuf2002

I replaced output hashed file with sequential file in the job design i showed above , the job took 20 mins to load all 10 million in the output seq file. Later i am using another job to load seq file into hash file and its very slow like 600 row/sec.

samsuf2002

Chulett I am using stage variables in transformer instead of pivot stage for doing vertical pivot ....my job design will be

Code: Select all

Seq file --- link partitioner----4 streams of trnfrm ----- link collector ----- hashed file

samsuf2002

Hi All , I am running job which does vertical pivot creating 150 columns in output hashed file, I have 10 million rows in input seq file . The job takes 11 hrs to create the hash file ....i am using link patitioner and link collector but it is not helping ....can any one please suggest me how can I ...

samsuf2002

I can mark it as resolved then. Thanks to all.

samsuf2002

Thanks to all, its working fine....i used the idea given by Rajee by holding code values in one stage variable and then using Field() to split the values in different output columns (in my case it will be 20 output columns)...... Now my concern is whether its worth using hashed file because my data ...

samsuf2002

No Chulett maximum number of pivoted columns can be more upto 20.

samsuf2002

Input id ---

Code: Select all

Key_col Code_col
123       100
123       200
123       300
345       500
345       600
345       700
345       800

Output should be --

Code: Select all

Key_col Code_col1 Code_col2 Code_col3 Code_col4
123        100       200       300      
345        500       600       700       800

[/list]

samsuf2002

Nivas- I did as you said and as i found in some of the helpful posts but now my output looks like key col A and B col having values X,Y,Z,W ....but my requirement is to get X,Y,Z,W in 4 different columns...

DSXchange

Search found 394 matches

Oracle stage performance

stored procedure stage

splitting records into multiple files

Creating hashed file takes too long