Multiple columns update

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Multiple columns update

Post by prasad v »

Hi

I have a requirement to load data into database column based on the value coming from input field.
Eg:

Input:

EmpNo, DeptNo, Sal
1234, 10,1000
1234,20,2000
1234,30, 3000
1234,40,4000
2345,10,100
2345,20,200
2345,30,300

Output:

Emp No, Dept_10_Sal,Dept_20_Sal,Dept_30_Sal,Dept_40_Sal
1234, 1000, 2000,3000,4000
2345,100, 200, 300,0
Here we have more than 1000 columns like this Dept_10_Sal, Dept_20_Sal........Dept_1000_Sal.

Can any body advise on this how do we approach this result.

Thanks
Prasad
Last edited by prasad v on Fri Jun 10, 2011 11:46 am, edited 1 time in total.
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Post by prasad v »

Hi Chulett,

Unfortunately i have created this thread in General forum. Can you please move this thread to Parallel extender forum.
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Re: Multiple columns update

Post by samyamkrishna »

Use the trnasformer stage to do the vertical pivot.

Based on the key column.
If the value in the key column is same then append det_10_sal:',':Dept_20_sal and so till the key columns value changes.
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Re: Multiple columns update

Post by prasad v »

samyamkrishna wrote:Use the trnasformer stage to do the vertical pivot.

Based on the key column.
If the value in the key column is same then append det_10_sal:',':Dept_20_sal and so till the key columns value changes.
Can you please explain a bit

Transformer processes the data row by row. how do we append this?
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Re: Multiple columns update

Post by samyamkrishna »

You take three stage variables.

stgvar1 = key column
stgvar2 = if stgvar1=stgvar3 then columnname:',':stgvar2 else ''
stgvar3 = stgvar1

now as long as the key columns are same it will keep appending.

make sure you sort the data on the key column and the transformer runs in a sequential mode.
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Post by prasad v »

Thanks Sam,

You mean stage var2 will be mappped to the outputfield in transformer. then sort -->take the last row-->?How would we write this into seperate field in the database?

Also there may be no value for some of the fields like in the second key value doesn't have 40 deptno. where we should write zero.
samyamkrishna
Premium Member
Premium Member
Posts: 258
Joined: Tue Jul 04, 2006 10:35 pm
Location: Toronto

Post by samyamkrishna »

yes the stgvar2 is mapped to the out put.
If you want to use for a value not present use another stgvar to verify that and put zero instead.
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Post by prasad v »

Thanks sam,

But we will not be knowing which one is missing. Bit confusion on this. Not at all getting finally how do we update
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

prasad v wrote:Hi Chulett,

Unfortunately i have created this thread in General forum. Can you please move this thread to Parallel extender forum.
:wink:

Done.
-craig

"You can never have too many knives" -- Logan Nine Fingers
pandeesh
Premium Member
Premium Member
Posts: 1399
Joined: Sun Oct 24, 2010 5:15 am
Location: CHENNAI, TAMIL NADU

Post by pandeesh »

if your source and target is also a oracle database, then you can load into oracle table via WM_CONCAT or LISTAGG analytic functions which are oracle inbuilt.
pandeeswaran
pandeesh
Premium Member
Premium Member
Posts: 1399
Joined: Sun Oct 24, 2010 5:15 am
Location: CHENNAI, TAMIL NADU

Re: Multiple columns update

Post by pandeesh »

prasad v wrote:Here we have more than 1000 columns like this Dept_10_Sal, Dept_20_Sal........Dept_1000_Sal.
I am curious to know which database you are referring?

In oracle we can't have more than 1000 columns.
pandeeswaran
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Post by prasad v »

Thanks Pandeesh,

We are using Oracle database. I will check these functions.

And We have around thse columns in the table.

Thanks
Naga
prasad v
Participant
Posts: 174
Joined: Mon Mar 30, 2009 2:18 am

Post by prasad v »

I tried in google for this fuction. but could not feel this suports my requirement. As i already mentioned, update should happen in the field based on the value of the column value. Column name is ends with value of column.

Code: Select all

SELECT deptno, wm_concat(ename) AS employees
FROM   emp
GROUP BY deptno;

    DEPTNO EMPLOYEES
---------- --------------------------------------------------
        10 CLARK,KING,MILLER
        20 SMITH,FORD,ADAMS,SCOTT,JONES
        30 ALLEN,BLAKE,MARTIN,TURNER,JAMES,WARD
Thanks
Naga
Devendrudu
Participant
Posts: 31
Joined: Tue May 24, 2011 12:16 am

Re: Multiple columns update

Post by Devendrudu »

Hi friends, this is will work by vertical pivoting.


|--------------------------------------filter
| | | | |
| | | | |
src------------->copy---------->removedupli------>lookup-----------trg



1. in copy stage take two o/ps

2.in remove duplicat based on key

3. in filter take 4 conditions
sal= deptno10
sal=deptno20
sal =deptno30
sal= deptno40

in o/p change col name deptno= deptno_10,

deptno= deptno_20,
like that u can give o/p and pass to target.

i think this will work i also done same type of scenario.
i am waiting for ur replay.
Devendra,Bangalore,India
Post Reply