Page 1 of 1

Inserting records in form of groups

Posted: Thu Jan 23, 2014 6:14 pm
by suryadev
I have a requirement where I need to send records in groups to the table.
Below is the example

Name Phone Acct
ABC 123 111
DEF 456 111
GHI 789 333
JKL 712 222
MNO 765 222

the input from the source is this and I need to send only records which have same account number at a time like 111 has 2 records, 222 has 2 records and 333 is 1 record. So the first 2 records should go and they have some transformations to be taken care which will handle only same account number records and after the transformations they get inserted into the table. Next the other group with 222 acct no has 2 records so they should do the transformations and then get loaded after the 111 and later 333. So each group having same account number should travel until the target following all the transformations and get inserted into the table.

Please suggest me any solution, thank you

Posted: Thu Jan 23, 2014 8:54 pm
by ray.wurlod
"Be more precise and complete with your specification" would be a good start. The answer may even jump out at you!

Posted: Thu Jan 23, 2014 9:33 pm
by chulett
Seems like "in order" would be more precise than "in groups". :?

Posted: Thu Jan 23, 2014 9:52 pm
by suryadev
Thank you Ray

Actually I did start and followed a process but it did not work, what I did is used looping in transformer like below

In the input transformer I sorted the with the Account number and used 3 stage variables
StageVar1:
Name is NumSavedRows Derivation is SaveInputRecord()
StageVar2:
Name is IsBreak Derivation is LastRowInGroup(Account)
StageVar3:
Name is NumRows Derivation is IF IsBreak THEN NumSavedRows ELSE 0
Loop While : @ITERATION <= NumRows
Loop Variable: SavedRowIndex
Loop Derivation: GetSavedInputRecord()

and all the three links to the output directly and all the way across to the target changed from parallel to sequential processing....but this did not work...pls suggest me if I need to update anything or some other process....

Posted: Fri Jan 24, 2014 7:12 am
by priyadarshikunal
Can you clarify what will be the benefit in sending records like that, if you are sending it to a database can't you just use "order by" there.

Posted: Fri Jan 24, 2014 8:18 am
by suryadev
When taken the same records given below as example our requirement is the account number 111 which has 2 records should go first as they will combine and form an XML column which will be inserted into the table as one record and later the 333 which has 1 record will go into one XML and get inserted into the table and then 222 follows the same path.

So my issue is each time when inserted 3 XML records are inserting but each one has all the 5 records in it even though I dragged all the way to Table sequentially.

Please suggest

Posted: Wed Jan 29, 2014 7:44 am
by priyadarshikunal
try running on one node if its issue just with order.

If issue with the xml created, you may need to look and xml generation part.

Posted: Wed Jan 29, 2014 5:08 pm
by suryadev
Tried different ways but did not work. So created a small job to insert account numbers in one file and sending each account number at a time to the job to do the transformations and loading the records. Thought to use two jobs than doing all the requirements in one job.