Use of combine records stage

dsguy · Post by **dsguy** » Sat Feb 17, 2007 4:21 am

Hi

I am trying to use a combine record stage Please let me know how to use or give me an example what exactly wht it does

Thanks in advance

ArndW · Post by **ArndW** » Sat Feb 17, 2007 5:26 am

dsguy - this site is staffed by unpaid volunteers, the premium memberships help defray the costs of running this site.
Since posters have a choice of which questions to respond to, you have the best chances of receiving help if your post details exactly what your problem is and what you've tried to address that issue.

I read the following into your question:

a) You haven't looked into the Parallel Job Developer's Guide or the on-line help.
b) You haven't experimented in trying to get this stage to work.

Thus my thought processes are pretty simple - if the poster hasn't put much effort into the problem, why should I?

Perhaps I am mistaken, in that case please accept my apologies, reply and post what you've tried and what doesn't work and I'm sure you'll get some assistance.

ray.wurlod · Post by **ray.wurlod** » Sat Feb 17, 2007 2:23 pm

The Combine Records stage combines identically-keyed records from its single input into a single record containing a vector of subrecords.

dsguy · Post by **dsguy** » Mon Feb 19, 2007 12:13 am

Hi ArndW

Sorry for the delay in my reply

I am currently working in DataSate PX 7.5.I have never got a chance to use combine records,make subrecords,make vector,promote surecord, split subrecord,split vector and i am not sure when to use these stages to get an understanding i am executing ther jobs using this stages.
It tried to get understand the same from the guide but no ware an example is given so i can understand it and I tried to work on it and job was aborting,

Sorry if I have not given enough inputs .

Thanks & Regards

ArndW · Post by **ArndW** » Mon Feb 19, 2007 2:30 am

dsguy - I apologize for my earlier post about assuming you hadn't tried anything.

What was your error message when you tried this?

dsguy · Post by **dsguy** » Mon Feb 19, 2007 3:14 am

Hi AndeW

Thanks for you time and your reply.
Actually what i assume a combine records stage does is for the same key column it combines the records.
For example

Input OutPut
CLCL_ID NAME CLCL_ID Name
1 A 1 ABC
1 B
1 C

Please let me knwo if my assumption is wrong

Please find the error which is encountered
1)sequential_File_67: Error when checking operator: Input "NAME" is not a value field.
2)Combine_Records_70: When checking operator: The modify operator keeps field "CLCL_ID" which is not present in the output interface.

Please let me now how to proceed on this

Thanks ,
dsguy

ArndW · Post by **ArndW** » Mon Feb 19, 2007 3:56 am

Start with the sequential file error. Do you have a column called NAME and can you view data or read the file without the combine records stage?

dsguy · Post by **dsguy** » Mon Feb 19, 2007 5:16 am

Hi AndrW

I have a columname NAME. The jobs runs fine with out a combine records stage . Please Let me knwo whether the example give by me for a cobine records stge is corrrect? IF not give an example so i can follow the same way.

Thanks & Regards

ArndW · Post by **ArndW** » Mon Feb 19, 2007 5:39 am

Are you declaring your combine stage output schema or using RCP? I just created a test job with 3 columns and combined without any problems.

ray.wurlod · Post by **ray.wurlod** » Mon Feb 19, 2007 2:55 pm

Please post the record schema from both the input and output links of your Combine Records stage.

nvuradi · Post by **nvuradi** » Mon Nov 05, 2007 10:56 pm

ray.wurlod wrote:Please post the record schema from both the input and output links of your Combine Records stage. ...

Even i got the same error message when i tried to write the combined record (assuming which is a vector) to seq file. but when i replaced sequential file with a dataset the job ran fine, but when i tried viewing the data from dataset i all i could see was

[() ()]
[() () () () () () () () () () () () ()]
[() () () () () () () () () () () () ()]
[() () () () () () () () () () () () () () () () () () () () () ()]

how do i convert to readable form so that my next job can do the processing on this.

Regards
Narendar

ray.wurlod · Post by **ray.wurlod** » Tue Nov 06, 2007 3:06 am

That's not the record schema. The record schema can be obtained from the generated OSH (in job properties, if enabled) or from the score.

nvuradi · Post by **nvuradi** » Tue Nov 06, 2007 3:12 am

ray.wurlod wrote:That's not the record schema. The record schema can be obtained from the generated OSH (in job properties, if enabled) or from the score. ...

The generated OSH for the Combine sub records stage is

#################################################################
#### STAGE: Combine_Records_72
## Operator
aggtorec
## Operator options
-subrecname 'text'
-key 'SUBM_I'

## General options
[ident('Combine_Records_72'); jobmon_ident('Combine_Records_72')]
## Inputs
0< 'Sequential_File_65:DSLink68.v'
## Outputs
0> [modify (
keep
text;
)] 'Combine_Records_72:DSLink69.v'
;

#################################################################

ray.wurlod · Post by **ray.wurlod** » Tue Nov 06, 2007 3:17 am

That's the stage OSH, not the record schemas. Record schemas begin with -record and contain column definitions enclosed in a pair of curly braces.

nvuradi · Post by **nvuradi** » Tue Nov 06, 2007 3:41 am

ray.wurlod wrote:That's the stage OSH, not the record schemas. Record schemas begin with -record and contain column definitions enclosed in a pair of curly braces. ...

The Input Record schema is
(
Key:int32 {quote=none};
A:string[max=255];
B:string[max=255];
C:string[max=255];
D:int32 {quote=none};
E:nullable int32;
)

And i used Key as the key in Combine Subrecords stage