DSXchange

I understand this topic has been discussed, however I am not still clear how to solve this issue.

Dataset1
----------
record
( SRC_RECORD_CODE: string[max=2];
SRC_CODE_SEGMENT: string[max=1];
SRC_LIB_SEGMENT: nullable string[max=30];
SRC_CODE_SURSEGMENT: string[max=2];
)

Dataset2
----------
record
( ID_SURSEGMENT: int64;
CODE_SURSEGMENT: string[max=2];
)

Dataset3
----------
record
( SRC_RECORD_CODE: string[max=2];
CODE_SEGMENT: string[max=1];
LIB_SEGMENT: nullable string[max=30];
SRC_CODE_SURSEGMENT: string[max=2];
ID_SURSEGMENT: int64;
)

Dataset4
----------
record
( SRC_RECORD_CODE: string[max=2];
CODE_SEGMENT: string[max=1];
LIB_SEGMENT: nullable string[max=30];
SRC_CODE_SURSEGMENT: string[max=2];
)

1. while I do look up with Dataset1 and Dataset2, I get an output as Dataset3. At the same time I have a reject link also pointing to dataset4 which gets about 15 rejected rows which I am not able to view by using the option "right click and view dataset4" and gives the below error1.

2. By using "Tools --> Data set management" I saw the rejected row and schema (list of all schema is listed above).

3. Please let me know IF RCP hasanything to do with this.

error1:
--------
##E TOPK 000000 12:11:12(020) <_PEEK_IDENT_> Input dataset does not have field: "SRC_CODE_SEGMENT".
##E TOPK 000000 12:11:12(021) <_PEEK_IDENT_> Input dataset does not have field: "SRC_LIB_SEGMENT".
##E TFSR 000019 12:11:12(023) <main_program> Could not check all operators because of previous error(s)
##E TCOS 000029 12:11:12(024) <main_program> Creation of a step finished with status = FAILED.
-------------------------------------------------------------------------
Instead of using the datset4 for reject, if I use a sequential file then I get these types of error2 and the job aborts.

error2:
--------
Sequential_File_131: Error when checking operator: Could not find input field "SRC_CODE_SEGMENT".
Sequential_File_131: When checking operator: When validating export schema: At field "SRC_LIB_SEGMENT": Exporting nullable field without null handling properties
Sequential_File_131: Error when checking operator: Could not find input field "SRC_LIB_SEGMENT".
main_program: Could not check all operators because of previous error(s)
main_program: Creation of a step finished with status = FAILED.

I think you have a input column that is not defined in the output or out put column that not defined in the input, I would try to check the meta data, or try to check the properties in the look up stage for reject.

Dear Ray

Could you give me any tips regarding this issue that I am facing.
I would really appreciate your help.

One more point to add to this topic

1. In lookup stage Condition not met=continue and Lookup failure=Reject

DataSet 2 does not have the columns SRC_CODE_SEGMENT and SRC_LIB_SEGMENT for which you have defined the join on. Both input streams must have the same column names for the join columns (I'm assuming you are using a merge or join rather than lookup in order to get this error)

hi Kryt0n

I am not using "Join" or "Merge", instead i am using Lookup and I am using Dataset2 as a "reference" that is why it is not having SRC_CODE_SEGMENT and SRC_LIB_SEGMENT these column come from my source dataset "dataset1"

From help files:
You cannot set the mapping or edit the column definitions for a reject link. The link uses the column definitions for the primary input link.

Give your dataset4 columns the same name as primary stream

Yes I understand these basic steps, however after doing all the things right in this simple lookup job, I am facing this issue.

I am given all the schema for all the dataset (dataset1, dataset2, dataset3 and dataset4).

so DataSet4 is as follows:
and not as specified in your original post?

Hi Kryt0n

Thanks now I understand the reason for error.

however for DATASET4 I did not type the column name and datatype, I just connected the reject link to the DATASET4.

Do you have any idea why it is not showing the column name and datatype which should be similar to DATASET1.

I dont know how to change the column name and datatype of a reject link

How I have no idea but that appears to be what it is complaining about... try disconnecting the reject stream from the lookup stage and reconnecting...

I did that also "disconnecting and reconnecting"
and then tired by "renaming the job and the dataset"
and then started a new parallel job.
as I had mentioned earlier if I use dataset4 is reject, the dataset4 is created but I can view the data, if I use sequential file instead of dataset4 then the job aborts,

I haven't been able to get a solution for this.


However I changed the name of column for the source similar to that of columns from reference. That is

Dataset1:(Acts as source)
----------
record
( RECORD_CODE: string[max=2];
CODE_SEGMENT: string[max=1];
LIB_SEGMENT: nullable string[max=30];
CODE_SURSEGMENT : string[max=2];
)

Dataset2(lookup dataset)
----------
record
( ID_SURSEGMENT: int64;
CODE_SURSEGMENT: string[max=2];
)

Dataset3(Target dataset)
----------
record
( RECORD_CODE: string[max=2];
CODE_SEGMENT: string[max=1];
LIB_SEGMENT: nullable string[max=30];
CODE_SURSEGMENT: string[max=2];
ID_SURSEGMENT: int64;
)

Dataset4:(reject dataset)
----------
record
( RECORD_CODE: string[max=2];
CODE_SEGMENT: string[max=1];
LIB_SEGMENT: nullable string[max=30];
CODE_SURSEGMENT : string[max=2];
)

Thank you.........

thanush9sep wrote:Dear Ray

Could you ...

Why me? And why when I have not yet responded to this post?

As you note, DataStage expects the column names to be the same for this particular check (same thing - same name). It seems to me that that's a reasonable requirement.

Name changing is free with a Copy stage.

Why you? Because you are The Man!