Regarding schema file
Moderators: chulett, rschirm, roy
Regarding schema file
Hi
I have created schema file below format and i tried to run the job with
4 columns with data and it is working fine.
i tried to run the job with 2 columns only but it is not running and it is throwing error for 3rd column delimeter is missing.
can anybody suggest me how can i do .
record {final_delim=';', delim=',', quote=none,padchar='#'}
(
name: string[max=255];
addres: nullable string[max=255];
street: nullable string[max=220];
road: nullable string[max=200];
)
I have created schema file below format and i tried to run the job with
4 columns with data and it is working fine.
i tried to run the job with 2 columns only but it is not running and it is throwing error for 3rd column delimeter is missing.
can anybody suggest me how can i do .
record {final_delim=';', delim=',', quote=none,padchar='#'}
(
name: string[max=255];
addres: nullable string[max=255];
street: nullable string[max=220];
road: nullable string[max=200];
)
N R REDDY
-
- Participant
- Posts: 117
- Joined: Wed Feb 06, 2013 9:24 am
- Location: Chennai,TN, India
Are you mentioning from the source the number of columns is 4 but the columns can have null values. Or not sure on number of columns? Anyway you can try as:
If from source they cant provide 4 delimiters, then read the data as single column and split the data in to columns in transformer using field function
If from source they cant provide 4 delimiters, then read the data as single column and split the data in to columns in transformer using field function
Thanks,
Prasanna
Prasanna
So you're saying that your target is static but your input (from a sequential file) is variable?
Edit: What Prasanna said - provided that they can always provided the columns in the same position each time. ie if there is a column X in the full schema in position 3, then column X will always arrive in position 3.
Edit: What Prasanna said - provided that they can always provided the columns in the same position each time. ie if there is a column X in the full schema in position 3, then column X will always arrive in position 3.
-
- Participant
- Posts: 117
- Joined: Wed Feb 06, 2013 9:24 am
- Location: Chennai,TN, India
There are lot of option to get it done. But
Understand how the source team is generating data.
Understand how the source team is generating data.
If there are values for only one non-key column will they send only one delimiter and to which column we will be mapping this?runtime i need to pick 1 key column and any 2 of those 3 columns
Thanks,
Prasanna
Prasanna
Hi
From source we will receive 1 key column and out of 3 remaining columns only 2 columns ,that we are not sure which 2 columns out of 3 columns we receive.
example if 3rd colums is not comming then we dont need to send those column to target..
thats way i mentioned all possible columns in schema file.
please suggest me how to do this
From source we will receive 1 key column and out of 3 remaining columns only 2 columns ,that we are not sure which 2 columns out of 3 columns we receive.
example if 3rd colums is not comming then we dont need to send those column to target..
thats way i mentioned all possible columns in schema file.
please suggest me how to do this
N R REDDY
When you say Any 2 of those 3 columns, are you suggesting that the input file could have a mix of columns?raju4u wrote:From soure i have 4 columns and runtime i need to pick 1 key column and any 2 of those 3 columns.
file is comma separated only,Please suggest me how to do this.
eg
Row 1 of the file could be Key,Col1,Col2
Row 2 of the file could be Key,Col1,Col3
Row 3 of the file could be Key,Col2,Col3
Row 4 of the file could be Key,Col1,Col3
etc
Or would their be separate input files in that example?
No , it is in this format
Row 1 of the file could be Key,Col2,Col3
Row 2 of the file could be Key,Col2,Col3
Row 3 of the file could be Key,Col2,Col3
or in this format
Row 1 of the file could be Key,Col1,Col2
Row 2 of the file could be Key,Col1,Col2
or in this format
Row 1 of the file could be Key,Col1,Col3
Row 1 of the file could be Key,Col2,Col3
Row 2 of the file could be Key,Col2,Col3
Row 3 of the file could be Key,Col2,Col3
or in this format
Row 1 of the file could be Key,Col1,Col2
Row 2 of the file could be Key,Col1,Col2
or in this format
Row 1 of the file could be Key,Col1,Col3
N R REDDY
-
- Participant
- Posts: 117
- Joined: Wed Feb 06, 2013 9:24 am
- Location: Chennai,TN, India
So there will be always two comma delimiters only. Change the schema as
record {final_delim=';', delim=',', quote=none,padchar='#'}
(
name: string[max=255];
col1: nullable string[max=255];
col2: nullable string[max=255];
)
In the transformer, based on the business rule map the columns to target
record {final_delim=';', delim=',', quote=none,padchar='#'}
(
name: string[max=255];
col1: nullable string[max=255];
col2: nullable string[max=255];
)
In the transformer, based on the business rule map the columns to target
Thanks,
Prasanna
Prasanna