Transformer stage function
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
Transformer stage function
I am reading a file in a single variable (longvarchar) in my job with a sequential file stage as my source.
the problem is, I do not know how many columns will be there in the input file but the columns are separated by pipe |
minimum number of columns in the file can be 19 and maximum 25.
After I read the file with seq stage, I plan to use a transformer stage to write the columns to the output.
can anyone please suggest me how do I accomplish this and what function I have to use in the transformer stage.
Thanks
Mark
the problem is, I do not know how many columns will be there in the input file but the columns are separated by pipe |
minimum number of columns in the file can be 19 and maximum 25.
After I read the file with seq stage, I plan to use a transformer stage to write the columns to the output.
can anyone please suggest me how do I accomplish this and what function I have to use in the transformer stage.
Thanks
Mark
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
Transformer stage function
I am readng the entire file in 1 column: "columnA"
I have a column Import stage:
Stage properties:
Input:
Import input column: ColumnA
output:
column method: schema file
schema file: /xx/xx/schema_xxx
options:
keep import column = false
reject mode: output
output tab / mapping tab:
No columns showed up. so i could not map anything.
I ran the job and it aborted.
errro message: import field "ColumnA" import validation failed
can anyone please suggest me
Thanks
Mark
I have a column Import stage:
Stage properties:
Input:
Import input column: ColumnA
output:
column method: schema file
schema file: /xx/xx/schema_xxx
options:
keep import column = false
reject mode: output
output tab / mapping tab:
No columns showed up. so i could not map anything.
I ran the job and it aborted.
errro message: import field "ColumnA" import validation failed
can anyone please suggest me
Thanks
Mark
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
Transformer stage function
the job design looks like this;
seq stage ---> column import ---> seq stage
Now the job completes successfully, but all the records are getting rejected from column import stage.
error message:
Column_Import,0: Short input record
Thanks
Mark
seq stage ---> column import ---> seq stage
Now the job completes successfully, but all the records are getting rejected from column import stage.
error message:
Column_Import,0: Short input record
Thanks
Mark
Then perhaps it's not able to handle the "19 to 25" number of columns aspect of this. If that's truly the case, you'll need to fall back on the transformer / field() combination. Unfortunately, I'm not anywhere where I can actually test / play with this.
Last edited by chulett on Mon Feb 22, 2010 1:46 pm, edited 1 time in total.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
Transformer stage function
just for testing mode, I kept the number of columns in the schema file as fixed. (20 columns)
then I run the job and I get the message Short Input Record. It also gives me a mesg, column import,0 Import was unsuccessful at record 0;
The job completed successfully but it is not giving me the expected result.
Thanks
Mark
then I run the job and I get the message Short Input Record. It also gives me a mesg, column import,0 Import was unsuccessful at record 0;
The job completed successfully but it is not giving me the expected result.
Thanks
Mark
-
- Participant
- Posts: 27
- Joined: Mon Jul 09, 2007 1:06 am
-
- Premium Member
- Posts: 263
- Joined: Fri Sep 23, 2005 6:49 am
Transformer stage function
I removed the Prefixes in any schema file and it worked.
Thanks all for your suggestions.
Thanks
Mark
Thanks all for your suggestions.
Thanks
Mark