Transformer stage function

[pavan_test]

I am reading a file in a single variable (longvarchar) in my job with a sequential file stage as my source.

the problem is, I do not know how many columns will be there in the input file but the columns are separated by pipe |

minimum number of columns in the file can be 19 and maximum 25.

After I read the file with seq stage, I plan to use a transformer stage to write the columns to the output.

can anyone please suggest me how do I accomplish this and what function I have to use in the transformer stage.

Thanks
Mark

[chulett]

In the transformer? Field(). That or a Column Import stage, I would think.

[pavan_test]

I am readng the entire file in 1 column: "columnA"

I have a column Import stage:

Stage properties:

Input:
Import input column: ColumnA

output:
column method: schema file
schema file: /xx/xx/schema_xxx

options:
keep import column = false
reject mode: output

output tab / mapping tab:

No columns showed up. so i could not map anything.

I ran the job and it aborted.

errro message: import field "ColumnA" import validation failed

can anyone please suggest me

Thanks
Mark

[pavan_test]

the job design looks like this;

seq stage ---> column import ---> seq stage

Now the job completes successfully, but all the records are getting rejected from column import stage.

error message:
Column_Import,0: Short input record

Thanks
Mark

[chulett]

Then perhaps it's not able to handle the "19 to 25" number of columns aspect of this. If that's truly the case, you'll need to fall back on the transformer / field() combination. Unfortunately, I'm not anywhere where I can actually test / play with this.

[pavan_test]

just for testing mode, I kept the number of columns in the schema file as fixed. (20 columns)

then I run the job and I get the message Short Input Record. It also gives me a mesg, column import,0 Import was unsuccessful at record 0;

The job completed successfully but it is not giving me the expected result.

Thanks
Mark

[chulett]

OK, what's different about the "short" records? Do they only have 19 fields in them?

[say2prabhu1]

1) Read each row as single char using varchar (Seq file Stage)

2) Use following function to read columns (Transformer stage)

Field(Row1,"|", 1)
Field(Row1,"|", 2)
.
.
.
Field(Row1,"|", 20)

You will get expected result ..

The above logic will applicable only for fixed columns

[chulett]

The above logic will applicable only for fixed columns

Delimited columns.

[pavan_test]

I removed the Prefixes in any schema file and it worked.

Thanks all for your suggestions.

Thanks
Mark