DSXchange

I am reading a file in a single variable (longvarchar) in my job with a sequential file stage as my source.

the problem is, I do not know how many columns will be there in the input file but the columns are separated by pipe |

minimum number of columns in the file can be 19 and maximum 25.

After I read the file with seq stage, I plan to use a transformer stage to write the columns to the output.

can anyone please suggest me how do I accomplish this and what function I have to use in the transformer stage.

Thanks
Mark

In the transformer? Field(). That or a Column Import stage, I would think.

I am readng the entire file in 1 column: "columnA"

I have a column Import stage:

Stage properties:

Input:
Import input column: ColumnA

output:
column method: schema file
schema file: /xx/xx/schema_xxx

options:
keep import column = false
reject mode: output

output tab / mapping tab:

No columns showed up. so i could not map anything.

I ran the job and it aborted.

errro message: import field "ColumnA" import validation failed

can anyone please suggest me

Thanks
Mark

the job design looks like this;

seq stage ---> column import ---> seq stage

Now the job completes successfully, but all the records are getting rejected from column import stage.

error message:
Column_Import,0: Short input record

Thanks
Mark

Then perhaps it's not able to handle the "19 to 25" number of columns aspect of this. If that's truly the case, you'll need to fall back on the transformer / field() combination. Unfortunately, I'm not anywhere where I can actually test / play with this.

just for testing mode, I kept the number of columns in the schema file as fixed. (20 columns)

then I run the job and I get the message Short Input Record. It also gives me a mesg, column import,0 Import was unsuccessful at record 0;

The job completed successfully but it is not giving me the expected result.

Thanks
Mark

OK, what's different about the "short" records? Do they only have 19 fields in them?

1) Read each row as single char using varchar (Seq file Stage)

2) Use following function to read columns (Transformer stage)

Field(Row1,"|", 1)
Field(Row1,"|", 2)
.
.
.
Field(Row1,"|", 20)

You will get expected result ..

The above logic will applicable only for fixed columns

say2prabhu1 wrote:The above logic will applicable only for fixed columns

Delimited columns.

I removed the Prefixes in any schema file and it worked.

Thanks all for your suggestions.

Thanks
Mark