DSXchange

Hi,

We are trying to write to a Sequential file stage and then read from the same file in the same job.

When we write to the file, its has a single column which contains the complete row. And when we read that file, we read with 50 columns by providing a delimiter character. But when we read the file, we get an additional binary null value at the end of the row. We tried changing the Line Termination from UNIX style to DOS style. But not able to resolve this.

Please suggest how could we eliminate this binary null at the end. I transferred and looked at the file from notepad. It looked perfect. Any suggestions...


Thanks.

First you need to determine how that null gets there. When you write the file, what termination and attributes are you using and are they the same as when you read the file?

hint: If you don't know this already - you can greatly improve throughput if you make this file a fifo one

We are not giving any termination character. When we write to the file, we give the Delimiter and Quote character as 000. When we read that file, we only change the Delimiter to be as "~", because this is how the columns are separated in the incoming data.

Looking at it in Notepad won't help you much...
Check the contents of the file in a Hex editor. Compare your first input file and the file you are generating at the end (which has the Null character at the end). Do you see a difference?, If you do, then you are inserting the NULL in your job. If not you need to backtrack and see where you are getting this NULL from.

ArndW, I had a question.
How do you make the file fifo?

vnspn - are you working with VarChar() or Char() columns? The 0x00 should not be present in the file. How are you determining that it is present?

Dsnew - there is a little tick box for named pipes in the sequential file stage.

What's the VERSION OF DATA STAGE?

ArndW - All columns are Char(). When we try doing a view data, the last column shows this binary null at the end.

shamshad - The DatStage version is 7.5.2

ArndW - All columns are Char(). When we try doing a view data, the last column shows this binary null at the end.

shamshad - The DatStage version is 7.5.2

With fixed length fields, how does a 0x00 end up in the last column? What tool are you using to determine this ("View Data" can't show nulls, so it must be some other tool)

When we give fixed length and say view data, we get error message and it does not show up any data.
The error message that we get is - "column unexpectedly ended by EOR"

Yes, we are seeing it using view data only. At the last column value, after the actual string value, it shows a thick line like , " | "

I guess the method of transfer from dos to unix is creating this problem ,

If you used binary mode of ftp, try using ascii or otherwise.

By the way what is the delimiter of the file.

Hope this helps.

I guess the method of transfer from dos to unix is creating this problem ,

If you used binary mode of ftp, try using ascii or otherwise.

By the way what is the delimiter of the file.

Hope this helps.

The delimiter between the columns is "~".

But, there is no transfer involved inbetween. The file is created in Unix server and also the read takes place from the same Unix server.

(1) What is the source for this text file that you are creating?
(2) How big is the record length?

You may want to try this

Write the file with these setting at target

RECORD LEVEL
Final Delimiter : end

FIELD DEFAULTS
Quote : None
Delimiter : end