dos to unix on Linux environment

[sonia jacob]

Hi All,

I use dos2unix to convert source files on the windows machine to unix file on a SunSolaris machine. After which I can process the file without the option "Record Delimitor String = Dos format". on the sequential file stage

But dos2unix does not seem to work on a Linux box.

Has anyone tried dos2unis on linux?, faced similar issue?

[DSguru2B]

dos2unix and unix2dos are present on some platforms. Try doing man dos2unix or man unix2dos, if there is help present, then it is most probably supported. Only your id doesnt have access to it. You need to contact your system admin for that.

[ray.wurlod]

Don't know if it exists on Linux or not. But even if it does, it may be in a directory not mentioned in your PATH environment variable (which specifies the directories to be searched when looking for a command).

What error message is generated?

[sonia jacob]

Don't know if it exists on Linux or not. But even if it does, it may be in a directory not mentioned in your PATH environment variable (which specifies the directories to be searched when looking for ...

Command : dos2unix file.csv file2.csv

Error:
dos2unix: converting file file.csv to UNIX format ...
dos2unix: problems converting file file.csv

[I_Server_Whale]

Like previously asked by DSGuru2B, do you have permission to write in the directory where your output file exists. Just want to make sure that this is not a permissions issue.

Whale.

[DSguru2B]

Try

Code: Select all

dos2unix file1 > file2

[ray.wurlod]

OK, that error message tells you that dos2unix does exist on your server machine, since it was the dos2unix command itself that generated the error message.

It's also telling you that there's something about file.csv that it did not like - could not convert. You will need to scrutinize file.csv with an intense scrute (or a hex editor) to work out what this is - maybe the FTP utility automatically converted the line terminators.

[DSguru2B]

ftp does convert it from dos to unix and back. I think we needed to ask this question first, why do you need to use that command in the first place???

[chulett]

FTP can convert it, I wouldn't say 'does'. Way too many packages seem to default to a binary transfer which means you end up with a DOS file on your UNIX system.

However, if you know that's what you are going to get why not just read it as a DOS file? Curious what's so horrible about using the DOS record delimiter setting?

[DSguru2B]

Well, I have always created test files on my desktop and ftp'd them over to the unix environment. Just a simple put command. It would convert it. No special command to convert it along with ftp. That is due to the fact that command line ftp uses ascii transfer mode as a default mode. The recieving machine saves those transfered bytes in the appropriate format, i.e, unix saves it in unix format, machintosh saves it in Mac format and so on.

[chulett]

No-one except gomers like us would use command line ftp from a Windows box.

Here, any business type folk who provide Excel csv files (typically) via ftp all use some sort of GUI package... and for that same reason we practically always get DOS files as a result. Go figure.

[DSguru2B]

Well, i guess then the OP needs to either convert the format explicity like he/she has been trying or as Craig said, use Dos record end setting. Check out thislink for a few more ways of doing that.

[sonia jacob]

No-one except gomers like us would use command line ftp from a Windows box.

Here, any business type folk who provide Excel csv files (typically) via ftp all use some sort of GUI package... and for that same reason we practically always get DOS files as a result. Go figure.

Well, I have always created test files on my desktop and ftp'd them over to the unix environment. Just a simple put command. It would convert it. No special command to convert it along with ftp. That is due to the fact that command line ftp uses ascii transfer mode as a default mode. The recieving machine saves those transfered bytes in the appropriate format, i.e, unix saves it in unix format, machintosh saves it in Mac format and so on.

We have all our sources files put on a Windows machine which is mounted on a Unix box (its like having mapped drive)

ETL connects to the windows directory using a sequential stage and reads the file with a records delimitor = DOS format. And we were good

Now the source files (.csv) started to have blank lines in them which sequential stage does not like. Hence we called a shell script that would convert the file to unix move to the unix etl server and remove the blank lines.

Before have a shell script to remove blank lines I did try to use reject link hoping that the sequential stage would just reject the record which is not per the format (blank lines). But it throws a warning which is a issue to us.

[snt_ds]

tr worked for me.

tr -d '\r' < FILENM > FILENM.dos2unix

thanks to all.

[chulett]

And who would you be?