Hi
We just migrated our datastage installation from AIX to LINUX and we have run into an issue with the sequential file stage. The sequential file stage is used to read a file and store the fields in an oracle table.
The sequential file is reading files that may have CRLF or just LF as record delimited.
In AIX, the sequential file stage, with the FINAL_DELIMITER set to END, is able to strip both CRLF and LF from the last field before storing it into a DB.
But under LINUX, it is only stripping the LF from CRLF leaving the CR as part of the last field which then gets stored in the DB along with the field value as a non printable character.
Short of changing the jobs and the scripts, what else can we do to make LINUX behave more like AIX where without any prompting it is able to understand that CRLF or LF are both record terminators and need to be stripped.
Sequential file CR/LF working differently in AIX vs LINUX
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Thanks for your responses guys but this is a strange problem
To answer your question Chulett, Every file will either have a CRLF or just LF. But we can't predict which file will have which line delimiter.
The issue is that when we had these jobs running in AIX environment, the sequential file stage, with its FINAL_DELIMITER=END, was somehow able to correctly process both kinds of files.
When a file had CRLF, it removed the CRLF from the last field and when it had LF, it removed the LF.
But now since we moved everything to LINUX, it is only removing LF from both kinds of files leaving CR untouched which is now ending up in the database as part of the last field.
Quesiton - What was making AIX handle both CRLF and LF correctly.
And how to we make LINUX behave in a similar manner.
To answer your question Chulett, Every file will either have a CRLF or just LF. But we can't predict which file will have which line delimiter.
The issue is that when we had these jobs running in AIX environment, the sequential file stage, with its FINAL_DELIMITER=END, was somehow able to correctly process both kinds of files.
When a file had CRLF, it removed the CRLF from the last field and when it had LF, it removed the LF.
But now since we moved everything to LINUX, it is only removing LF from both kinds of files leaving CR untouched which is now ending up in the database as part of the last field.
Quesiton - What was making AIX handle both CRLF and LF correctly.
And how to we make LINUX behave in a similar manner.
vishal