DSXchange

My assumption is that you are writing to a RDMS. If so then the reject reasons are available for each record if you utilize the reject link. You can get the DBMSCODE, LASTERR,REJECTEDCODE and SQL State.

Thanks for all the suggestions...none of them worked. We finally wrote and active X plug in using the MD5 algorithm which resulting in no more duplicates. It was reasonably fast and after our initial load the deltas were very small. Again the 'official' word from DS support is that CRC32 will produc...

We may want to break this into another thread because the I have been burned when not using column seperators. Padding is something I have never been forced to do...but there is always something new. Yes, I am still in Las Vegas...if I don't get this fixed I may not be here for much longer. Flying h...

If you don't put a some qualifier then you run the risk of getting the same CRC32 results(!!???) for different rows...ie 123 in col1 and 4 in col2 (123,4) should be different than 12 in col1 and 34 in col2 (12,34).

1,2005-08-19 13:17:10.883,18527139,CDS_REDEMPTION,1179677,+,1
1,2005-08-29 10:48:50.470,18776558,CDS_PLAYER,144612819,+,1

Both results in CRC value of 160456061

So the '|' is not the problem.

leslie( BTY just for those who don't know I am MWM better looking than Dennis or Mr. Ray or Mr. Duke.....)

I have a 6 million row input which results in over 2000 CRC32 duplicates. What you will find interesting is that if you start removing data from my supplied examples you will finally get unique answers...more is not better. CRC32 has been around forever. I find it hard to believe that the DS communi...

Per DataStage customer support:javascript: "The CRC32 function in DataStage does support character strings above 32 bits. There is a limitation on checksums to determine duplicate strings and it can return duplicate values." I have several tables which do not have any keys assigned. I buil...

I am not positive that we had any jobs running last night. I will check the projects as suggested. I am suprised that the log, if there is one, would be in a specific project. What if you have jobs running from several projects. leslie

So you come in this morning and the DataStage Services are stopped...All you can find is the 'Not ME' ghost present. Is there any log kept showing when the DS Services are started and stopped?

I will take a look at fuser. Doing more testing with the Folder stage I found that if a file in the folder is open with an editor package, in this case UltraEdit; running the job will result in the filename being passed to the transformer. Since my files are quite large, 25+ meg or greater, I found ...

The folder stage gives me the filenames of the MSSQL log files. But I need some way to determine if a file is still be dumped to by the MSSQL logger. Does anyone have any suggestions for checking to see if a DOS file is open before beginning to read from it. I have tested with the Folder stage and I...

If the comma is always in the same field the then Ereplace function would be one way to make the comma a space. Also I try to never utilize a comma as the separator unless I have no choice. Using a | (pipe) and no "'s seems to keep DS Server happy.

I am running a test as I type....The current 'best' appears to be to drop a FF between the read from DB2 and write to DB2. If I have say 5 million rows to process it will be faster to drop to a ff before beginning the load to DB2. I can't use the bulk loader because the DB2 is on the mainframe. At f...

I agree with your reponse. I just walked into this account today and was told that was the reason for landing the data in the job. I will post some more information once I dig a bit deeper.

leslie

I am reading and writing to the same mainframe/DB2 table using the DSDB2 stage. The only way I have been successful is by utilizing an intermediate file. I wondered if any users have tried using a hash table instead of a Flat file as in intermediate file. Is there any performance improvements using ...