DSXchange

The problem arises when I try to send data to another server... As I stated above, when writing to a local file inestead get a transfer rate of 14000 rows/sec. Writing the SQLLDR datafile when using bulk loader is about 6000 rows/sec. The transfer of these data with sqlldr gives med 1000 rows/sec. T...

When I look at the logs I get the following entry for performance statistics: TestfAvtalerFraFil_MindreFil_Bulk..CTransformerStage1: DSD.StageRun Performance statistics Name Percent Count Minimum Average TestfAvtalerFraFil_MindreFil_Bulk..FilAvtale.AvtalerInn 0 222265 0 0 TestfAvtalerFraFil_MindreFi...

What does this variable do?

I have tried all of the suggestions above: - Running in manual mode and then doing the sqlldr part manually. This gave me a transfer rate of about 1000 rows/sec. - Placing an IPC-stage between the Transformer and the Bulk-loader stage. This gave me a performance of about 600 rows/sec I tried connect...

The pingtime from the DS-server to the oracle server is 0.13 ms.

There are no indexes and no constraints at all on the target table. The table is generated from a template and indexes are created after the data load is completed.

I can't say I have control of this yet. I developed a job with an Oraociblk target. This had about the same performance as the ordinary OCI stage. I also tried this with and IPC-stage to divide the reading from the flat file and the writing of the bulk stage into two different processes, no performa...

Bulkloader was not as easy to figure out as I thought.... On my DS-server (7.5.1A) I have two different Bulk loader stages. One is called Oraociblk (Oracle 8.1 Bulk Loader). This one has loads of parameters to be set. The other is ORABULK (Oracle 7 Load), this is the one that corresponds to the Deve...

It seems this is a problem with network congestion/infrastructure.

I changed the job to write directly to a flat file instead of the database. This gave me 5600 rows/sec. When using the database the server's load was almost non-existent according to top.

I'll see what Bulkloader can do.

Set to zero, I commit at the end of the transaction.

I have a problem with some jobs that have been migrated from DS 5.2. Some of the jobs run faster than on the old version, but some are slowed fown by a factor of 5-10. One job reads 13.000.000 records from a flat file, performs a hashed-lookup (preloaded to memory) and insert into a table in a oracl...

Found it. Thanks for the quick answers.

Michael

I've just imported a large DS project onto a new server. The project contains about a hundred-something jobs. From what I understand there is a feature called Compile all (I found a comparisn of this and ordinary compile elsewhere on this forum). Do anyone know where I can find this compile all func...

I can't get the ps -ef to give me the right result. This is a solaris server, ps works sligtly different from on Linux at least. In addition I'm using DS 5.2 on this machine. The Jobs are still running. Have now been running for 10 hrs. The only output I get from ps that I think is related to Datast...

Yes. The file should be quite a lot larger.

The file that currently resides in the destination folder, created by the job in question, is about 93% smaller than the file generated last month.

I've checked that there is enough disk space on the server to actually dump the data onto the disk.