DSXchange

Has anyone had poor performance issues after enabling direct i/o? The system I'm currently working on appeared to be using large volumes of memory for file caching at a system level. I had the volume we're using for storing the data files, datasets and lookup filesets switched to direct i/o and this...

Arnd - I'm busy running one of your jobs and it stores everything in the project area!

Personally, I like to use the project area because then you have all the Universe commands available to you. I know you can set a pointer in the VOC to get around this but if you want to check whether a row exists its so much easier to just go into Administrator and type in a bit of SQL. BUT... a lo...

As Josh said, its probably padding that's causing the problem. If you haven't got $APT_STRING_PADCHAR set then by default Datastage will pad chars with a \0 null character, in a similar way to C. Try adding $APT_STRING_PADCHAR=0x20 to the parameters. It'll tell Datastage to use spaces to pad chars o...

Thanks for that bit of Perl John, can't really send you the live data as its financial information but I'll definitely give it a go :D For gateleys and anyone else who's interested, here's a chopped down example of the Basic version... Enjoy! JobName="ConvertCSV" ! Open infile and outfile ...

Cheers for the ideas. Not being a Perl expert, I tried using Unix scripting to convert the file. It worked fine but just took too long to run. In the end I knocked up a basic routine which does about 100,000 rows a minute. Good enough for me!

Thanks again,
Dave

Can anyone think of a neat solution to this problem? I've got a csv file with approximately 200 columns, which could potentially change position. The first row defines the column names and I would like to pivot the data so that I end up with one record per column along these lines: Input file: Key C...