DSXchange

We are about to upgrade from 8.7 on grid plus lsf to IIS 11.5.

We are really confused if we should go with IIS 11.5 on grid or IIS on Hadoop not BigIntergrate or BigQuality.

Any thoughts how do regular datastage jobs would behave on Hadoop cluster.

How would all input files/scripts would really work on a hadoop cluster.

any thoughts from Gurus is much appreciated.

Thanks

One thing to remember is that you wont get any gain of performance on anything that involves small files. Hadoop file system (HDFS) is not working well with small files. A small file can be defined as any file that is much smaller than the Hadoop block size. The Hadoop block size is usually set to 64,128, or 256 MB.

Thanks.