DSXchange

Hi, I was wondering if anyone wrote the parser needed for the Operational Metadata generated XML files to feed splunk.

Also, since IBM bought Platform LSF and it is a recommended Grid Resource Manager for DataStage (should you chose to go grid), has anyone written a splunk parser for the LSF logs? The splunk site has a mention of this, but it's basically another user asking if anyone has done it yet.


Thanks in advance.

Bump.

Still looking to see if anyone out there has paired Splunk + DataStage.

Thanks

Paul, were you able to integrate Splunk with DataStage ?

Paul, skatha,
At our Datastage site, we are/have extracting required information from DS logs - however, we are not using splunk/ELK or other log parsers. We are using perl/awk and similar tools to do this.
Is there something specific that you are looking for ?

Thx

How about just using DS with xmlInput Stage to parse them? They are very simple xmls...

Ernie

it's not the datastage side that was of any issue. It is the coupling of both products that I needed assistance on.

And no, I have not had the time to get splunk feeding off of the LSF logs.
I do not want to make a datastage log parser since the act of extracting a log is something I want to avoid.
Creating a datastage job to parse the XML files when you have 20K jobs running per day is also something I do not want to do since that means a ton more datastage jobs, and as a result could create an invinite loop if not coded properly.

So basically nobody out there is using splunk to mine any datastage metadata it would seem.

I have splunk set up for the NMON data dashboards. But something more akin to DataStage jobs would have been nice.

DSODB can be a source of the data, but the dashboards it presents are not 100% what I am looking for.

Would be nice to get a GANNT chart view of a daily project run. Sequencers + jobs with visual dependencies.

I might have to try and code something with highcharts and a table dump from DSODB. That stuff is just not my strong suite, I've managed to get by on my good looks alone so far.