Web-Service Tracking/Logging?

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
w951bvc
Participant
Posts: 12
Joined: Mon Aug 16, 2004 10:53 am
Location: Milwaukee, WI

Web-Service Tracking/Logging?

Post by w951bvc »

Hello;

I searched quite a bit this morning, but have found nothing really directly about this topic - so I'll go ahead and ask. Apologies if I'm repeating something.

We're using web-services; is there any way to keep track of exactly when a service is called? We're primarily interested in somehow logging the traffic so we know precisely when we were called and how heavily - ideally we could also capture the input fields.

I'm not looking for precise solutions, just general suggestions (if there are any). We ran into a wall using files/data-sets due to our services all running multiple instances. At this point the only workable option we've found is to have the service write to a database; obviously not the best solution since performance takes a hit.

Also pondering having a script perhaps continually scrape the job log, but if that even would work it seems a pretty inelegant way to get that information.

Any suggestions?

Thanks for your time,

Richard
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Take a look at tools like Actional and others like it. They used to have a great tool for front-ending web services, monitoring their traffic, etc. etc.

Also great in this area are the various xml "appliances". Data Power is one that can do some very serious things with web services, including payload adjustment, application of ws-* security paradigms, and much more. I'd be surprised if they can't also do monitoring and auditing of your web service invocations.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
w951bvc
Participant
Posts: 12
Joined: Mon Aug 16, 2004 10:53 am
Location: Milwaukee, WI

Post by w951bvc »

Thanks for the information. We have another group that's looking into software outside of DataStage that can do this monitoring for us, and I'll pass this along. I'm looking for some way to accomplish this inside the DataStage (or InfoSphere) application itself, but it's looking more and more like there's no good way to do so.

Richard
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

At one point years ago we had a user exit for coding your own monitoring things in java based on events within RTI (the former ISD), but it didn't get enough usage, and there are so many other things that tools specializing on this space can address.....from security, to payload alterations, to counts, monitoring, auditing, protocol changes and more...... and those things are transparent to the service.

It would be cool to have those things, but best that each tool sticks to its strengths.....leave the service creation, function and performance to ISD and DataStage...and have appropriate soa "framework" interfaces handle that other type of plumbing.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply