Web Version for DataStage

kduke · Post by **kduke** » Sat Aug 16, 2003 12:49 pm

Chandra

I have a web version of just the stuff in the Director for sell on my website. You can download a copy and email me for a free 30 day trial auth code. It is called DsWebMon. I also have DwNav which is available on www.DataStageXchange.com (DSX) and www.Duke-Consulting.com. It is free for the first 30 days too.

DsWebMon is a very thin client. I do not know how well it will perform in your situation but I expect it to be fast. I think it would be fast even on a modem. I designed it to be available for DataStage developers so they could view log files and start and stop jobs from your hotel room or from home. It does not require VPN or other special setup. I demo it on my laptop. All my tools are desined to make my life easier because I do not want to drive in a 2:00 in the morning just to reset some job because they typed in the wrong parameter value or some other simple problem. I also do not want to fly back early for a similar reason. I am very surprised others are not interested in time saving tools like these. Life could be a whole easier if you had a little help with a good tool. Might get a little more sleep too. Huh TJ?

I think the future of all our jobs is going to be done remotely. DwNav is a tool to help me analyze your jobs remotely. A developer can extract the data and browse the ETL without the Designer. Soon I will be able to report differences between 2 versions of a job without digging through ever screen to see if only the before and after routines changed or one column's metadata changed. If you are new to project then how long does it take you to figure out what table is updated by what job so you can add one column to it. I am very lazy. I hate retyping parameters in every job. Why not copy them. Surely we are smarter than this.

This website was designed to sell tools like these. If others have tools that make our lives better then please write to admin on either this site or DSX.

Kim.

Kim Duke
DwNav - ETL Navigator
www.Duke-Consulting.com

ray.wurlod · Post by **ray.wurlod** » Sat Aug 16, 2003 5:24 pm

Despite the fact that you're planning to lodge your work on Source Forge, I doubt that you'll get much help without some hope of financial reward for the helper(s).

You should also check out Ascential's web site, to learn about Web Services (in release 7, due any day now). I have not been able to review Kim's software yet (busy!) but the demo on the DSX (or Kim's own site) seems cool.

However, if you're seeking to redesign the DataStage clients to run in a browser, you're up for some heavy duty technical knowledge acquisition (how to manipulate objects in the DataStage Repository) and this knowledge simply does not come for free. In fact it's arguable that that knowledge, not being in the public domain, is proprietary intellectual capital owned by Ascential; I wish you luck resolving your desire to publish open source methodology for accessing the DataStage Repository with the legal eagles at Ascential (who, by the way, are planning to re-vamp the Repository and the Engine over the next one or two major releases).

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

vmcburney · Post by **vmcburney** » Sun Aug 17, 2003 6:20 pm

Surely the slow response times are caused by your network? On a fast pentium across a good network I don't see any response time problems. Maybe you should be talking to your network support to find out what quality of access you have to your development server.

If this was a client/server product with a large number of production users you could justify the cost of moving to something like .Net, or putting data into locally cached objects, however ETL is usually a small number of development users with high end machines and good network access, so none of the ETL vendors have seen the need to create a browser based version.

If your company has a poor network and need to use traditional client/server products they should consider a Metaframe server.

Vincent McBurney
Data Integration Services
www.intramatix.com

kduke · Post by **kduke** » Sun Aug 17, 2003 7:36 pm

Vincent

There were specific patch levels which were slow on specific servers. Sun had a problem even on fast networks and fast clients. Ray or Ken posted on such instance. If you have a lot of jobs then you can see significant slow downs especially around 1500. It seems to be faster with release 6 but still it is a thick client. Even the Director is thick. To solve that all we did was display the data on the client. No way I want to run Director over a modem.

Kim.

Kim Duke
DwNav - ETL Navigator
www.Duke-Consulting.com

kcbland · Post by **kcbland** » Sun Aug 17, 2003 7:48 pm

The clients are definetely needing optimization. When they added all of the underlying hooks for impact analysis, performance went out the window. Ever try compiling a brand new function in a project with 600+ jobs, and wait FOREVER while it makes sure it's not used anywhere? Agony.

Kenneth Bland

vmcburney · Post by **vmcburney** » Mon Aug 18, 2003 8:14 am

Interesting, I haven't tried using DataStage in a project that big. Even a web front end wouldn't do much for performance if the usage analysis is occuring on the server. I do think a web based interface would be good for the Director and the Manager but it would be a difficult job to come up with a Designer that was as easy to use as the Windows version.

Vincent McBurney
Data Integration Services
www.intramatix.com

kduke · Post by **kduke** » Mon Aug 18, 2003 9:39 am

Chandra

Well then buy mine. It is a lot cheaper than building your own. It is very fast.

Kim.

Kim Duke
DwNav - ETL Navigator
www.Duke-Consulting.com

Teej · Post by **Teej** » Mon Aug 18, 2003 1:28 pm

quote:Originally posted by kduke
[br]Life could be a whole easier if you had a little help with a good tool. Might get a little more sleep too. Huh TJ?

Oh, I agree. Umm. Hmm. I seems to be missing something here...

-T.J.

* * *

... now if this can make breakfast, my life is complete.

chulett · Post by **chulett** » Wed Aug 20, 2003 2:35 pm

Definitely! If you are not making use of the Categories, you really should be. Then, instead of your entire job inventory being refreshed every time, only the contents of the current Category are pulled in at any one time.

-craig

ray.wurlod · Post by **ray.wurlod** » Wed Aug 20, 2003 5:11 pm

Don't forget, in the Director client, to ensure that the Show Categories option is checked under the View menu. Otherwise you get every job in the project, sorted by job name (a large sort in a project with many jobs will help to slow things nicely!).

Ray Wurlod
Education and Consulting Services
ABN 57 092 448 518

kduke · Post by **kduke** » Thu Aug 21, 2003 9:47 am

Chandra

Are you sure it is corrupted? Try compiling it. They do something funny when they rename a job. I think they do a save as new name and then delete the old job. I do not think that should not corrupt the job unless you do not have write permission to the directory.

Kim.

Kim Duke
DwNav - ETL Navigator
www.Duke-Consulting.com