DataStage Server Engine - UniVerse

[ag_ram]

All,

I need some clarifications on DatStage Engine(UniVerse System) for a better understanding of DataStage Internals.

1. Is there any other Multivalue RDBMS shipped out other than UniVerse and UniData from IBM?

2. Is there any standards in SQLs to be used only for Multilvalue RDBMS such as UniVerse SQL. Whether UniVerse SQL has been standardised to be applicable to all Multivalue RDBMS.

3. Are UniVerse processors ProVerb, ReVise and RetriVe visible to the DataStage Users? If yes, where can we see such those procesors?

4. It has been a known fact that the stand-alone product UniVerse RDBMS is designed to replace(control) most of the Operating System(OS) tasks. Wondering that how far this UniVerse can supplant the OS taks.

5. What is the difference between UniVerse SQL and Pick SQL? Is there only name change or any functional difference?

6. What is the difference between UniVerse file and SQL Tables? In what way they are getting differed? What are such tables in the DataStage internal files?

7. What are the differences between UniVerse files and DataStage files in the DataStage internal files?

[ray.wurlod]

1. Yes. See this document for the history and current players. They include D3, mvBase, mvEnterprise, UniVision, Reality, jBase, ONware, Open Insight, and JOI.

2. No published standards but, since UniVerse and UniData claim to have more than 70% of the market, then that's pretty much a de facto standard. There are surprisingly few extensions to "regular" SQL.

3. They're not even visible to UniVerse users. They just use them. So can DataStage users.

4. It is not a well-known fact. It's a completely false assertion. Even the origin of UniVerse was to be something that ran on UNIX. (Windows version came later.) Perhaps you would care to elaborate on where this alleged "fact" is so well-known.

5. Please advise what "Pick SQL" is. It is not a term of which I am aware.

6. Every SQL table is a hashed file that has, in addition, an internal storage area for security and integrity constraints and entries in "system tables" that record the existence of users, schemas, tables, views, columns and associations. "Ordinary" hashed files lack these extras. DataStage uses an appropriate mix of ordinary hashed files and SQL tables.

7. That question is totally silly. UniVerse has not been part of DataStage since 2000, which I have indicated at least three times previously.

[ag_ram]

Thanks Ray for your answers and pointer,

I wanted to ask this,

5. What is the difference between UniVerse BASIC and Pick BASIC? Is there only name change or any functional difference?

Some mistake happened. Please forgive.

[ray.wurlod]

UniVerse BASIC is a superset of Pick BASIC. Indeed, UniVerse BASIC is not one language, but six, and one of its variants (not the one used originally in DataStage) is very close to Pick BASIC. None of that affects DataStage; it's more a matter for those who develop applications under UniVerse. And there are very many of those.

[ag_ram]

Thanks you Ray. A few more questions for a better understanding of Server Engine.

1. In Engine directory, a folder named ADMIN_BP seems to have more UniVerse BASIC programs which predominantly used for Transaction Logging purpose. If this is certain, are we able see the Transaction logs created while UniVerse Client(more internal) making transaction with UniVerse Database server (You may say, DataStage Repository at DataStage v7.5 and above)? if possible, How we need to be prepared?

2. And more specific, a couple more of BASIC programs named - CONV.DICTS, CONVERT.PROC, CONVERT.VOC which stand to mean that they are purposely designed to have a conversion of Dictionary, VOC, Format from PICK System to Universe system. Why such an unheard term being played out here? What is the necessity of having a conversion from PICK to UniVerse?

3. What kind of Application Programs do we have in the UniVerse Engine directory APP.PROGS? How they are really a matter of running the whole DataStage Server Engine via perhaps, UniVerse Clients.

4. What some of executable files mean here in DataStage Engine, to name - basic.exe(known that this is not a BASIC compiler either), prime.exe, smat.exe, spool.exe ...

5. What type of BASIC programs present under DataStage Server Engine foder BP? Even here we can see that term PI/OPEN in most of the Programs.

6. What does MICRODATA mean out here? Which is by assumption more concerned about terms Prime INFORMATION and How?

[daignault]

The UniVerse environment was first designed as a Database environment. It was determined that the database engine would lend itself to an ETL type environment.

The code between the Datastage environment and the UniVerse environment was split, but there is still legacy items contained with the Datastage engine. Items such as APP.PROGS (Application Programs), and the CONVERT programs are all from that legacy.

These items can be ignored.

Regards,

Ray Daignault

[ray.wurlod]

Except that UniVerse was first designed as an Application Development environment (more than just a database environment), I believe Ray's answer addresses all the points in your most recent post. If they don't, all the answers are to be found in the UniVerse documentation. Just be aware that few, if any, of these things are used by DataStage, because DataStage has been set up to manage its own administration (for better or worse). There is NO transaction logging in DataStage Repository.

Hey, why not take a sequence of UniVerse classes from IBM?

[ag_ram]

Thanks Ray Daignault , ray.wurlod for your clarification and advice on UniVerse DBMS Training and perhaps i will make some plans on that.

ray.wurlod wrote:

...that UniVerse was first designed as an Application Development environment (more than just a database environment).

This makes clear in the part of deriving the answer for question like - In DataStage v8.0, How DataStage has UniVerse as its Server engine but not including it to be the DataStage repository.

By a search in Universe Documentations, i just came to deduce some more points here.

ADMIN_BP -->
1. Account creation, deletion, modification, Importation.
2. Routines for Catalogging, Compiling programs.
3. Routines for Conversion of PI/Open Applications to UniVerse Enviroment.
4. Transaction logging and recovery related routines.

As IBM stopped marketting PI/Open product since Oct 2004 and there was a need to convert all PI/Open based Applications to Universe, UniVerse had to adapt the necessity of having some BASIC programs to make the conversion happen.

And I am almost done with the search but still unable to find answer for this question.

4. What some of executable files mean here in DataStage Engine, to name - basic.exe(known that this is not a BASIC compiler either), prime.exe, smat.exe, spool.exe ...

Please help me answer this.

[ag_ram]

Thanks Ray Daignault , ray.wurlod for your clarification and advice on UniVerse DBMS Training and perhaps i will make some plans on that.

ray.wurlod wrote:

...that UniVerse was first designed as an Application Development environment (more than just a database environment).

This makes clear in the part of deriving the answer for question like - In DataStage v8.0, How DataStage has UniVerse as its Server engine but not including it to be the DataStage repository.

By a search in Universe Documentations, i just came to deduce some more points here.

ADMIN_BP -->
1. Account creation, deletion, modification, Importation.
2. Routines for Catalogging, Compiling programs.
3. Routines for Conversion of PI/Open Applications to UniVerse Enviroment.
4. Transaction logging and recovery related routines.

As IBM stopped marketting PI/Open product since Oct 2004 and there was a need to convert all PI/Open based Applications to Universe, UniVerse had to adapt the necessity of having some BASIC programs to make the conversion happen.

And I am almost done with the search but still unable to find answer for this question.

4. What some of executable files mean here in DataStage Engine, to name - basic.exe(known that this is not a BASIC compiler either), prime.exe, smat.exe, spool.exe ...

Please help me answer this.

[ray.wurlod]

PI/Open still exists. Prime INFORMATION still exists. They were among the databases that Informix sold to IBM in 2001.

No-one is going to take the time to explain what everything in $DSHOME/bin does. You use very few of them directly - they support what the engine does. Occasionally you will find on DSXchange that some of them are useful from the command line, for example list_readu executable.

[rameshrr3]

Most executable files in $DSHOME/bin implement an equivalent UniVerse Comand at the OS Level, but I doubt if its all documented.

for example the smat OS command implements someting nearly equivalent to the SEMAPHORE.STATUS UV command.

HTH
Ramesh

[ag_ram]

Ray,
Could you please answer following questions as well in order to derive a clear statement.

1.

As IBM stopped marketting PI/Open product since Oct 2004 and there was a need to convert all PI/Open based Applications to Universe, UniVerse had to adapt the necessity of having some BASIC programs to make the conversion happen.

a. Whether the Information i made was wrong or the sequence of sentences with that reasoning but not information i made was wrong.

b. Till my knowledge, as long as the Information provided is evident(IBM stopped marketting PI/Open product since Oct 2004), Who markets the PI/Open product?

2. Most important, What kind of features UniVerse tended to add from/against PRIME INFORMATION , PI/Open Database systems?

3. Very needed one, In most of the posts Ray mentioned about basic.exe as not a BASIC Compiler but what he missed to last that sequence(maybe I rarely missed it out) of saying what it is really. Can you now just elaborate the reply more than pointing it out as not a BASIC Compiler?

Code: Select all

C:\Ascential\DataStage\Engine\bin>basic
usage: BASIC FILE ITEM...

This is the response we got while bling-executing it.

Please help me on these questions.

[ArndW]

I'm not going to go into the other questions but will address (3).

A Compiler is generally considered to be a program which translates text written in a human-type language into a series of instructions (object code) that can then be loaded and then executed directly by a machine. In the case of UV/BASIC the resultant code is actually pseudocode which is not really object code meant to be executed but is read by the DataStage engine and interpreted at runtime.

For this reason the more pedantic among us maintain that DS jobs are not truly compiled.

[daignault]

OK, First and formost....Why do you need this information? What are you trying to prove/learn about the Datastage product. Why is a 15 year old history of the product needed?

Now, per your question on basic.exe, when is a compiler not a compiler?

When the code is compiled to ByteCode or pcode (depending on the age of the reader).

When a program is "compiled" in UniVerse BASIC it is converted down to Pcode (for Pascal) or ByteCode as it is known in the Java language. The code is broken down and then interpreted by the run machine. You can see the breakdown of instructions via the VLIST command from the TCL prompt.

This is in contrast with the old Dartmouth BASIC (or DOS GWBasic) which did not analyse the code but interpreted the English code.

Regards,

Ray Daignault [/b]

[ag_ram]

ArndW,

In the case of UV/BASIC the resultant code is actually pseudocode which is not really object code meant to be executed but is read by the DataStage engine and interpreted at runtime.

For this reason the more pedantic among us maintain that DS jobs are not truly compiled.

Thanks ArndW.

A small question: Where the Pseudo code would be available (RT_CONFIGnnn/RT_BPnnn.O/RT_BPnnn)?

daignault,

OK, First and formost....Why do you need this information? What are you trying to prove/learn about the Datastage product. Why is a 15 year old history of the product needed?

Because of curiosity. Let us never ask Why we need the old things. ray.wurlod has got more than 10 years experiance on what i asked. So let us wait for his answer on the same.

Now, per your question on basic.exe, when is a compiler not a compiler?

You can see in lot of posts where the clear-cut was made on basic.exe that it is not a BASIC Compiler.

Rest of your information seems to be satisfying me. Thanks daignault.