problem importing XML Table defs...

[chuckd248]

Has anyone experienced problems importing XML table definitions in to DS 8.x???... is anyone aware of a known issue???...

I'm trying to import an XML table definition and while the source is correct, it does not create the table definition... does the source file have to be .XML or should I be using the schema file (.xsd)???...

I know nothing of XML and this is my first attempt at importing this kind of table def so if the questions sound stupid, that is why...

Any help would be appreciated...

[eostic]

It's not a stupid question, but you do have a lot of reading and playing to do........here and in the standard documentation.

As for the XML Metadata Importer, you can use either an XML doc or an xsd....assuming the window comes up properly, you can "open" your document or xsd......

You should then see a "tree" of your available elements, attributes, nodes, etc. Open the tree and review what you see. Ultimately you will want to "check" only the lowest level attributes or elements (labelled with a "T") for selection...you will see a normal DS tabledef "growing" in the lower window.

Start simple. Just pick one node of elements (where the elements are all at the same level --- things like name, address, phone, etc. are often at one level) and select those for your tabledef.

Check one of them as a "key".

Then keep reading, here and in the doc......also, find the "xml best practices" guide that is menioned in many places on this forum. Importing XML is only one part of the equation....

a) you need to decide how you are going to be reading it (from disk, from a queue, as a filename, from an rdbms, etc.)
b) you need to know about the xmlInput Stage and how to pass the string or filename into that Stage
c) you need to learn about everything from null handling to validation, to the behavior of nested and/or parallel nodes in the original document....

....and that's just for reading xml.

XML can be fairly simple, or extremely advanced and complex.

Ernie

[chuckd248]

We are getting the XML from an .XSD file via MQ... the .XSD file is as follows:
-------------------------
<?xml version="1.0" encoding="UTF-8"?>
<!-- $Workfile: ClassStaff.xsd $ $Revision: 2 $ $Date: 2/16/10 12:08p $ -->
<xs:schema xmlns="http://schools.nyc.gov/DOE.Services.StudentInfo" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:bons1="http://schools.nyc.gov/DOE.Services.Location" xmlns:ns1="http://schools.nyc.gov/DOE.Services.Common" targetNamespace="http://schools.nyc.gov/DOE.Services.StudentInfo" elementFormDefault="qualified" attributeFormDefault="unqualified">
<xs:import namespace="http://schools.nyc.gov/DOE.Services.Location" schemaLocation="../Location/LocationCodeList.xsd"/>
<xs:import namespace="http://schools.nyc.gov/DOE.Services.Common" schemaLocation="../DOECommonTypes/Determinator.xsd"/>
<xs:import namespace="http://schools.nyc.gov/DOE.Services.Common" schemaLocation="../DOECommonTypes/Name.xsd"/>
<xs:import namespace="http://schools.nyc.gov/DOE.Services.Common" schemaLocation="../DOECommonTypes/Language.xsd"/>
<xs:include schemaLocation="SourceSystemClassReference.xsd"/>
<xs:complexType name="ClassStaff">
<xs:sequence>
<xs:element name="odsClassID" type="xs:token" minOccurs="0"/>
<xs:element name="sourceSystemClassID" type="SourceSystemClassReference" minOccurs="0"/>
<xs:element name="adminSchoolLocationCodes" type="bons1:LocationCodeList" minOccurs="0"/>
<xs:element name="staffRole" minOccurs="0">
<xs:annotation>
<xs:documentation>Teacher or Advisor</xs:documentation>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="xs:token">
<xs:enumeration value="Teacher"/>
<xs:enumeration value="Advisor"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="primary" minOccurs="0">
<xs:annotation>
<xs:documentation>Yes/No for whether this staff person is the one primarily responsible for this class or course section</xs:documentation>
</xs:annotation>
<xs:simpleType>
<xs:restriction base="xs:token">
<xs:enumeration value="Yes"/>
<xs:enumeration value="No"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="personnelID" type="xs:token" minOccurs="0"/>
<xs:element name="employeeFileNumber" type="xs:token" minOccurs="0"/>
<xs:element name="staffType" type="ns1:Determinator" minOccurs="0"/>
<xs:element name="name" type="ns1:Name" minOccurs="0"/>
<xs:element name="paraProfessionalType" type="ns1:Determinator" minOccurs="0"/>
<xs:element name="language" minOccurs="0">
<xs:complexType>
<xs:complexContent>
<xs:extension base="ns1:Language">
<xs:sequence>
<xs:element name="languageName" type="xs:normalizedString" minOccurs="0"/>
<xs:element name="languageDescription" type="xs:normalizedString" minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:schema>
------------------

As you can see, the file contains several "xs:import" entries as well as an "xs:include" statement...

When I try to import the table definition it SEEMS to be populating the tree structure with data elements from those from the multiple import statements but does not include any of the data elements in the file after the include statement...

Thoughts/comments???...

[chulett]

Perhaps there's a way to do this in 8.x but earlier versions did not support those "includes" and I had to combine the xsd files manually to get one it liked with everything in it.

[eostic]

The current XML importer does not support "import" for xsd's. The best solution is to ask for a complete "instance" document, or create one (there are lots of tools that will do that). Then import from the xml instance instead of the xsd.

The only significant caveats to importing with the xml document are:

a) be sure you have a "complete" document (where every element and attribute is populated in the first iteration, or else you might miss important metadata.

b) you columns will have varchar datatypes [no biggie since that is what they really are anyway]...... you'll be converting them as necessary downstream if you put them to an appropriate target.

Ernie

[chulett]

I didn't realize there were tools out there that would do this, here I thought I was the tool. So to speak.

Any that you would recommend, Ernie?

[eostic]

I used to use XMLSpy, but my license ran out. It did it nicely as I recall. This is one of those things I only need once in awhile, so I'll try to find whatever I can. Often at customer sites, the author of the xsd will have a tool and can immediately generate an xml doc for me.

A quick search on the web for "create xml from xsd" generated a lot of hits, one of which I haven't played with but was installed on my notebook, simply because I have Microsoft .NET Studio 2003 so that I can compile my EE jobs in a Windows DataStage configuration. Its called xsd.exe. Haven't tried it, but it looks pretty simple to use. Creating xml documents is not a function for every day use, so even if the syntax is clunky, it would be fine.

I did a little bit of reading about Liquid XML, and thought it looked nice from their web site, but I haven't downloaded their trial yet.

I like tool called XMLMax, but from what I can tell, it lets me create xsd from xml but not the other way around. It's a great tool for reading really LARGE xml docs.

Ernie

[eostic]

ah. boo. xsd.exe is nice, but it does the opposite only, from what I can tell. Just took a closer look and it nicely creates xsd from xml, but not xml from xsd.

Ernie

[chulett]

Thanks. Put me in the "used to use" camp for XMLSpy as well, a license for that is pretty darn expensive. I've also used the freeware 'Community' edition of Liquid XML and it's pretty nice for what is basically 'cripple-ware'.

[asorrell]

The current XML importer does not support "import" for xsd's.

Ernie - is this because of a bug in the importer? The documentation seems to state fairly clearly that it is a supported feature...

From Page 6 of the XML Pack Guide on Using the XML Metadata Importer:

Opening an XML source

You can process an XML schema file (.xsd) or an XML document (.xml). The file can be located on your file system or accessed with a URL.

[chulett]

That's not what the bit you quoted is referencing, Andy. That "URL" comment just means you don't need a landed file for the metadata, you can get it directly from the source URL. It just plain old doesn't support it.

[eostic]

Right...I'm referring to the formal "import" syntax within an xsd. It is a formal "command" that is available for xsd's that "points" to additional xsd's. (like an "include" statement in many other languages).

That's what the current XML importer does not support.

Ernie

[asorrell]

Ok - thanks for the clarification!