XML Stage in 8.5 FP1

[gsingh]

Hi All,

Can any one please explain me how to use the XML Stage. I have a requirement where in I need to read xml files, and do some transformations and load it into Database.

I used schema library manager, but i'm not knowing how to exactly import the metadata.
Also, i need to put a reject link at XML Stage, so that the input records that doesn't match the schema should get rejected.


Thanks,
gsingh

[eostic]

Ok..there are multiple approaches, 100's of entries in this forum, and some very good documentation....(places you should also start reading and looking into)...and also, various choices you can make in 8.5.1....

Some additional starting questions for you that will help....

How complex are the xml documents? (how many nodes...how nested, etc.)
Do you have an xml schema definition?
Where are they located? (the xml docs....in message queues, in files on disk, in an rdbms, etc.)?
How large are they? (how large is the biggest "single" individual document)?
How many elements/attributes (that will become columns in a database target)?
Are you reading only?...or also writing in this process?

Ernie

[gsingh]

Ok..there are multiple approaches, 100's of entries in this forum, and some very good documentation....(places you should also start reading and looking into)...and also, various choices you can make ...

Thanks for you quick response.
I have used oxygen genrator for getting the schema files.

Now I have a question, can you please tell me if XML stage which is being used as input stage support reject links. If it supports reject link can you tell me how to configure the settings for using the reject link?

Thanks,
gsingh

[eostic]

....not until you answer the questions above..... ; ) there are many approaches....all of which support reject possibilities.

Ernie

[businessintelli]

Hi ernie,

The files are located on disk.
Yes we have the schema definition which i generated using Oxygen generator.
Each file is of 1GB
Each file has upto 50 columns
It is writing into a Database
Here we are reading from xml, we are not writing to xml.

Please let me know if we can put the reject link.

Thanks,
gsingh

[eostic]

Ok...there's LOTS to learn, but here are the "basics":

a) You can use the new XML Stage to read xml documents from disk.
b) You need to import your xsd into the "library" using the Library manager or the Library tab within the Assembly editor
c) At a minimum, you will need and xmlParser Step....in there, you will see options for validation. You will choose "Strict" and then adjust as needed for various exception types and then select "Reject" as a process option.
d) Once you do that, you will have a success code and a message that you can act upon.
e) Use a "switch" step to review the success code, which will ultimately result in "good" and "bad" xml.
f) process the good xml further and send its columns to one output link via the output Step and send the bad xml and success code to another output link.

That's VERY high level. You will need to spend time inside the Assembly editor, getting comfortable with the Steps and how you include them in the process.

Ernie

[FranklinE]

Don't be afraid to experiment. I had similar beginner's questions at one point, and Ernie's seemingly vague (high-level!) responses were actually very helpful.

I'm working in 7x, so I don't know if this is still true, but I was able to import the schema using sample xml data files in the expected formats. A separate xmd may not be absolutely necessary... but I also didn't learn anything about them, so I have more to learn, too.

Good luck, and don't expect me to comment on details. I'm barely a baby step ahead of you.

[eostic]

....one more thing....if the files are really that big (1G) then you will absolutely need the new xmlStage with the steps above.

However....if you have any xml documents for this application or another that are much smaller [<about 200M] (or for readers of this thread that are not at 8.5), the existing xmlInput Stage will work just fine.

Ernie