DSXchange

OK, I found a fix for the immediate problem. Instead of a sed command that looks for the end of the file and adds a newline, I've got it finding the closing tag of my xml and replacing it with that tag plus a linebreak. I've now got it taking multiple files, transforming them and writing the text ou...

Hi, I've got a directory full of gz compressed xml files that I want to stream through an xml transformer to parse them to delimited text. The xml transformer needs to know where each file ends in order to accept them as valid xml. But the xml within the gz files, which originate in an external Web ...

That is the issue I was referring to.

Ray,

I see. So is it the case that values read from file can not be changed during the same sequence?

I may have found my solution. As I understand it, parameter values are set when a job is compiled and can't be changed while the job is running. However, I thought I had read that using a parameter value file was a way around that limitation. I created a new sequence to run the get_total_page_count ...

I'm storing a total page count from a Web service in a parameter value file, then attempting to access that value in a sequence loop in order to loop through that total number of pages. However, as long as I'm writing the parameter value file and calling it from the loop in the same sequence, the pa...

I've got my job working (mostly), so I'm going to mark this resolved. Ray, I was typing it in manually because it doesn't appear that user variables show up in the menu. I had not tried #s. However, Ernie, I went ahead and added in another stage to iterate the count in a job parameter. So inside my ...

At the moment, I'm putting the activity variable, "start_page_loop.$Counter" (but no quotes) into the input argument in the Web Services Client stage in my job. Result: It appears to be using it literally and I'm getting an xml error for "start_page_loop.$Counter" being an invali...

Yep, I've got the parameter in my start loop, but now I'm trying to get the loop's counter from the sequence and into a job. I've found the User Variables Activity and have specified a "Count" variable there with the expression "start_page_loop.$Counter". Now how to get "Cou...

I'm back with a little progress using a sequence and loop stages. I've created a job that makes a Web service request for a row count and the corresponding total number of pages, then writes that total page value to a parameter set. Then in the sequence start loop stage I use that parameter as the l...

I'm working on employee data at the moment, and might get as high as 50-60,000 rows, although I hope we'll ultimately just be calling for updated records and not the whole set. Future phases may involve transactional data that could theoretically be more. I don't think I'm constrained to EE jobs, bu...

What dictates this "maximum number per page"? That sounds like it is something that the author has put into the service? Yes, it's specified in the Web service. There are parameters for Page (which page to get) and Count (number of rows per page) So if I make a request for page 1 with cou...

I'm relatively new to both DataStage and Web services and would like some general direction for an integration task I'm working on. Our team is working with a new HR management system and attempting to pull data from it via Web services into our existing Oracle database. I've successfully done this ...

Hello, I've got another one for y'all. We have a job that reads in a dataset, does change capture against the Oracle table, and then applies the changes back to the Oracle table. It includes one column which is a CLOB in Oracle and a LongNVarChar in DataStage. It works fine, except that every time i...

Pivot Enterprise + Transformer worked. Thanks again!