DSXchange

Hello all,

I have a situation where in I am reading from source (Mysql) with an ODBC Stage. Also populate a hashed file for lookup in the same job using a different ODBC stage (Same Source)
As expected the main ODBC Stage waits for the Hash file to be populated before it starts processing the rows.

The issue is that I hitting a transactional system and the data keeps changing. For the time it takes to populate the hash file, the data would have changed in the source, so my source SQL pulled in and the data I populated in the hashed file are not in sync.

My question -
- Is there a way to trigger the queries in a job at the same time ?
- If I run all my queries(ODBC Stages) first from the source to the flat files in the same job, will all of the queries be executed at the same time?
- Is there an option to read the same information from a changing table (source). if you have multiple stages hitting it (But pulling it based in different criteria) in the same job.

Long story short: I need a way to read data from a changing table in the source using different queries/joins (ODBC Stages) in the same job and not have to worry about any real time changes. Something like to get the data from a snapshot in time.

Note: Source does not have a modification timestamp on that table.

One option - try using a single source stage with multiple output links. Another - one query to stage the entire source table locally, then you can take your time pulling out what you need.

Thanks for taking the time and responding Craig.

If we use same ODBC Stage with multiple Links does it considered as one session or does it spawn multiple sessions ?
I Like your second solution is a good one too.

One stage = one connection / session that all of the links would leverage. I've done that in the past for multiple simultaneous sources but never checked to see exactly how 'simultaneous' they really truly were.