Hello all,
I have a situation where in I am reading from source (Mysql) with an ODBC Stage. Also populate a hashed file for lookup in the same job using a different ODBC stage (Same Source)
As expected the main ODBC Stage waits for the Hash file to be populated before it starts processing the rows.
The issue is that I hitting a transactional system and the data keeps changing. For the time it takes to populate the hash file, the data would have changed in the source, so my source SQL pulled in and the data I populated in the hashed file are not in sync.
My question -
- Is there a way to trigger the queries in a job at the same time ?
- If I run all my queries(ODBC Stages) first from the source to the flat files in the same job, will all of the queries be executed at the same time?
- Is there an option to read the same information from a changing table (source). if you have multiple stages hitting it (But pulling it based in different criteria) in the same job.
Long story short: I need a way to read data from a changing table in the source using different queries/joins (ODBC Stages) in the same job and not have to worry about any real time changes. Something like to get the data from a snapshot in time.
Note: Source does not have a modification timestamp on that table.
Read from a changing table
Moderators: chulett, rschirm, roy