Hi,
We are migrating from oracle 10g to teradata 13.1. The DS jobs targeted to oracle ,now should point to teradata.
About 500 jobs highly parameterized and use wrapper to trigger jobs in sequence.
What are the major concerns and challenges ?
Datastage 8.1 Server , OS- AIX 5.3.
migrating from oracle to teradata
Moderators: chulett, rschirm, roy
Don't go overboard on your requested sessions to TD. More is not always better. Keep them in sync with your parallelism. Our TD DBAs told us that going over 32 sessions is a waste. If it's 20GB or more of data, take 32 sessions, otherwise go less. Just this morning we had a job take a few aborts due to Terasync timeouts. It could not supply 16 sessions in a timely fashion. It eventually did. But the job in question could have ran just fine with 8.
Have an external mechanism to queue up a TD job in case your database goes down for maintenance. You don't want to submit a DataStage job if you know your target DBMS is down. We did ours in our Job Scheduler.
Have an external mechanism to queue up a TD job in case your database goes down for maintenance. You don't want to submit a DataStage job if you know your target DBMS is down. We did ours in our Job Scheduler.
I'm assuming it is way too late for any suggestions if your plans are already set. I've been through an Oracle-to-Teradata forklift project, heard the comments from both sides and seen the ultimate result - failure. They dumped Teradata and went back to Oracle. It probably should never have been attempted in the first place except a salesperson got the ear of someone in power and convinced them that all the alleged "problems" with their data warehouse could be solved simply by dumping Oracle and replacing it with Teradata. Just like that. Yah, right. We grunts knew what the real issues were but we were never consulted.
While things will certainly run, Teradata isn't a relational database in the same traditional sense that Oracle is. What you should be doing is engaging someone familiar with both, especially the conversion of one to the other, so that your Oracle structures can be properly migrated / converted to Teradata structures - ones that take advantage of that particular architecture.
If the next question is "what is needed?" I can't help there. I was one of the Oracle guys, not the one Teradata person they brought on. I wasn't amongst the ones struggling with the forklift. I was, however, amongst the people let go when they decided they didn't need us any longer. We knew things were not going well and kept in touch with those left behind.
Good luck with your project, hopefully things will work out better for you. It seems to me that a lot will depend on expectations - in our case, they expected... magic.
While things will certainly run, Teradata isn't a relational database in the same traditional sense that Oracle is. What you should be doing is engaging someone familiar with both, especially the conversion of one to the other, so that your Oracle structures can be properly migrated / converted to Teradata structures - ones that take advantage of that particular architecture.
If the next question is "what is needed?" I can't help there. I was one of the Oracle guys, not the one Teradata person they brought on. I wasn't amongst the ones struggling with the forklift. I was, however, amongst the people let go when they decided they didn't need us any longer. We knew things were not going well and kept in touch with those left behind.
Good luck with your project, hopefully things will work out better for you. It seems to me that a lot will depend on expectations - in our case, they expected... magic.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
can only agree with the 'dont just forklift' the database comments - to be honest it probably will work if you dont spend much time tuning - but will be wasting a lot of expensive clock cycles.
Have been through a db2 ---> teradata migation a few years back (~3000 jobs) and a few points bar the forklift no-no
a) Reconcilation - you need a robust and ideally automated reconcilation process, I am presuming you will do a parallel run of your old oracle env and new teradata one - this will throw up various issues that may not be down to the migration process but will still show up material differences in the results, in our case we found several sloppy bits of scheduling that meant the two systems had diff results as was processing certain data in different orders.
b) You can automate some of the conversion work, we did it via some internal java code that would take an xml export from datastage , parse it , and replace the clauses relating to db2 stages and replace them with the teradata equivalents - for your volume of jobs though it may be worth handcranking rather than developing an 'automated' solution
c) You also need to think about your contingency situation for the database (if you have a DR setup) - dont know much about oracle but with db2 provision of contingency is kind of out the db2 box, with teradata it can be more complex - ideally for a new implementation that is being developed for Teradata from scratch would go for a dual load solution - however as you are migrating this may not be achievable as involves a lot of re-work of the ETL
d) Reconcillation - I know have already mentioned this - but is crucial
Have been through a db2 ---> teradata migation a few years back (~3000 jobs) and a few points bar the forklift no-no
a) Reconcilation - you need a robust and ideally automated reconcilation process, I am presuming you will do a parallel run of your old oracle env and new teradata one - this will throw up various issues that may not be down to the migration process but will still show up material differences in the results, in our case we found several sloppy bits of scheduling that meant the two systems had diff results as was processing certain data in different orders.
b) You can automate some of the conversion work, we did it via some internal java code that would take an xml export from datastage , parse it , and replace the clauses relating to db2 stages and replace them with the teradata equivalents - for your volume of jobs though it may be worth handcranking rather than developing an 'automated' solution
c) You also need to think about your contingency situation for the database (if you have a DR setup) - dont know much about oracle but with db2 provision of contingency is kind of out the db2 box, with teradata it can be more complex - ideally for a new implementation that is being developed for Teradata from scratch would go for a dual load solution - however as you are migrating this may not be achievable as involves a lot of re-work of the ETL
d) Reconcillation - I know have already mentioned this - but is crucial
-
TeradataDICoE_MRR
- Premium Member
- Posts: 5
- Joined: Thu Apr 14, 2011 1:30 pm
- Location: Minneapolis
Converting from Oracle to Teradata
Hi,
As a DataStage and Teradata guy I would be happy to answer questions on the migration as you proceed. The comments about not just moving the structures are pretty sound - the database architectures are different so why would anyone assume that the structures should be the same. What out for data type differences - there are lots of good references for this on the web.
Good luck,
Mike R.
As a DataStage and Teradata guy I would be happy to answer questions on the migration as you proceed. The comments about not just moving the structures are pretty sound - the database architectures are different so why would anyone assume that the structures should be the same. What out for data type differences - there are lots of good references for this on the web.
Good luck,
Mike R.
Michael Ruland
DI Architect and Senior Consultant
CoE Data Integration
Teradata Corp
michael.ruland@teradata.com
DI Architect and Senior Consultant
CoE Data Integration
Teradata Corp
michael.ruland@teradata.com