DSX Splitter

A forum for discussing DataStage<sup>®</sup> basics. If you're not sure where your question goes, start here.

Moderators: chulett, rschirm, roy

Post Reply
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

DSX Splitter

Post by PhilHibbs »

There are many DSX splitters. This is mine. It's written in Perl, tested on ActivePerl 5.10. Currently ignores DSDATATYPES and DSSTAGETYPES as I don't have any examples of those to code against and test on. Creates a subdirectory named after the .dsx file, and expands everything into subdirectories under that. Every object gets its own file and its own copy of the header. Should work with concatenated DSX files but not tested it.

Code: Select all

@rem = '--*-Perl-*--
@echo off
perl "%~dpnx0" %*
goto endofperl
@rem ';
#!perl
#line 8
use strict;
use warnings;
use File::Path;

my $i = "";
my $job;
my $id;
my $sid;
my $file;
my $prev;

my $header = "";
my $header2 = "";

my $dir = $ARGV[0];
$dir =~ s/\.dsx$//;
my $sub;
my $vol;
my $path;
my $name;

rmtree ("$dir");
mkdir($dir);

while ( <> ) {
  if (/ =\+=\+=\+=$/ .. /^=\+=\+=\+=/) {
    unless (/ =\+=\+=\+=$/) { # Don't concatenate the line prior to the beginning =+=+=+=
      $_ = $prev . $_;
    }
    unless (/^=\+=\+=\+=/m) { # Don't skip the rest of the processing on the ending =+=+=+=
      next;
    }
  }

  if (/^BEGIN HEADER/../^END HEADER/) {
    if (/^BEGIN HEADER/) {
      $header = $_;
    } else {
      $header .= $_;
    }
    next;
  }
  if (/^${i}BEGIN /) {
    if (/ DSJOB/) {
      $sub = "Jobs";
      $header2 = "";
      print;
    }
    if (/ DSSHAREDCONTAINER/) {
      $sub = "SharedContainers";
      $header2 = "";
      print;
    }
    if (/ DSEXECJOB/) {
      $sub = "ExecJobs";
      $header2 = "";
      print;
    }
    if (/ DSTABLEDEFS/) {
      $sub = "TableDefs";
      $header2 = $_;
      $i = "   ";
      print;
    }
    if (/ DSTRANSFORMS/) {
      $sub = "Transforms";
      $header2 = $_;
      $i = "   ";
      print;
    }
    if (/ DSROUTINES/) {
      $sub = "Routines";
      $header2 = $_;
      $i = "   ";
      print;
    }
    $job = $header.$header2.$_;
    next;
  }
  if ($sub eq "Jobs" or $sub eq "SharedContainers" or $sub eq "ExecJobs") {
    if (/^   Identifier "(.+)"/) {
      $id = $1;
      $sid = "";
    }
  }
  if ($sub eq "ExecJobs") {
    if (/^      Identifier "(.+)"/) {
      $sid = $1;
    }
  }
  if ($sub eq "TableDefs") {
    if (/^      Identifier "(.+)"/) {
      $id = $1;
      $id =~ s/\\\\/\//g;
    }
  }
  if ($sub eq "Transforms" or $sub eq "Routines") {
    if (/^      Identifier "(.+)"/) {
      $id = $1;
    }
  }
  if (/^      Category "(.+)"/ or
      ($id eq $sid and /^      F28 "(.+)"/)) {
    $file = $1;
    $file =~ s/\\\\/\//g;
    $file =~ s/ /_/g;
    $file .= "/" . $id . ".dsx";
    ($vol,$path,$name) = File::Spec->splitpath( "$dir/$sub/$file" );
    chop $path;
    mkpath($vol.$path);
    print "$vol$path\n$file\n";
    open OUT,">$dir/$sub/$file" or die "Can't open output file $dir/$sub/$file\n";
    print OUT $job;
  }

  if (defined $file) {
    print OUT $_;
  } else {
    $job .= $_;
  }

  if (/^${i}END (DSJOB|HEADER|DSSHAREDCONTAINER|DSEXECJOB|DSRECORD)/) {
    unless (defined $file) {
      $file = $id . ".dsx";
      ($vol,$path,$name) = File::Spec->splitpath( "$dir/$sub/$file" );
      chop $path;
      mkpath($vol.$path);
      print "$vol$path\n$file\n";
      open OUT,">$dir/$sub/$file" or die "Can't open output file $dir/$sub/$file\n";
      print OUT $job;
    }
    if (/DSRECORD/) {
      my $trailer = $header2;
      $trailer =~ s/BEGIN/END/;
      print OUT $trailer;
    }
    close OUT;
    $file = undef;
  }

  if (/^END/) { $i = "" }

} continue {
  $prev = $_;
}

__END__

:endofperl
Phil Hibbs | Capgemini
Technical Consultant
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

Thanks!
rbhudsuder
Participant
Posts: 8
Joined: Mon Aug 09, 2004 3:49 am

Would also post something, but ...

Post by rbhudsuder »

I would like to post the following:
An old DSX splitter: ParseDSX.pl was tuned to also work with shared containers and transforms.
dsx_import.bat: a windows batch file that takes the output directory of the splitter and imports and compiles all components into the given destination project.
An sh+awk script that will check the output of dsx_import.bat, evaluate it and produce a humanly readable result.

Is there no way to avoid having to paste the code into the text of the message but somehow attach files that will be represented by icons, to be downloaded by clicking them?
Or you can ask me for the files at marton.kadar at no gmail, just mail.com
PhilHibbs
Premium Member
Premium Member
Posts: 1044
Joined: Wed Sep 29, 2004 3:30 am
Location: Nottingham, UK
Contact:

Re: Would also post something, but ...

Post by PhilHibbs »

rbhudsuder wrote:Is there no way to avoid having to paste the code into the text of the message but somehow attach files that will be represented by icons, to be downloaded by clicking them?
I don't think so - unless you can host the files somewhere else and link to them. Dropbox is good for that - just put the file in the Public directory, right-click and select the Dropbox menu and choose Copy Public Link, and post the URL that points to the copy of the file on Dropbox's servers. Cheap as free.
Phil Hibbs | Capgemini
Technical Consultant
Post Reply