This is a framework for importing and exporting Torque record objects to and from XML using the org.apache.commons.betwixt package. It depends on the Torque Betwixt generator add-on templates that produce a Schema specific Betwixt mapping file and a DTD file.
This is intended to support generalized import/export functionality that can be created once and will automatically be updated whenever the Torque Database schema is modified.
This can be useful in a variety of ways, such as User and Role importing; transferring subsets of an application's data from production to testing for debugging; or supporting application "property tables" (e.g. Roles and Rights tables) that need to be changed in an installation as a new version is rolled out.
Note: This is based on using Torque V3.2.1 or greater.
The torque-betwixt-runtime-<ver>.jar and the two jars in the lib directory need to be included in your class path for builds and in production. It is also co-dependent on the Torque Runtime's required libraries, but these should be already install if you are using Torque.
For Maven users here's the dependancy info for your project/pom.xml. Note that the common-betwixt doesn't have SNAPSHOT versions on the net and that torque-betwixt is not on the net (yet).
<dependency>
<groupId>torque-addon</groupId>
<artifactId>xml-import-export</artifactId>
<version>1.0</version>
<properties>
<dist.bundle>true</dist.bundle>
</properties>
</dependency>
<dependency>
<groupId>commons-betwixt</groupId>
<artifactId>commons-betwixt</artifactId>
<version>0.8-SNAPSHOT</version>
<url>http://jakarta.apache.org/commons/betwixt/</url>
<properties>
<dist.bundle>true</dist.bundle>
</properties>
</dependency>
<dependency>
<groupId>commons-digester</groupId>
<artifactId>commons-digester</artifactId>
<version>1.7</version>
<url>http://jakarta.apache.org/commons/digester/</url>
<properties>
<dist.bundle>true</dist.bundle>
</properties>
</dependency>
The main drivers in this package are the Importer and Exporter classes. The Importer class is used to convert XML input into Torque objects contained in a Dataset object. The Exporter class does the reverse, takes Torque objects contained in a Dataset and produces XML output. The XML produced will conform (and is syntax checked) to the DTD generated by the torque-betwixt-templates add-on.
Common to both the Importer and Exporter classes is the Resolver object which is responsible for finding the DTD and Betwixt mapping files needed to perform XML transformation. Generally, these will be located somewhere on your application's class path (or in a jar) as determined by your build process and/or torque-betwixt-templates settings.
Also common is the Dataset object which a container for records that have been imported or are to be exported. In addition, it support two import processing attributes. This allows the XML being imported to use the following syntax to define what action needs to be done with this:
In addition, there are various ways that the import process can be defines so that these attributes are overridden or ignored.
Exporting
Exporting Torque objects to XML is simple. Here's some example code:
Dataset xmlRecords = new Dataset();
List records = MyTablePeer.doSelect(criteria);
xmlRecords.addRecordList(records);
xmlRecords.setDescription("Created on " + new Date());
FileWriter outputWriter = new FileWriter( outputFile );
// Define the betwixt map file and dtd file for your application
// Note: Good candidate here for a subclass with app specific settings.
Resolver resolver = new Resolver();
resolver.setDtdPackage("org/my/package/betwixt");
resolver.setDtdFileName("myDB-betwixt.dtd");
resolver.setMapFilePackage("org/my/package/betwixt");
resolver.setMapFileName("myDb.betwixt");
resolver.setDtdUri("http://my.org/xml/myDB-betwixt.dtd");
Exporter exporter = new Exporter();
exporter.exportData(xmlRecords, outputWriter, resolver);
This will produce a file like:
<?xml version='1.0' ?>
<!DOCTYPE dataset SYSTEM "http://my.org/xml/myDB-betwixt.dtd" >
<dataset name="default" description="Created on Tue Aug 29 09:06:58 EDT 2006">
<myTable>
<col1>....</col1>
<col2>....</col2>
<col3/>
...
</myTable>
<myTable>
....
</dataset>
With each <myTable> being a record in that table. Note that if columns are added or dropped from MyTable, this code will still work (assuming the betwixt mapping and DTD files have been regenerated).
Importing
Importing data from XML is actually a two step process. The first is the conversion of the XML input into Torque objects contained in a Dataset object. The second is to take this objects and modify the DB based on the desired action.
In addition, there are several other functions that need to be done during both these processes. First, the XML input used in the first part needs to be validated. Then in the second part, the overall transaction / error processing needs to be managed. E.g., rolling all imported information if an error occurs, and the like.
Finally, as things are imported, it's nice to be able to show users what is happening and/or create a import report and/or notify a sending process of the ongoing status.
The Importer object acts as a controller for all these requirements. It sets up Betwixt with information from the Resolver object to validate and import XML into a Dataset object. It also acts as a transaction manager and dispacher to call RecordHandler objects which perform the DB modification. (More on this below). Finally, it allows for XMLEventListener implimentors to be registered to be notified about various actions as they occure.
Processing the imported objects can often call for application specific actions to be taken. E.g, if a user is being deleted via XML, there may be a lot of other information that needs to be deleted. Or if users will be transferred from one server to another, the user name should be used as a unique id and not a DB Id number.
This is handled by allowing the creation of RecordHandlers for specific object types. Record handlers manage the actual delete, update, or create actions (and reporting their success or failure).
When processing the imported data, the Importer object looks for a registered RecordHandler that has been associated with the record's class or any superclass. It then calls the appropriate recordHandler method.
There is simple generic BaseObjectRecordHandler class supplied with this package. It assumes that all records have a primary key and uses this to process the data.
Here's some sample code on how to import data. Note that Importer has NO pre-registered record handlers.
FileReader importReader = new FileReader ( inputFile );
// Define the betwixt map file and dtd file for your application
// Note: Good candidate here for a subclass with app specific settings.
Resolver resolver = new Resolver();
resolver.setDtdPackage("org/my/package/betwixt");
resolver.setDtdFileName("myDB-betwixt.dtd");
resolver.setMapFilePackage("org/my/package/betwixt");
resolver.setMapFileName("myDb.betwixt");
resolver.setDtdUri("http://my.org/xml/myDB-betwixt.dtd");
SimpleListener status =
new SimpleListener(new OutputStreamWriter(System.out));
org.apache.torque.betwixt.Importer importer =
new org.apache.torque.betwixt.Importer();
// Register record handlers.
// Note: Another good app specific subclass candidate.
importer.registerHandler(MyTable.class,
new MyTableRecordHandler());
importer.registerHandler(BaseObject.class,
new BaseObjectRecordHandler());
importer.registerListener(status);
importer.importData( importReader, resolver );
The source distribution can either be (re)built using Maven or using Ant. There are two library dependancies that can not be obtained from the Ibilio repository. These are:
These need to go into either the Maven repository or the target/lib directory (for ant).