Generic Mendix data exporter/importer

By 28 January 2015 Mendix No Comments

In this post I will explain how I created a very useful reusable Mendix module for the following user story:

As an admin I want to export some of the application’s data, so that I can easily import it in other environments of the same application.

In Mendix, you can create specific export functionality for your entities, using Domain-to-XML-mappings. However, this is a lot of work for and you would need to develop this specifically to the domain model for each application (or even multiple exports within a single application). Another drawback of this approach is that you can only export a simple tree structure. More complex reference structures, like circular references, are not easy to do: to get an idea of what I mean, look at how the export/import functionality of the DBReplication module in the Mendix app store is implemented.

I wanted to create a generic way to export data. Simply selecting the entities and associations of the objects that you want to export should be enough.

The (meta-)domain model

I created the following domain model that will be used to contain the data for exporting and importing. It is basically an abstraction of the Mendix object model:

abstractedDomainModel

 

  • An Export contains 0 or more ExportedObjects. An ExportedObject has a name (EntityName) and an identifying number (ObjectID, unique within the Export. The boolean SearchExistingObject is used to indicate if this ExportedObject needs to be created as a new Mendix object on import (if false), or (if true) that the import should try to find an existing Mendix object of this type with the given attributes where IsKey = true.
  • An ExportedObject contains 0 or more ExportedAttributes. We save the name and the string representation of the value of the attribute in the ExportedAttributes. The boolean IsKey is only used when searching for existing objects. Even for empty attributes an object is creates (vit AttributeValue=empty), because attirbutes can have a default value, which needs to be cleared when importing, if empty.
  • An ExportedObject also can have references to other objects. This is done via 0 or more ExportedAssociation objects. The name of the association is saved in this object. For each reference (the number can be 0 or 1 for references, and 0 or more for reference sets), an ExportedReference objects is created. An ExportedReference saves the ID of the ExportedObject that it refers to (also if the object has SearchExistingObject set to true).

Export definitions

I will allow the user to create “Export Definitions”.  An ExportDefinition defines the entities of which the objects (optionally constrained by XPath) and associated objects, will be exported when the user clicks the Export-button. For example, you could have 1 ExportDefinition to export all your database synchronization settings from the DBReplication-module (database connection settings, table mappings, import calls, etc.).

These settings must be persistant, so I’ve created the following domain model (the self-references are only used to to provide dropdowns in the user interface):

definitionDomainModel

For each association an entity is the parent of, the user can set the way referred objects are exported via the attribute ReferenceHandling, which has 3 possibilities:

  • Don’t export: The association is disregarded and won’t be in the export.
  • Export, create objects on import: The association + the referred objects will be fully exported.
  • Export, on import search for existing object with key(s): The association will be exported, but of the referred objects only key attributes will be exported (the user can select which attributes). On import, these key attributes will be used the look up an existing object in the database, outside of the exported XML.

An Export-object is based on a ExportDefinition, so combined, the domain looks like this (the XMLFile entity is used for the xml file upload/downloads):

XMLImportExport

The user interface

to-do

Creating the XML on export

When the user click the export-button, the following microflow is executed.

Ivk_ExportTemplate

 

The selected ExportDefinition is passed to a Java action that will return an Export-object. After the Export-object has been generated by the Java-magic, it is converted to XML by Mendix, using a Domain-to-XML-mapping. This results in a file, which the user downloads.

For more complex Mendix/Java-combo’s, I always like to use the design pattern where we create a seperate Java class to do the processing. This is the class Exporter, which is called by the above Java action. The method createExport() does the actual processing and creates and returns an Export-MendixObject. Click to see the source code:

Processing the XML on import

When an XML-file is uploaded, the user does not need to have the ExportDefinition in his database, which is a great advantage. This way, you can import your data on any environment, which has the same Mendix model as the one where the data was exported. After selecting an XML-file, the following microflow is executed when the user clicks ‘Upload’:

importMicroflow

First, the XML file is translated to an Export object, including all related objects, using standard Mendix functionality, with this XML-to-domain mapping:

XML_to_domain_mapping

Next, the Export object is given to a Java action, which creates all the objects and sets links between them:

The method handleImport() of the java class Importer does all the work here. Its source is displayed below (click to view):

The Java code uses Mendix’s core functions to create and fill the objects by name. Objects that need to be search by key are retrieved from the database. Finally, the associations between all objects created and found are set, and the objects are committed. That’s it! Now to try it out: I’ll leave that to you.

This was my final Mendix blog entry. Of course, you can contact me by email if you need help on your Mendix projects.

 

 

Leave a Reply

Your email address will not be published.