Introducing eTrack

eTrack is an application that manages tasks. This project is meant to be a comprehensive tutorial that demonstrates the use of several Eclipse technologies, and current best practices for architecture, design, development, testing, build, and deployment.  I would like the architecture to be as open as possible to accommodate different implementations of the various components of the application.  There will, however, be a couple of things set in stone.  The application will be based on OSGi, have a RESTful client / server architecture, and will use HTTP to communicate between client and server.

To get started, the initial implementation will use EMF for the domain model, MongoDB for the backing store, Restlet for the RESTful web services, Jetty for the web container, and simple HTML web pages for the client.  It would also be nice, at some point, to have a GWT client, a SproutCore client, and Mylyn integration.

The project is hosted at EclipseLabs and the place to start is:

http://code.google.com/a/eclipselabs.org/p/etrack/

I would like to use Gerrit for source control, and Tycho and Jenkins for build, but I’m currently not aware of any free hosting for these services.  If I can’t find free hosting, I might consider hosting on my home server and mirroring the source on GitHub for public access.  Issue tracking will initially be done on the EclipseLabs Issues page until we can “eat our own cooking”.

Here is a first attempt at a domain model (I’ve taken a little liberty with the UML notation in that solid diamonds denote containment within the same resource while open diamonds denote cross-document containment):

I hope to start working on the implementation at EclipseCon.  If you are interested in participating, please comment below describing what you would like to work on, or find me at EclipseCon and I’ll be happy to discuss this with you.

Mongo EMF

If you have worked with the Eclipse Modeling Framework (EMF), then you already know how easy it is to create domain models.  If you haven’t, then I highly recommend giving it a try.  I develop all of my domain models using EMF.  EMF has built-in support for persisting models to files in XMI, XML, and binary format.  The persistence API is highly extensible, and I’ve recently been working with Ed Merks on some code that persists EMF models to MongoDB.  The project is called mongo-emf and is hosted on EclipseLabs. Version 0.3.1 is being used in a real application and is fairly stable.  This version supports all of the basic CRUD functions along with a basic query language for finding objects.  This blog post will cover the CRUD functions while Ed Merks will have a blog post soon on using the query model.

One feature that I hope you will find attractive is that there are no annotations or XML configuration files required.

If you are attending EclipseCon 2011, I will be giving a talk titled Hardware Developer’s Workbench: a Case Study in which I will discuss how we are using this technology in our application.

Quickstart

Here’s an example on how easy it is to insert an object into MongoDB using EMF.

ResourceSet resourceSet = new ResourceSetImpl();
EList uriHandlers = resourceSet.getURIConverter().getURIHandlers();
uriHandlers.add(0, new MongoDBURIHandlerImpl());

Person user = ModelFactory.eINSTANCE.createMyObject()
user.setName("Test User");
user.setEmail("test@example.org");

Resource resource = resourceSet.createResource(URI.createURI("mongo://localhost/app/users/"));
resource.getContents().add(user);

try
{
  resource.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Let’s look at the code above in detail.  Line 1 is the normal way you create an EMF ResourceSet.  Lines 2 and 3 hook the MongoDB URI handler that interfaces EMF to MongoDB to the ResourceSet. Lines 5 – 7 set up the model instance. Line 9 creates the EMF resource using a URI that has a specific pattern explained below.  Line 10 simply adds the model instance to the EMF Resource.  Line 14 saves and inserts the model instance to MongoDB.

By adding just two additional lines of code and using a mongo URI, you are able to utilize the full power of EMF with your objects persisted in MongoDB.

Mongo EMF URIs

All EMF resources are identified by a URI.  The URI you use when persisting models to MongoDB must have the form:

mongo://<host>/<database>/<collection>/<id>

• host is the hostname of the server running MongoDB (the port may be specified if not default)
• database is the name of the MongoDB database to use
• collection is the name of the MongoDB collection
• id is the unique identifier of the object in MongoDB (optional on create / insert)

The URI path must have exactly three segments. Anything else will cause an IOException on a load() or save().

Create

When inserting a new object into MongoDB, the id segment of the URI is optional and typically the empty string.  By default, MongoDB assigns a unique ID to an object when it is inserted into the database. In this mode, the URI of the EMF resource is automatically updated to include the MongoDB generated ID value.  Going back to the example in the Quickstart, the resource URI will be similar to mongo://localhost/app/users/4d6dc268b03b0db29961472c after the call to save() on line 14.

It is also possible for the client to generate the ID and persist the object to MongoDB using that ID.  The ID can be any string value that is unique within the MongoDB collection.  The quickstart example above would be modified as follows:

Resource resource = resourceSet.createResource(URI.createURI("mongo://localhost/app/users/1"));
resource.getContents().add(user);

try
{
  resource.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Retrieve

To retrieve an object from MongoDB, you load an EMF Resource using the unique URI for the object.

ResourceSet resourceSet = new ResourceSetImpl();
EList uriHandlers = resourceSet.getURIConverter().getURIHandlers();
uriHandlers.add(0, new MongoDBURIHandlerImpl());

Resource resource = resourceSet.getResource(URI.createURI("mongo://localhost/app/users/4d6dc268b03b0db29961472c"), true);
Person user = (Person) resource.getContents().get(0);

Update

To update an object in MongoDB, you simply save the Resource containing the object.

user.setEmail("mongo-emf@example.org);

try
{
  user.eResource().save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Delete

To delete an object in MongoDB, you call delete() on the Resource containing the object.

try
{
  user.eResource().delete(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Collections

A collection of objects may be stored (contained) within a parent object, or stored as individual objects in a MongoDB collection. For objects that are contained as part of a parent object, when you operate on (load, save) the parent object, you operate on (load, save) the entire collection. When a collection is stored as individual objects in a MongoDB collection, each object is contained its own EMF Resource and must be managed individually by the client. If you allow MongoDB to generate the object ID, you may bulk insert a collection of objects in a single save() call. The resource is modified to contain a single Result result object with a proxy to each of the inserted objects. You may iterate over the proxies to load each object into its own Resource. Here is an example of bulk insert:

ResourceSet resourceSet = new ResourceSetImpl();
EList uriHandlers = resourceSet.getURIConverter().getURIHandlers();
uriHandlers.add(0, new MongoDBURIHandlerImpl());

Resource resource = resourceSet.createResource(URI.createURI("mongo://localhost/app/users/"));

for(int i = 0; i < 10; i++)
{
  Person user = ModelFactory.eINSTANCE.createMyObject()
  user.setName("User " + 1);
  user.setEmail("user" + i + "@example.org");
  resource.getContents().add(user);
}

try
{
  resource.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Result result = resource.getContents().get(0);

for(EObject eObject : result.getValues()
{
  Person user = (Person) eObject;
  System.out.println("Person " + user.getName() + " has URI: " + user.eResource.getURI());
}

EMF References

The way in which EMF references are persisted in MongoDB depends on the settings you specified when you created your EMF Ecore and Generator models. Two Ecore settings that affect persistence are: Containment, and Resolve Proxies. The Generator setting that affects persistence is: Containment Proxies. There are three types of references to consider: non-containment, containment, and bi-directional cross-document containment.

Non-containment References

A non-containment reference is modeled by setting Containment = false in the Ecore model. A non-containment reference can be to any other object in the database, a file, or on some other server. The target object may be contained by the referencing object, or could be in some other resource. Non-containment references are always persisted as a proxy. If the target object is in a separate resource from the referencing object, the target object must be persisted, on creation, before the object with the reference so that the proxy URI contains the ID of the target object.

Containment References

A containment reference is modeled by setting Containment = true in the Ecore model. A containment reference is persisted as a nested object in the same MongoDB document as the referencing object if Resolve Proxies = false in the Ecore model or Containment Proxies = false in the Generator model. If Resolve Proxies = true in the Ecore model and Containment Proxies = true in the Generator model, the reference will be persisted as a proxy if the target object is contained in its own Resource (cross-document containment). If the target object is not contained in its own Resource, it will be persisted as a nested object of the referencing object.

Bi-directional Cross-document Containment References

Bi-directional cross-document containment references need special consideration when it comes to saving the objects. For the proxies to be properly persisted, three calls to save() must be made on creation. One of the two objects must be saved twice. The other object must be saved once between the two saves to the other object. For example, consider a bi-directional reference between a Person and an Address, the code to save the two objects would be as follows:

user.setAddress(address);

try
{
  address.save(null);
  user.save(null);
  address.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Modeling Restrictions

There is one minor restriction that you must follow when creating your EMF Ecore model. The following keys are reserved for internal use and may not be used as attribute or reference names:

• _id
• _eId
• _eClass
• _timeStamp
• _eProxyURI

Mongo EMF Project Bundles

The project consists of several core bundles, an example project, a JUnit test bundle, and a JUnit test utility bundle.

• org.eclipselabs.emf.query – this bundle contains the model query support
• org.eclipselabs.mongo – this bundle contains an IMongoDB OSGi service that provides basic connectivity to a MongoDB database
• org.eclipselabs.mongo.emf – this bundle contains the MongoDBURIHandlerImpl that provides EMF persistence to MongoDB
• org.eclipselabs.mongo.freemarker – this bundle contains support for persisting FreeMarker templates in MongoDB
• org.eclipselabs.mongo.emf.examples – this bundle contains a simple example using MongoDBURIHandlerImpl
• org.eclipselabs.mongo.emf.junit – this bundle contains the JUnit tests for the core bundles
• org.eclipselabs.mongo.junit – this bundle contains utilities for users developing their own JUnit tests

Unit Testing

The org.eclipselabs.mongo.junit bundle contains two useful classes that can make writing your unit tests easier.

MongoDatabase

This class is a JUnit @Rule that will clear your MongoDB database after each test. The rule requires the BundleContext and the name of the database to be passed as parameters to its constructor. Here is an example on how to use that rule:

	@Rule
	public MongoDatabase database = new MongoDatabase(Activator.getInstance().getContext(), "junit");

MongoUtil

This utility class contains functions for creating a ResourceSet, getting one or more objects from the database, getting the ID of an object, registering the IMongoDB service with OSGi, and comparing two EObjects. This class has extensive JavaDoc comments that explain the usage of each function.

Launch Configurations

Your launch configuration must include the following bundles and their dependencies:

• org.eclipselabs.emf.query
• org.eclipselabs.mongo
• org.eclipselabs.mongo.emf
• org.eclipse.equinox.ds (or equivalent)

The org.eclipselabs.mongo bundle uses OSGi declarative services to register the IMongoDB service. This service is used by MongoDBURIHandlerImpl to connect to MongoDB.

References

1. Eclipse Modeling Framework
2. MongoDB
3. EclipseLabs mongo-emf project