Restlet Extensions for OSGi Environments 0.5.0 Released

Posted on by

I have released Restlet Extension for OSGi Environments version 0.5.0.  This project provides a framework for building RESTful OSGi web services using dynamic applications, routers, filters, and resources.  The project has been renamed from Restlet integration with OSGi & Equinox, and has also been refactored to be a Restlet extension.  This build is temporary until the extension is officially released as part of Restlet.  This release requires Restlet 2.1M7 or later.

MongoEMF 0.4.0 Released

Posted on by

I have released MongoEMF 0.4.0.  Some of the notable changes include:

  • Reading objects from a query Result has been significantly improved
  • Write performance for bulk stores has improved
  • Added MongoResourceImpl for improved load performance when using URI mappings
  • Added OPTION_USE_ID_ATTRIBUTE_AS_PRIMARY_KEY to store objects using the ID attribute value as the MongoDB primary key

For a complete list of the changes, see the Release Notes.

OSGi at REST

Posted on by

Introduction

I’ve been working on a project to make building RESTful OSGi web applications as simple as creating a basic OSGi service, and I think I have a solution that comes pretty close to this goal. The project is currently called restlet-integration-with-equinox as hosted on EclipseLabs. I am in the process of moving the project to GitHub, and it will be renamed to Restlet Extension for OSGi Environments.  This framework integrates the Restlet RESTful web framework with OSGi allowing you to dynamically register resources, filters, routers, and applications as part of a RESTful web application.

The project originally started out supporting the registration of applications and resources as OSGi services or with an Equinox extension point.  This all worked fine until I started adding the ability to register routers and filters.  The code became complicated and ugly, so I branched the repository and started over.  As the second iteration progressed, it too became complicated, and it occurred to me that the core of the problem (delayed dynamic binding) was identical to the problem that OSGi declarative services has to solve.  That’s when I had that AH-HA moment where the problem becomes clear, and you get to delete all of that ugly, complicated, code.  The solution is to use OSGi declarative services and its dynamic binding capabilities to stitch together the resources, filters, routers, and applications.  And, since you are using OSGi services, you can add and remove resources at runtime and it all just works.

Framework

The Restlet Extension for OSGi Enironments framework is organized as a set of service components and interfaces for you to register as declarative services to form your RESTful web application.  Using these service components, it’s possible to implement a simple application with a router and some resources, or a very complex graph of filters, routers, and resources as demonstrated below.

Application

Here is the API specification for the service interfaces:

Service Interfaces

Here is the API specification for the service components (I have omitted the service interfaces that each provider implements):

Service Components

FilterProvider and ResourceProvider are abstract and must be extended as described below.  RouterProvider and ApplicationProvider can be used without extending them.  The remainder of this post will walk you through examples of creating RESTful web applications using this technology.

Setup

Before you begin development of your application, the target platform needs to be crated with the following frameworks:

If you would like to minimize your target platform, the following bundles are the minimum required:

  • javax.servlet
  • org.eclilpse.equinox.common
  • org.eclipse.equinox.ds
  • org.eclispe.equinox.http.jetty
  • org.eclipse.equinox.http.servlet
  • org.eclipse.equinox.util
  • org.eclipse.osgi
  • org.eclipse.osgi.services
  • org.mortbay.jetty.server
  • org.mortbay.jetty.util
  • org.restlet
  • org.restlet.ext.servlet
  • org.eclipselabs.restlet
  • org.eclipselabs.restlet.servlet

If you are not familiar with setting up your target platform, here are the detailed steps:

  1. Download Restlet JEE 2.0.8 and Restlet Extensions for OSGi Environments 0.3.0.  You do not need to download Equinox – we’ll get that from a p2 repository.
  2. Uncompress the downloads into a targets directory.
  3. Launch Eclipse (I’m using Indigo) and select Preferences -> Plug-in Development -> Target Platform.
  4. Click Add…
  5. Choose Nothing: Start with an empty target definition.
  6. Click Next
  7. Name the target platform.
  8. Click Add…
  9. Choose Directory and click Next.
  10. Add <target directory>/restlet-jee-2.0.8/lib
  11. Click Finish.
  12. Click Add…
  13. Choose Directory and click Next.
  14. Add <target directory>/restlet-ext-osgi
  15. Click Finish.
  16. Click Add…
  17. Choose Software Site.
  18. Work with: http://download.eclipse.org/releases/indigo.
  19. Expand EclipseRT Target Platform Components.
  20. Check Equinox Target Components.
  21. Click Finish.
  22. Click Finish.
  23. Check the target definition you just created to make it Active.
  24. Click OK.

Edit Target Definition 1

Let’s test the target definition.  Create an OSGi launch configuration with the following bundles:

  • javax.servlet
  • org.eclipse.equinox.http.jetty
  • org.eclipse.equinox.http.servlet
  • org.eclipse.osgi
  • org.eclipse.osgi.services
  • org.mortbay.jetty.server
  • org.mortbay.jetty.util

Set Default Auto-Start: true (not ideal, but it gets you up and running quickly) and add the following VM argument: -Dorg.osgi.service.http.port=8080.

Run Configurations 3

Run the application and point your web browser to http://localhost:8080/.  You should get a 404 error.  This lets you know that jetty is responding to your request, but there are currently no resources to handle the request.  For more information on setting up OSGi as a web server, see my blog post: OSGi as a Web Application Server.

Hello RESTful World

For this tutorial we are going to create a single resource at the URI http://localhost:8080/hello with a plain text representation.  We need to create and wire together three objects: Application, Router, and ServerResource.  Start by creating a new Plug-in Project called org.eclipselabs.restlet.examples.app.  You do not need a bundle activator.

Open the MANIFEST.MF and add the following imported packages:

  • org.eclipselabs.restlet
  • org.restlet
  • org.restlet.resource

Create a new package: org.eclipselabs.restlet.examples.app and add the following HelloResource:

package org.eclipselabs.restlet.examples.app;

import org.restlet.resource.Get;
import org.restlet.resource.ServerResource;

public class HelloResource extends ServerResource
{
  @Get("txt")
  public String sayHello()
  {
    return "Hello RESTful World";
  }
}

In that same package, add the following HelloResourceProvider:

import org.eclipselabs.restlet.ResourceProvider;
import org.restlet.Context;
import org.restlet.resource.Finder;

public class HelloResourceProvider extends ResourceProvider
{
  @Override
  protected Finder createFinder(Context context)
  {
    return new Finder(context, HelloResource.class);
  }
}

At the root of the project create an OSGI-INF folder to hold our service declarations.  In the OSGi-INF folder, create the following declarative service as application.xml:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app">
   <implementation class="org.eclipselabs.restlet.ApplicationProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IApplicationProvider"/>
   </service>
   <property name="alias" type="String" value="/"/>
   <reference bind="bindRouterProvider" cardinality="1..1" interface="org.eclipselabs.restlet.IRouterProvider" name="IRouterProvider" policy="static" unbind="unbindRouterProvider"/>
</scr:component>

Create the following declarative service as router.xml:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app.router">
   <implementation class="org.eclipselabs.restlet.RouterProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IRouterProvider"/>
   </service>
   <reference bind="bindResourceProvider" cardinality="1..n" interface="org.eclipselabs.restlet.IResourceProvider" name="IResourceProvider" policy="dynamic" unbind="unbindResourceProvider"/>
</scr:component>

Create the following declarative service as hello.xml:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app.hello">
   <implementation class="org.eclipselabs.restlet.examples.app.HelloResourceProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IResourceProvider"/>
   </service>
   <property name="paths" type="String">
     /hello
   </property>
</scr:component>

If you are not using the component definition editor to create your declarative services, be sure to add the following to the MANIFEST.MF:

Service-Component: OSGI-INF/application.xml, OSGI-INF/router.xml, OSGI-INF/hello.xml

You are now ready to run and test your web application.  Add the following bundles to the launch configuration you created in the Setup step:

  • org.eclipselabs.restlet.examples.app
  • org.eclipse.equinox.common
  • org.eclipse.equinox.ds
  • org.eclipse.equinox.util
  • org.eclipselabs.restlet
  • org.eclipselabs.restlet.servlet
  • org.restlet
  • org.restlet.ext.servlet

Run the application and point your browser to http://localhost:8080/hello.  You should see “Hello RESTful World”.

Run Configurations 2

One of the neat features of Restlet is that you can specify more than one representation in a resource.  Let’s add an HTML representation to our hello resource:

package org.eclipselabs.restlet.examples.app;

import org.restlet.resource.Get;
import org.restlet.resource.ServerResource;

public class HelloResource extends ServerResource
{
  @Get("txt")
  public String sayHello()
  {
    return "Hello RESTful World";
  }

  @Get("html")
  public String getDocument()
  {
    StringBuilder html = new StringBuilder();
    html.append("<html>\n");
    html.append("  <body>\n");
    html.append("    <h2>Hello RESTful World</h2>\n");
    html.append("   </body>\n");
    html.append("</html>\n");
    return html.toString();
  }
}

Now, when you reload http://localhost:8080/hello in your browser, you should see “Hello RESTful World” in a Heading 2 style.  The complete solution for this example application can be downloaded from the Restlet Extensions for OSGi Environments downloads page.

Applications

Your OSGi web application will consist of one or more Restlet Application instances.  An application is created by declaring an OSGi service component of type ApplicationProvider which provides the service IApplicationProvider, and has an “alias” property that specifies the root path of the application.  The alias of each Restlet application in your OSGi application must be unique, start with “/” and not end with “/” with the exception of “/” itself.  It should not be necessary to extend ApplicationProvider, but you must declare the service in your own bundle.  The ApplicationProvider has a dependency on the IRouterProvider service.  This dependency should be declared as static with a cardinality of 1..1 and bind functions (un)bindRouterProvider().

Suppose you wanted to create more than one Application.  Each Application needs its own instance of a Router.  How does OSGi declarative services know which IRouterProvider to bind to which ApplicationProvider?  When you declare the dependency from ApplicationProvider to IRouterProvider, there is a target parameter you can set to specify the exact IRouterProvider to bind against.  Here is an example of declaring two applications each binding to a unique router using the component.name property given to the service instance.

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app1">
   <implementation class="org.eclipselabs.restlet.ApplicationProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IApplicationProvider"/>
   </service>
   <property name="alias" type="String" value="/app1"/>
   <reference bind="bindRouterProvider" cardinality="1..1" interface="org.eclipselabs.restlet.IRouterProvider" name="IRouterProvider" policy="static" target="(component.name=org.eclipselabs.restlet.examples.app.router1)" unbind="unbindRouterProvider"/>
</scr:component>

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app2">
   <implementation class="org.eclipselabs.restlet.ApplicationProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IApplicationProvider"/>
   </service>
   <property name="alias" type="String" value="/app2"/>
   <reference bind="bindRouterProvider" cardinality="1..1" interface="org.eclipselabs.restlet.IRouterProvider" name="IRouterProvider" policy="static" target="(component.name=org.eclipselabs.restlet.examples.app.router2)" unbind="unbindRouterProvider"/>
</scr:component>

Routers

Every Restlet Application instance needs at least one Router instance.  Technically speaking, this isn’t entirely true, but for all practical purposes, you are going to want a Router.  A router is created by declaring an OSGi service component of type RouterProvider which provides the service IRouterProvider.  The RouterProvider has a dependency on IResourceProvider and IRouterProvider services.  The dependency on IResourceProvider should be declared as dynamic with a cardinality of 1..n and bind functions (un)bindResourceProvider().  This allows more than one resource to bind to a router and allows the resources to come and go without taking down the application.  The dependency on IRouterProvider is optional and if declared should be dynamic with a cardinality of 0..1 and bind functions (un)bindDefaultRouterProvider().  The default router is asked to handle the URI routing when none of the resources attached to the router can handle the URI.  The default router will make more sense when I discuss filters below.

When an application contains multiple routers, a target filter must be applied to the IResourceProvider services so that the resources are bound to the correct router.  Since there can be many resources attached to a router, and each resource has a unique component.name, it’s not practical to use the component.name as a filter.  I recommend giving each resource a property with a common value that is unique to a router.  For example, if a resource has the property: type = public, then the router provider can bind to resources with the target (type = public).  For the case of multiple applications, you may want to create a property that identifies the application such as: app = app1.  The binding target is an LDAP filter, so you can combine both the app and type properties: (&(app = app1)(type = public)).  Here is an example of declaring two routers for two different applications:

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app.router1">
   <implementation class="org.eclipselabs.restlet.RouterProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IRouterProvider"/>
   </service>
   <reference bind="bindResourceProvider" cardinality="1..n" interface="org.eclipselabs.restlet.IResourceProvider" name="IResourceProvider" policy="dynamic" target="(app=app1)" unbind="unbindResourceProvider"/>
</scr:component>

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app.router2">
   <implementation class="org.eclipselabs.restlet.RouterProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IRouterProvider"/>
   </service>
   <reference bind="bindResourceProvider" cardinality="1..n" interface="org.eclipselabs.restlet.IResourceProvider" name="IResourceProvider" policy="dynamic" target="(app=app2)" unbind="unbindResourceProvider"/>
</scr:component>

Resources

Resources are where you define exactly how an external client interacts with your application by responding to HTTP GET, PUT, POST, and DELETE requests.  You must create two classes for every logical resource your application provides: the resource class itself extending ServerResource from the Restlet framework, and a resource provider extending ResourceProvider from the Restlet Extension for OSGi Environments framework.  A resource is created by declaring an OSGi service component of the type you extended from ResourceProvider which provides the service IResourceProvider.  The ResourceProvider does not require dependencies on any other services, but does require a property named paths which is a string array of paths relative to the application alias.  The resource is registered with the router for each path declared in the paths property.  Note that the PDE component definition editor does not support array properties, so you will have to switch to the Source tab to create the paths property.  It is necessary to extend ResourceProvider to provide the classloading “glue” between Restlet and OSGi.  Your resource provider should override Finder createFinder(Context context) and return an instance of a new Finder passing in the Context and the class of your resource.

Here is an example of two resources that respond to http://localhost:8080/hello and http://localhost:8080/example:

package org.eclipselabs.restlet.examples.app;

import org.restlet.resource.Get;
import org.restlet.resource.ServerResource;

public class HelloResource extends ServerResource
{
  @Get("txt")
  public String sayHello()
  {
    return "Hello RESTful World";
  }
}

import org.eclipselabs.restlet.ResourceProvider;
import org.restlet.Context;
import org.restlet.resource.Finder;

public class HelloResourceProvider extends ResourceProvider
{
  @Override
  protected Finder createFinder(Context context)
  {
    return new Finder(context, HelloResource.class);
  }
}

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.app.hello">
   <implementation class="org.eclipselabs.restlet.examples.app.HelloResourceProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IResourceProvider"/>
   </service>
   <property name="paths" type="String">
     /hello
   </property>
</scr:component>

package org.eclipselabs.restlet.examples.resource;

import org.restlet.resource.Get;
import org.restlet.resource.ServerResource;

public class ExampleResource extends ServerResource
{
  @Get
  @Override
  public String toString()
  {
    return "Example";
  }
}

package org.eclipselabs.restlet.examples.resource;

import org.eclipselabs.restlet.ResourceProvider;
import org.restlet.Context;
import org.restlet.resource.Finder;

public class ExampleResourceProvider extends ResourceProvider
{
  @Override
  protected Finder createFinder(Context context)
  {
    return new Finder(context, ExampleResource.class);
  }
}

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.resource">
   <implementation class="org.eclipselabs.restlet.examples.resource.ExampleResourceProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.IResourceProvider"/>
   </service>
   <property name="paths" type="String">
   	/example
   </property>
</scr:component>

Filters

Filters, just as the name implies, provide a mechanism for filtering a request as it is routed to the resource.  The typical use-case for a filter is to protect resources from unauthorized access.  Filters are created similar to resources by extending both Filter and FilterProvider.  A filter is created by declaring an OSGi service component of the type you extended from FilterProvider which provides the service IFilterProvider.   The FilterProvider does not require dependencies on any other services.  Your filter provider should override Filter createFilter(Context context) and return an instance of a new Filter passing in the Context.

Filters can be wired to routers, resources, and other filters.  The wiring is done by placing a service dependency on IFilterProvider in the component being filtered.  Notice that RouterProvider, ResourceProvider and FilterProvider all extend from RestletProvider which has a bindFilterProvider() function.  When declaring filters, you will probably want to use the component.name property as a target.  Here is an example of an authentication and authorization filter:

package org.eclipselabs.restlet.examples.security;

import org.restlet.Context;
import org.restlet.Request;
import org.restlet.Response;
import org.restlet.data.Cookie;
import org.restlet.security.Authenticator;
import org.restlet.security.User;

public class SessionAuthenticator extends Authenticator
{
  public SessionAuthenticator(Context context)
  {
    super(context, false, new SessionEnroller());
  }

  @Override
  public boolean authenticate(Request request, Response response)
  {
    Cookie sessionCookie = request.getCookies().getFirst("session");
    Account account = AccountManager.getInstance().getAccount(sessionCookie.getSecond());

    if(account != null)
    {
      User user = new User(account.getCredential().getId(), account.getID());
      request.getClientInfo().setUser(user);
      return true;
    }

    return false;
  }
}

package org.eclipselabs.restlet.examples.security.providers;

import org.eclipselabs.restlet.components.FilterProvider;
import org.restlet.Context;
import org.restlet.routing.Filter;

import org.eclipselabs.restlet.examples.security.SessionAuthenticator;

public class SessionAuthenticationFilterProvider extends FilterProvider
{
  @Override
  protected Filter createFilter(Context context)
  {
    return new SessionAuthenticator(context);
  }
}

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" immediate="true" name="org.eclipselabs.restlet.examples.authentication">
   <implementation class="org.eclipselabs.restlet.examples.security.SessionAuthenticationFilterProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.providers.IFilterProvider"/>
   </service>
</scr:component>

package org.eclipselabs.restlet.examples.security;

import org.restlet.Request;
import org.restlet.Response;
import org.restlet.security.Authorizer;
import org.restlet.security.Role;

public class SessionAuthorizer extends Authorizer
{
  @Override
  protected boolean authorize(Request request, Response response)
  {
    String accountID = request.getOriginalRef().getSegments().get(2);
    return accountID != null && accountID.equals(new String(request.getClientInfo().getUser().getSecret()));
  }
}

package org.eclipselabs.restlet.examples.security.providers;

import org.eclipselabs.restlet.components.FilterProvider;
import org.restlet.Context;
import org.restlet.routing.Filter;

import org.eclipselabs.restlet.examples.security.SessionAuthorizer;

public class SessionAuthorizerFilterProvider extends FilterProvider
{
  @Override
  protected Filter createFilter(Context context)
  {
    return new SessionAuthorizer();
  }
}

<?xml version="1.0" encoding="UTF-8"?>
<scr:component xmlns:scr="http://www.osgi.org/xmlns/scr/v1.1.0" name="org.eclipselabs.restlet.examples.authorization">
   <implementation class="org.eclipselabs.restlet.examples.security.providers.SessionAuthorizerFilterProvider"/>
   <reference bind="bindFilterProvider" cardinality="1..1" interface="org.eclipselabs.restlet.providers.IFilterProvider" name="IFilterProvider" policy="static" target="(component.name=org.eclipselabs.restlet.examples.authentication)" unbind="unbindFilterProvider"/>
   <service>
      <provide interface="org.eclipselabs.restlet.providers.IFilterProvider"/>
   </service>
</scr:component>

Service Dependency Injection

The Restlet Extensions for OSGi Environments framework provides support for dependency injection into your resources.  If your resource extends InjectedServerResource and your resource provider creates an instance of InjectedFinder, you may then use @Inject in your resource to obtain references to other services such as the OSGi LogService.  Here is an example of creating a resource that uses dependency injection with the OSGi LogService.

package org.eclipselabs.restlet.examples.resources;

import javax.inject.Inject;

import org.eclipselabs.restlet.di.eclipse.InjectedServerResource;
import org.osgi.service.log.LogService;

public class MyServerResource extends InjectedServerResource
{
  protected void log(int level, String message)
  {
    log(level, message, null);
  }

  protected void log(int level, String message, Throwable exception)
  {
    LogService logService = this.logService;

    if (logService != null)
      logService.log(level, message, exception);
  }

  @Inject
  private LogService logService;
}

References

  1. Restlet – RESTful web framework
  2. Restlet Extension for OSGi Environments – Restlet integration with OSGi
  3. Restlet in Action – Book on Restlet
  4. OSGi as a Web Application Server – Blog post on setting up OSGi as a web application server

Update Sept 28, 2011: Changed the project name to Restlet Extension for OSGi Environments and updated link to GitHub.

MongoEMF 0.3.2 Released

Posted on by

I have released MongoEMF 0.3.2.  This build adds support for storing OSGi log entries to MongoDB, and includes bug fixes for queries against dates and defaults values.

OSGi Logging in MongoDB

I’ve added a new bundle, org.eclispelabe.mongo.emf.log, that will convert an OSGi LogEntry into an EMF object and store it into MongoDB.  Storing your OSGi log entries into MongoDB is a simple matter of configuring the ILogService (declarative service) with the OSGI ConfigurationAdmin service.  I have created a LogServiceConfigurator helper class if you’d rather not deal with ConfigurationAdmin.  The log service requires two properties:

  • baseURI
  • logLevel

The baseURI must be a valid MongoEMF collection URI of the form mongo://host/database/collection/ - for example: mongo://localhost/data/logs/.  The logLevel is set to one of the logging levels defined in the OSGi LogService: LOG_ERROR, LOG_WARNING, LOG_INFO, and LOG_DEBUG.

Here is an example configuration using the LogServiceConfigurator for logging errors and warnings:

 
LogServiceConfigurator.configureLogService(URI.createURI("mongo://localhost/junit/logs/"), LogService.LOG_WARNING);

Querying Default Values

The standard EMF serializations: XML, XMI, and Binary do not store default values for attributes.  The Mongo EMF binding was written this way as well to maintain consistency, but has the side effect of not being able to query objects based on attributes set to their default value.  To solve this problem, we’ve added the option:

MongoDBURIHandlerImpl.OPTION_SERIALIZE_DEFAULT_ATTRIBUTE_VALUES

Setting this option to Boolean.TRUE when saving your object to MongoDB will force attributes with default values to be persisted to MongoDB.  Here is a simple example:

Resource resource = resourceSet.createResource(uri);
resource.getContents().add(object);
HashMap<String, Object> options = new HashMap<String, Object>(1);
options.put(MongoDBURIHandlerImpl.OPTION_SERIALIZE_DEFAULT_ATTRIBUTE_VALUES, Boolean.TRUE);

try
{
  resource.save(options);
}
catch(IOException w)
{
  e.printStackTrace();
}

Introducing eTrack

Posted on by

eTrack is an application that manages tasks. This project is meant to be a comprehensive tutorial that demonstrates the use of several Eclipse technologies, and current best practices for architecture, design, development, testing, build, and deployment.  I would like the architecture to be as open as possible to accommodate different implementations of the various components of the application.  There will, however, be a couple of things set in stone.  The application will be based on OSGi, have a RESTful client / server architecture, and will use HTTP to communicate between client and server.

To get started, the initial implementation will use EMF for the domain model, MongoDB for the backing store, Restlet for the RESTful web services, Jetty for the web container, and simple HTML web pages for the client.  It would also be nice, at some point, to have a GWT client, a SproutCore client, and Mylyn integration.

The project is hosted at EclipseLabs and the place to start is:

http://code.google.com/a/eclipselabs.org/p/etrack/

I would like to use Gerrit for source control, and Tycho and Jenkins for build, but I’m currently not aware of any free hosting for these services.  If I can’t find free hosting, I might consider hosting on my home server and mirroring the source on GitHub for public access.  Issue tracking will initially be done on the EclipseLabs Issues page until we can “eat our own cooking”.

Here is a first attempt at a domain model (I’ve taken a little liberty with the UML notation in that solid diamonds denote containment within the same resource while open diamonds denote cross-document containment):

DomainI hope to start working on the implementation at EclipseCon.  If you are interested in participating, please comment below describing what you would like to work on, or find me at EclipseCon and I’ll be happy to discuss this with you.

Mongo EMF

Posted on by

If you have worked with the Eclipse Modeling Framework (EMF), then you already know how easy it is to create domain models.  If you haven’t, then I highly recommend giving it a try.  I develop all of my domain models using EMF.  EMF has built-in support for persisting models to files in XMI, XML, and binary format.  The persistence API is highly extensible, and I’ve recently been working with Ed Merks on some code that persists EMF models to MongoDB.  The project is called mongo-emf and is hosted on EclipseLabs. Version 0.3.1 is being used in a real application and is fairly stable.  This version supports all of the basic CRUD functions along with a basic query language for finding objects.  This blog post will cover the CRUD functions while Ed Merks will have a blog post soon on using the query model.

One feature that I hope you will find attractive is that there are no annotations or XML configuration files required.

If you are attending EclipseCon 2011, I will be giving a talk titled Hardware Developer’s Workbench: a Case Study in which I will discuss how we are using this technology in our application.

Quickstart

Here’s an example on how easy it is to insert an object into MongoDB using EMF.

ResourceSet resourceSet = new ResourceSetImpl();
EList uriHandlers = resourceSet.getURIConverter().getURIHandlers();
uriHandlers.add(0, new MongoDBURIHandlerImpl());

Person user = ModelFactory.eINSTANCE.createMyObject()
user.setName("Test User");
user.setEmail("test@example.org");

Resource resource = resourceSet.createResource(URI.createURI("mongo://localhost/app/users/"));
resource.getContents().add(user);

try
{
  resource.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Let’s look at the code above in detail.  Line 1 is the normal way you create an EMF ResourceSet.  Lines 2 and 3 hook the MongoDB URI handler that interfaces EMF to MongoDB to the ResourceSet. Lines 5 – 7 set up the model instance. Line 9 creates the EMF resource using a URI that has a specific pattern explained below.  Line 10 simply adds the model instance to the EMF Resource.  Line 14 saves and inserts the model instance to MongoDB.

By adding just two additional lines of code and using a mongo URI, you are able to utilize the full power of EMF with your objects persisted in MongoDB.

Mongo EMF URIs

All EMF resources are identified by a URI.  The URI you use when persisting models to MongoDB must have the form:

mongo://<host>/<database>/<collection>/<id>

  • host is the hostname of the server running MongoDB (the port may be specified if not default)
  • database is the name of the MongoDB database to use
  • collection is the name of the MongoDB collection
  • id is the unique identifier of the object in MongoDB (optional on create / insert)

The URI path must have exactly three segments. Anything else will cause an IOException on a load() or save().

Create

When inserting a new object into MongoDB, the id segment of the URI is optional and typically the empty string.  By default, MongoDB assigns a unique ID to an object when it is inserted into the database. In this mode, the URI of the EMF resource is automatically updated to include the MongoDB generated ID value.  Going back to the example in the Quickstart, the resource URI will be similar to mongo://localhost/app/users/4d6dc268b03b0db29961472c after the call to save() on line 14.

It is also possible for the client to generate the ID and persist the object to MongoDB using that ID.  The ID can be any string value that is unique within the MongoDB collection.  The quickstart example above would be modified as follows:

Resource resource = resourceSet.createResource(URI.createURI("mongo://localhost/app/users/1"));
resource.getContents().add(user);

try
{
  resource.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Retrieve

To retrieve an object from MongoDB, you load an EMF Resource using the unique URI for the object.

ResourceSet resourceSet = new ResourceSetImpl();
EList uriHandlers = resourceSet.getURIConverter().getURIHandlers();
uriHandlers.add(0, new MongoDBURIHandlerImpl());

Resource resource = resourceSet.getResource(URI.createURI("mongo://localhost/app/users/4d6dc268b03b0db29961472c"), true);
Person user = (Person) resource.getContents().get(0);

Update

To update an object in MongoDB, you simply save the Resource containing the object.

user.setEmail("mongo-emf@example.org);

try
{
  user.eResource().save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Delete

To delete an object in MongoDB, you call delete() on the Resource containing the object.

try
{
  user.eResource().delete(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Collections

A collection of objects may be stored (contained) within a parent object, or stored as individual objects in a MongoDB collection. For objects that are contained as part of a parent object, when you operate on (load, save) the parent object, you operate on (load, save) the entire collection. When a collection is stored as individual objects in a MongoDB collection, each object is contained its own EMF Resource and must be managed individually by the client. If you allow MongoDB to generate the object ID, you may bulk insert a collection of objects in a single save() call. The resource is modified to contain a single Result result object with a proxy to each of the inserted objects. You may iterate over the proxies to load each object into its own Resource. Here is an example of bulk insert:

ResourceSet resourceSet = new ResourceSetImpl();
EList uriHandlers = resourceSet.getURIConverter().getURIHandlers();
uriHandlers.add(0, new MongoDBURIHandlerImpl());

Resource resource = resourceSet.createResource(URI.createURI("mongo://localhost/app/users/"));

for(int i = 0; i < 10; i++)
{
  Person user = ModelFactory.eINSTANCE.createMyObject()
  user.setName("User " + 1);
  user.setEmail("user" + i + "@example.org");
  resource.getContents().add(user);
}

try
{
  resource.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Result result = resource.getContents().get(0);

for(EObject eObject : result.getValues()
{
  Person user = (Person) eObject;
  System.out.println("Person " + user.getName() + " has URI: " + user.eResource.getURI());
}

EMF References

The way in which EMF references are persisted in MongoDB depends on the settings you specified when you created your EMF Ecore and Generator models. Two Ecore settings that affect persistence are: Containment, and Resolve Proxies. The Generator setting that affects persistence is: Containment Proxies. There are three types of references to consider: non-containment, containment, and bi-directional cross-document containment.

Non-containment References

A non-containment reference is modeled by setting Containment = false in the Ecore model. A non-containment reference can be to any other object in the database, a file, or on some other server. The target object may be contained by the referencing object, or could be in some other resource. Non-containment references are always persisted as a proxy. If the target object is in a separate resource from the referencing object, the target object must be persisted, on creation, before the object with the reference so that the proxy URI contains the ID of the target object.

Containment References

A containment reference is modeled by setting Containment = true in the Ecore model. A containment reference is persisted as a nested object in the same MongoDB document as the referencing object if Resolve Proxies = false in the Ecore model or Containment Proxies = false in the Generator model. If Resolve Proxies = true in the Ecore model and Containment Proxies = true in the Generator model, the reference will be persisted as a proxy if the target object is contained in its own Resource (cross-document containment). If the target object is not contained in its own Resource, it will be persisted as a nested object of the referencing object.

Bi-directional Cross-document Containment References

Bi-directional cross-document containment references need special consideration when it comes to saving the objects. For the proxies to be properly persisted, three calls to save() must be made on creation. One of the two objects must be saved twice. The other object must be saved once between the two saves to the other object. For example, consider a bi-directional reference between a Person and an Address, the code to save the two objects would be as follows:

user.setAddress(address);

try
{
  address.save(null);
  user.save(null);
  address.save(null);
}
catch(IOException e)
{
  e.printStackTrace();
}

Modeling Restrictions

There is one minor restriction that you must follow when creating your EMF Ecore model. The following keys are reserved for internal use and may not be used as attribute or reference names:

  • _id
  • _eId
  • _eClass
  • _timeStamp
  • _eProxyURI

Mongo EMF Project Bundles

The project consists of several core bundles, an example project, a JUnit test bundle, and a JUnit test utility bundle.

  • org.eclipselabs.emf.query – this bundle contains the model query support
  • org.eclipselabs.mongo – this bundle contains an IMongoDB OSGi service that provides basic connectivity to a MongoDB database
  • org.eclipselabs.mongo.emf – this bundle contains the MongoDBURIHandlerImpl that provides EMF persistence to MongoDB
  • org.eclipselabs.mongo.freemarker – this bundle contains support for persisting FreeMarker templates in MongoDB
  • org.eclipselabs.mongo.emf.examples – this bundle contains a simple example using MongoDBURIHandlerImpl
  • org.eclipselabs.mongo.emf.junit – this bundle contains the JUnit tests for the core bundles
  • org.eclipselabs.mongo.junit – this bundle contains utilities for users developing their own JUnit tests

Unit Testing

The org.eclipselabs.mongo.junit bundle contains two useful classes that can make writing your unit tests easier.

MongoDatabase

This class is a JUnit @Rule that will clear your MongoDB database after each test. The rule requires the BundleContext and the name of the database to be passed as parameters to its constructor. Here is an example on how to use that rule:

	@Rule
	public MongoDatabase database = new MongoDatabase(Activator.getInstance().getContext(), "junit");

MongoUtil

This utility class contains functions for creating a ResourceSet, getting one or more objects from the database, getting the ID of an object, registering the IMongoDB service with OSGi, and comparing two EObjects. This class has extensive JavaDoc comments that explain the usage of each function.

Launch Configurations

Your launch configuration must include the following bundles and their dependencies:

  • org.eclipselabs.emf.query
  • org.eclipselabs.mongo
  • org.eclipselabs.mongo.emf
  • org.eclipse.equinox.ds (or equivalent)

The org.eclipselabs.mongo bundle uses OSGi declarative services to register the IMongoDB service. This service is used by MongoDBURIHandlerImpl to connect to MongoDB.

References

  1. Eclipse Modeling Framework
  2. MongoDB
  3. EclipseLabs mongo-emf project

Rational Team Concert 3.0 on OS X

Posted on by

Update 4/19/2011: The server zips are back and you no longer have to install on linux and move to OS X.

Rational Team Concert 3.0 is a nice development tool built on Eclipse that runs just fine on Mac OS X even though it’s not officially supported.  This guide will help you install and configure RTC 3.0 on OS X.  The client is very easy to install.  The server used to be easy to install, but some recent changes in the RTC builds has made it much more difficult.  Basically, you install the server on Linux, and transfer it to your OS X box.

Installing RTC 3.0 Server on Linux

Since the RTC Server is no longer available as a zip download, you must install the server on a supported platform such as Linux.  Once the server is installed, you can transfer the content to OS X and it will just work.

  • Fire up a Linux box, Parallels, or VMWare (I use Parallels with Ubuntu)
  • Head over to https://jazz.net/downloads and select Rational Team Concert
  • Select Linux x86-32/64 for your platform and click Download Rational Team Concert
  • After logging and accepting the the terms, RTC-Web-Installer-Linux-3.0.zip will be downloaded
  • Unzip the archive into a folder and run launchpad.sh
  • Uncheck Install in a shared location for multiple users
  • Click Jazz Team Server and CCM Application under Install the Server
  • The Installation Manager will start – login with your jazz.net id and password
  • Click Next
  • Agree to the license and click Next
  • Keep the default directory locations and click Next
  • Keep the default installation directory and click Next
  • Select the desired translations and click Next
  • Click Next
  • Select Not upgrading from previous product release and click Next
  • Click Install
  • After the server is installed, click Finish

Moving RTC Server to OS X

Transfer the contents of the JazzTeamServer directory to OS X.  Here is the method I used:

  • cd ~/IBM
  • zip -r jazz.zip JazzTeamServer
  • Copy jazz.zip to your OS X box (I use a Parallels shared folder)
  • unzip jazz.zip into the directory of your choice (I use /Library)
  • cd JazzTeamServer/server
  • rm -rf jre

Configuring RTC Server

You configure the RTC 3.0 Server as you would for any other platform.  If you are using OS X Server and want to set up LDAP authentication, skip that configuration during the setup.  The setup wizard will not configure it correctly for OS X.  I have instructions below on how to configure the Tomcat JNDI realm by hand below.

  • ./server.startup
  • Point your web browser to https://localhost:9443/jts/setup
  • Login id: ADMIN password: ADMIN
  • Click Next
  • Set the Public URI Root to https://<hostname&gt;:9443/jts (where <hostname> is the hostname of your system)
  • Click Test Connection and then click Next
  • Keep the default Derby database settings and click Next
  • Set up Email Notification as desired and click Next
  • Set up your account and click Next
  • Click Register Applications and click Next
  • Keep the default URI, click Test Connection and click Next
  • Keep the default Derby configuration and click Next
  • Click Finalize Application Setup and click Next
  • Click Finish
  • (Optional) Set up LDAP for use with OS X Server
    • ./server.shutdown
    • edit tomcat/conf/server.xml
    • Disable the database realm
    • Configure the JNDI realm as shown below
    • ./server.startup
<Realm className="org.apache.catalina.realm.JNDIRealm"
    connectionName=""
    connectionURL="ldap://<hostname>:389"
    roleBase="cn=groups,dc=<host>,dc=<domain>,dc=com"
    roleName="cn"
    roleSearch="(memberUid={1})"
    roleSubtree="true"
    userBase="cn=users,dc=<host>,dc=<domain>,dc=com"
    userSearch="(uid={0})"
    userSubtree="true"/>
  • Install your license as you normally would

You should now have a functional server.  If you are using OS X Server, remember that the LDAP groups are case sensitive, so make sure you create a JazzUsers group and not a jazzusers group.

Starting RTC Server on Boot

If you would like the RTC Server automatically started when OS X boots, put the following in /Library/LaunchDaemons/com.ibm.rtc.plist

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.ibm.rtc</string>
  <key>OnDemand</key>
  <true/>
  <key>Program</key>
  <string>server.startup</string>
  <key>AbandonProcessGroup</key>
  <true/>
  <key>RunAtLoad</key>
  <true/>
  <key>WorkingDirectory</key>
  <string>/Library/JazzTeamServer/server</string>
</dict>
</plist>

At the command prompt: sudo launchctl load com.ibm.rtc.plist

Installing RTC Client on OS X

Installing the client on OS X is super simple using the P2 repository.

  • From the All Downloads tab on jazz.net, scroll down to the section containing the zips and download the P2 Install Repository
  • unzip the downlaod
  • From the Eclipse IDE, choose Help -> Install New Software …
  • Add the directory to the unzipped P2 repository
  • Check Rational Team Concert and install using the wizard