ModeShape

An open-source, federated content repository

ModeShape 5.3.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 5.3.0.Final.

This minor release addresses 11 bugs and 2 enhancements, most notably:

  • adding replica set support for the Mongo DB binary store to both the JSON and Wildfly configuration (see MODE-2635)
  • adding the ability to update the configuration for external sources “on the fly”, meaning that the repository does not need to be restarted each time
    an external source is added or changed (see MODE-2650)

You can migrate from ModeShape 3 and ModeShape 4 by following our migration guide. If you’re starting to use ModeShape for the first time, make sure you read the getting started guide.

The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: uncategorized

ModeShape 4.6.2.Final is available

The ModeShape community is proud to announce the immediately availability of ModeShape 4.6.2.Final.

This is a micro-release off the 4.x branch which addresses a couple of repository backup issues: MODE-2587 and MODE-2611

Being a micro-release, the artifacts for this version will only be available in Maven via the JBoss or Central repositories and will not be published on the main ModeShape site

Filed under: uncategorized

ModeShape 5.2.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 5.2.0.Final.

This minor release addresses 13 bugs and 5 enhancements, most notably:

  • add minimal partition support so that in case of network partitions a cluster of ModeShape repositories won’t corrupt data
  • update CMIS to use OpenCMIS 1.0 and also support ACLs for the CMIS connector
  • store binary files in Amazon S3

You can migrate from ModeShape 3 and ModeShape 4 by following our migration guide. If you’re starting to use ModeShape for the first time, make sure you read the getting started guide.

The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: uncategorized

ModeShape 4.6.1.Final is available

The ModeShape community is proud to announce the immediately availability of ModeShape 4.6.1.Final.

This is a micro-release off the 4.x branch which addresses only one issue: MODE-2615

Being a micro-release, the artifacts for this version will only be available in Maven via the JBoss or Central repositories and will not be published on the main ModeShape site

Filed under: uncategorized

ModeShape 5’s persistence changes

Starting with ModeShape 3 in early 2012, all repository nodes were internally represented using JSON documents and stored as BSON values in Infinispan. Although we relied upon some Infinispan features, for the most part ModeShape was merely storing its data using a very basic key-value API.

As ModeShape evolved through the 3.x and 4.x versions, we started having some data persistence issues that were largely outside of our control. ModeShape could be deployed inside JBoss AS (eventually known as Wildfly), so we chose our version of Infinispan based upon the version that was shipped with JBoss AS. Unfortunately, when we found bug in Infinispan, those bugs would be fixed in releases that were not yet included in JBoss AS, meaning we couldn’t get the fixes for quite some time. Using Infinispan also made the repository configuration and internals quite complex. Plus, changes in Infinispan’s persistence stores sometimes meant that persisted data could not be read by newer versions of Infinispan.

But most importantly, in certain situations we saw data corruption render a repository’s content largely unusable. This is a complicated issue that we previously outlined in detail this forum post and this issue.

Therefore, our primary goal with ModeShape 5 was to make sure that the repository data is stored in a more durable and strongly consistent manner that avoided the aforementioned corruption issues. This meant that we had to take a more conservative approach to persistence and give up claims of high scalability and performance (which are fine with eventual consistency, but not with strong consistency which is a must-have for ModeShape).

Already having the design of storing BSON documents in a key-value store helped us a lot, since it meant we only had to come up with transactional, strongly consistent, key-value store alternatives.

ModeShape 5’s initial release  comes with three such stores out of the box.

1. RDBMS store

This was the obvious choice, since relational databases provide strong consistency guarantees with good transactional support, at least with READ_COMMITTED isolation level that ModeShape requires. Enterprise users still trust and use relational databases a great deal. Using a relational database store meant users can still cluster multiple repositories together, as long as all those repositories use the same shared database.

ModeShape 5 comes out-of-the-box with support for H2, Oracle, MySql and PostgreSQL. Repository data is persisted in the form of BLOBs using the same internal BSON format we’ve used since ModeShape 3. We’ve also designed the store in such a way so that in the future, we can add specialized storage types that take advantage of the capabilities of different databases (for example PostgresSQL’s JSONB in 9.5 and above).

Configuration is again much simpler than ModeShape 3 and 4 with an equivalent store, as can be seen from the documentation.

2. File system store

This store uses an embedded H2 database to persist information on the local disk. Internally we use its very nice MVStore API, the lower-level key-value engine used within H2’s normal relational and SQL engine. It provides good transactional support and stores/streams binary objects (like our BSON documents) with optional features like compression and encryption.

For users which don’t want to store the repository data in a RDBMS and who also aren’t interested in clustering, this should be the default go-to store in ModeShape 5.

Configuring such a store is trivial and doesn’t require any additional configuration files (see our documentation for examples)

3. Transient in-memory key-value store

This is the default store when nothing is explicitly configured, and data is only persisted in-memory and is lost as soon as the process stops. Therefore, this is not suitable for production but is a very simple and natural option for testing and exploration. Internally, it uses H2’s MVStore API without persistence.

Other key-value stores

We can add support for other key-values stores in the future, provided they:

  • are strongly consistent;
  • support ACID transactions; and
  • run on Java 8 or above

We’re also happy to hear any suggestions or to evaluate any contributions from our community members.

Performance

Our preliminary tests indicate that all the above stores perform at least as well as their previous Infinispan counterparts in local, non-clustered modes. In fact, they should perform better in write-intensive cases while probably performing slightly slower for read-intensive cases, since Infinispan always had an in-memory cache layer on top of every store.

Which should you use?

We recommend using the file store for non-clustered cases. It’s simple, fast, and doesn’t require an external process. A second option to consider is the JDBC store with an embedded H2 database.

When clustering however, the only suitable option is the relational store with a shared JDBC store. As outlined above and as mentioned in the documentation, strict serializability required by the JCR API comes at a cost: all cluster members must coordinate their operations and use a shared persistent store. To help provide this coordination and to avoid write-contention on the same nodes, ModeShape employs global cluster locking (via JGroups) to ensure nodes can only be modified by one cluster member at a time. We believe that this is only way in which we can ensure the JCR consistency  requirements when running in a cluster.

 

 

 

 

 

Filed under: features, performance, releases, repository, uncategorized,

ModeShape 3.7.1.Final is available

Yesterday we released ModeShape 3.7.0.Final, but we discovered a problem in the BOMs that could pose a problem for any application builds that rely upon some of the default dependency versions in these BOMs (see MODE-2133).

So, we fixed that and pushed out 3.7.1.Final that contains corrected BOMs. This is the only change, so please switch to 3.7.1.Final instead of 3.7.0.Final.

Filed under: uncategorized

Coming soon: manually invoke sequencers

ModeShape has had sequencers almost since the very beginning of the project. When you upload files or add other content to a repository, ModeShape sequencers can automatically process that content to extract useful information and represent it as additional content in the repository.

For example, let’s say that you are going to store (among other things) an XSD file in your repository. Sure, ModeShape stores the actual content as well as metadata about the XSD, such as the name, the MIME type, date it was uploaded, date it was saved, and any other properties your application wants to store. But what about the stuff inside the file? You know, the element declarations, attribute declarations, namespaces, complex types, simple types, etc.?

ModeShape sequencers can get at that stuff. ModeShape comes with quite a few sequencers, including one that can parse the XSD file content and build nodes for each of the element definitions, attribute definitions, complex types, simple types, groups, and all the other concepts. We call this derived content, and having that in your repository is very useful. Imagine being able to find all schemas that contain an element declaration that matches some criteria!

All your application has to do was upload the file — the rest is automatic.

Sequencing workflow

What about cases when you don’t want it to be automatic? For example, you might want to upload and sequence the file all before saving it. Or, you might want some files to be sequenced, and others not to be. Until now, doing this required some pretty complicated workarounds.

However, next month’s ModeShape 3.6 will allow you to configure sequencers that should only be able to be invoked explicitly. We call these “manual sequencers” (in contrast to the “automatic sequencers” we’ve always had), and they are exactly the same as the good ol’ automatic sequencers except for one thing: they don’t have path expressions.

If you recall, a path expression is a pattern that tells ModeShape which content should be sequenced by a particular sequencer. ModeShape watches the changes to the repository, and any time it sees a new or changed node at a path that matches a sequencer’s path expression, ModeShape will automatically run that sequencer on that new/changed node. Every automatic sequencer need at least one valid path expression. But when a sequencer is to be invoked manually, there’s no need to configure a path expression!

So configuring manual sequencer is easy enough, but how do you invoke a sequencer? Very easily.

Consider this scenario: we’ve used a session to upload a file at “/files/schemas/Customers.xsd“, and this node has a primary type of “nt:file“. (This means the file’s content is stored in the “jcr:data” property the “jcr:content” child node.) The session has not yet saved any of this information, so it is still in the session’s transient state. The repository has a manual sequencer named “XSD Sequencer”, so all we want to do is manually invoke it on our newly-uploaded file and have the generated content put directly under the “/files/schemas/Customers.xsd” node, adjacent to the “jcr:content” node. The code is simple:

// Find the file content ...
Node fileNode = session.getNode("/files/schemas/Customers.xsd");
Property content = fileNode.getProperty("jcr:content/jcr:data");

// Pick where we want the new content to be placed ...
Node output = fileNode; // could be anywhere!

// Invoke the specific sequencer ...
boolean success = session.sequence("XSD Sequencer", content, output);

The sequence(...) method returns “true” if the sequencer generated output, or “false” if the sequencer couldn’t use the input and instead did nothing. Remember that when sequence(...) does return, any generated output is only in the session’s transient state and “session.save()” must be called to persist this state.

For more detail about how to configure and use manual sequencers in ModeShape 3.6 or later, please see our documentation. Then, watch this space for news about the 3.6 release.

Filed under: uncategorized

Planning for ModeShape 4.0

That’s right – we’re thinking about ModeShape 4.0 and what it might look like. We’ve literally just created an initial plan, so it’s very early. However, please give it a read and give us your input and feedback! Below are answers to some big questions, but if you have others please contact us or add a comment to the planning document.

Why are we talking about 4.0?

There are several reasons. Firstly, we have a few deprecated methods that we’d like to remove, and we’ll almost certainly move to JCR 2.1 (aka, JSR-333) which is nearing approval. Secondly, some of the bigger changes will not be compatible with the 3.x configuration format. In particular, we’re rethinking everything about how we manage and maintain our indexes, including how they’re configured in clustered repositories.  However, please note that we do not plan to change the storage format, making it much easier to upgrade from 3.x to 4.0.

What does this mean for 3.x?

We still plan to fix issues and add minor enhancements while we work on 4.0 features, and plan to release version 3.6 in roughly 6 weeks. It’s likely that we’ll even release 3.7, since 4.0 won’t be stable enough for production use by then.

Will I be able to upgrade from 3.x to 4.0?

Yes! Since the storage format won’t change, it really becomes just a software upgrade. We expect that most users will be able to easily upgrade with no changes to their application (unless they’re using methods deprecated in JCR 2.0 or ModeShape’s public API). However, many people will likely need to change their configuration files, though we’d like to make that as easy as possible. (Ideally, you may just need to remove some parts from your 3.x configuration.)

Filed under: uncategorized

ModeShape 3.2 and JBoss EAP

Beginning with ModeShape 3.2, we now support installing ModeShape on top of an installation of JBoss EAP 6.1, and we no longer support installing on top of a JBoss AS7 installation. When we announced this change, there was some initial confusion about what this means for community users. Now that we’ve released 3.2.0.Final, I want to repeat that this is nothing but great news for all community users, including those already using JBoss AS7. Let’s look at why this is so.

If you have been following ModeShape for a while, you’ll probably remember that for well over a year we were building a kit that installed ModeShape as a subsystem into an AS7.1.1.Final installation, and that ModeShape then reused the Infinispan subsystem that was included in AS7 as well as all of the configuration and management tooling of AS7. Unfortunately, there were no subsequent releases of AS7.1, which meant that the Infinispan version remained unchanged despite the Infinispan community marching on with many more releases. Essentially, ModeShape was stuck on the same older Infinispan version that was included in AS7.1.1.Final.

Why is that? Well, the JBoss AS community project is focused on rapid development and evolution of features. Meanwhile, Red Hat takes certain community releases of AS and then spends a huge amount of time and effort to test, fix, improve, qualify it and then release it as JBoss EAP. Each EAP release is then supported for many years, making it a very stable platform to run in production.

However, things changed significantly in March 2013, when JBoss announced that it would start making available to the community all of the alpha, beta, and candidate releases for EAP, starting with JBoss EAP 6.1.0.Alpha1. This is a big deal because this makes the stable EAP releases (which are by definition more stable than the upstream community releases) available to the community!

Most ModeShape users want a stable platform in which they can run ModeShape and their services and application. And that’s EAP. So with ModeShape 3.2 we changed our kit to install into the EAP 6.1 platform. It was also a plus that the newer EAP 6.1 releases used some of the latest stable versions of Infinispan.

Now, one of the most-often asked questions centered around whether EAP 6.1.0.Alpha could be used for production. The FAQs make it clear that the answer is that, yes, EAP 6.1.0.Alpha1 (and any other released EAP Alphas) can be used without restriction (other than the LGPL). However, if you desire, you can easily upgrade to a free developer or paid production subscription to use later, even more stable releases.

This is a win-win for all involved. ModeShape gets a more up-to-date server platform while all our users get a far more stable server in which they can deploy their ModeShape-based applications and services!

So give ModeShape 3.2 a try.

Filed under: uncategorized

ModeShape Tools for Eclipse

ModeShape Tools is a set of Eclipse plugins for working with ModeShape and JCR repositories. The plugins have been available for a little while, but we wanted to make sure you knew about them.

Editor for CND files

The first plugin we’ll talk about is an Eclipse editor for the Compact Node Definition (CND) format defined in the JCR 2.0 (JSR-283) specification. Although a client application can connect to a JCR repository and programmatically register node types, it is usually easier to place the node type definitions inside a CND file and to use that file to register all the node types contained within it. ModeShape Tool’s CND editor makes it easy to edit such files.

(The JCR 2.0 API does not define how an implementation should support registering node types via CND files, but many of the implementations have a proprietary way to do it. ModeShape extends the javax.jcr.nodetype.NodeTypeManager interface with several “registerNodeTypes” methods that each take a java.io.File, a java.net.URL, or an java.io.InputStream containing CND content. We’re hoping the JSR-333 effort for JCR 2.1 adopts our approach.)

The CND editor is a 2-page Eclipse editor used for editing *.cnd files. The first page is form-based and consists of sections for managing namespace mappings, node type definitions, property definitions, and child node definitions. The second page is a readonly view of the file content.

Take a look at the CND editor with the properties and child node sections collapsed:

CndFormPage

When you select a node type, the editor shows the information about that node type on the right side. This includes the node type’s name, attributes, supertypes, and property and child node definitions. Add, change, or remove a property definition or child node definition by double-clicking the particular row in the section’s table or by using the section’s toolbar. Here’s a view of the editor with the properties and child node sections expanded:

CndFormPage2

The CND editor’s second page is a readonly source view of the *.cnd file. You can control the verbosity of the CND notation by a preference. The source page looks like this:

CndTextPage

Here’s a few other features of the CND editor:

  • automatic validation of the CND file, with errors displayed in the header section
  • copy and paste CND elements within the same editor or between CND editors
  • automatic URI completion when adding standard (built-in) namespaces
  • the node type section has a search filter to quickly find and select a node type, even when there are hundreds of node types in the file
  • the ability to show/hide property definitions and child node definitions inherited from supertypes
  • attach comments to any CND element
  • Ctrl-space completion and “look ahead” typing for available supertypes and required types.
  • built-in help

You can read more detail about the CND Editor in our online documentation.

The CND Editor can edit any CND files for ModeShape or any other JCR 2.0 repository implementation. It is also possible to just install just this plugin. So even if you’re not using ModeShape, we think you’ll still find the CND Editor very useful.

Publishing files to ModeShape

The second ModeShape Tools plugin provides a way to upload files from your Eclipse workspace to a ModeShape repository. It can publish entire projects, folders, or just selected files. You select the ModeShape server, repository, workspace, and area within the workspace where the file(s) are to be placed.

To use, simply select the files and/or directories you want to publish, and then choose “ModeShape->Publish” from the context menu. You’ll then see a dialog that confirms the details of the publishing request:

PublishWizard

Simply select “Finish” to publish the files to ModeShape.

The publishing tool can also:

  • un-publish (or delete) resources from a publish area of a ModeShape server
  • indicate which files should also be versioned (with “mix:versionable”) when they are published
  • show the output of the publishing operations in the message console view
  • show in a Servers view the repositories, workspaces, and publish areas for multiple ModeShape servers
  • always ignore certain file types from being included in the publishing operations, configured via preferences
  • built-in help

Installation

Each of these features can be installed separately or together by following these steps:

  1. Start up Eclipse 3.7 (Juno), then choose “Help > Install New Software…”
  2. Enter http://download.jboss.org/jbosstools/updates/stable/juno/soa-tooling/modeshape/ and hit “Enter”.
  3. When the site loads, select which feature(s) you want to install, or click the “Select All” button.
  4. Click “Next”, agree to the license terms, and install.

You can read more about ModeShape Tools in our documentation

Want to contribute?

These plugins are open source and are on GitHub, licensed under EPL and LGPL. We’re always looking for contributors to help out. If you would like to contribute, find a bug, want to suggest a new feature, or have any questions, please let us know in our discussion forum and we’ll get you started. The JIRA for these plugins is here.

Filed under: features, news, open source, releases, tools, uncategorized

ModeShape is

a lightweight, fast, pluggable, open-source JCR repository that federates and unifies content from multiple systems, including files systems, databases, data grids, other repositories, etc.

Use the JCR API to access the information you already have, or use it like a conventional JCR system (just with more ways to persist your content).

ModeShape used to be 'JBoss DNA'. It's the same project, same community, same license, and same software.

ModeShape

Topics