ModeShape

An open-source, federated content repository

ModeShape 5’s persistence changes

Starting with ModeShape 3 in early 2012, all repository nodes were internally represented using JSON documents and stored as BSON values in Infinispan. Although we relied upon some Infinispan features, for the most part ModeShape was merely storing its data using a very basic key-value API.

As ModeShape evolved through the 3.x and 4.x versions, we started having some data persistence issues that were largely outside of our control. ModeShape could be deployed inside JBoss AS (eventually known as Wildfly), so we chose our version of Infinispan based upon the version that was shipped with JBoss AS. Unfortunately, when we found bug in Infinispan, those bugs would be fixed in releases that were not yet included in JBoss AS, meaning we couldn’t get the fixes for quite some time. Using Infinispan also made the repository configuration and internals quite complex. Plus, changes in Infinispan’s persistence stores sometimes meant that persisted data could not be read by newer versions of Infinispan.

But most importantly, in certain situations we saw data corruption render a repository’s content largely unusable. This is a complicated issue that we previously outlined in detail this forum post and this issue.

Therefore, our primary goal with ModeShape 5 was to make sure that the repository data is stored in a more durable and strongly consistent manner that avoided the aforementioned corruption issues. This meant that we had to take a more conservative approach to persistence and give up claims of high scalability and performance (which are fine with eventual consistency, but not with strong consistency which is a must-have for ModeShape).

Already having the design of storing BSON documents in a key-value store helped us a lot, since it meant we only had to come up with transactional, strongly consistent, key-value store alternatives.

ModeShape 5’s initial release  comes with three such stores out of the box.

1. RDBMS store

This was the obvious choice, since relational databases provide strong consistency guarantees with good transactional support, at least with READ_COMMITTED isolation level that ModeShape requires. Enterprise users still trust and use relational databases a great deal. Using a relational database store meant users can still cluster multiple repositories together, as long as all those repositories use the same shared database.

ModeShape 5 comes out-of-the-box with support for H2, Oracle, MySql and PostgreSQL. Repository data is persisted in the form of BLOBs using the same internal BSON format we’ve used since ModeShape 3. We’ve also designed the store in such a way so that in the future, we can add specialized storage types that take advantage of the capabilities of different databases (for example PostgresSQL’s JSONB in 9.5 and above).

Configuration is again much simpler than ModeShape 3 and 4 with an equivalent store, as can be seen from the documentation.

2. File system store

This store uses an embedded H2 database to persist information on the local disk. Internally we use its very nice MVStore API, the lower-level key-value engine used within H2’s normal relational and SQL engine. It provides good transactional support and stores/streams binary objects (like our BSON documents) with optional features like compression and encryption.

For users which don’t want to store the repository data in a RDBMS and who also aren’t interested in clustering, this should be the default go-to store in ModeShape 5.

Configuring such a store is trivial and doesn’t require any additional configuration files (see our documentation for examples)

3. Transient in-memory key-value store

This is the default store when nothing is explicitly configured, and data is only persisted in-memory and is lost as soon as the process stops. Therefore, this is not suitable for production but is a very simple and natural option for testing and exploration. Internally, it uses H2’s MVStore API without persistence.

Other key-value stores

We can add support for other key-values stores in the future, provided they:

  • are strongly consistent;
  • support ACID transactions; and
  • run on Java 8 or above

We’re also happy to hear any suggestions or to evaluate any contributions from our community members.

Performance

Our preliminary tests indicate that all the above stores perform at least as well as their previous Infinispan counterparts in local, non-clustered modes. In fact, they should perform better in write-intensive cases while probably performing slightly slower for read-intensive cases, since Infinispan always had an in-memory cache layer on top of every store.

Which should you use?

We recommend using the file store for non-clustered cases. It’s simple, fast, and doesn’t require an external process. A second option to consider is the JDBC store with an embedded H2 database.

When clustering however, the only suitable option is the relational store with a shared JDBC store. As outlined above and as mentioned in the documentation, strict serializability required by the JCR API comes at a cost: all cluster members must coordinate their operations and use a shared persistent store. To help provide this coordination and to avoid write-contention on the same nodes, ModeShape employs global cluster locking (via JGroups) to ensure nodes can only be modified by one cluster member at a time. We believe that this is only way in which we can ensure the JCR consistency  requirements when running in a cluster.

 

 

 

 

 

Filed under: features, performance, releases, repository, uncategorized,

ModeShape 5.0.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 5.0.0.Final. This is the first major release from the 5.x series, and addresses 52 bug fixes, new features, and improvements.

ModeShape 5.0 has a number of very significant changes and features, including:

  • Improved consistency – ModeShape 5 no longer uses Infinispan and provides instead its own persistence stores. We suggest reading this forum post for an in depth explanation of this decision.
  • JDK 8 support – ModeShape 5 is compiled with and runs on Java 8.
  • Wildfly 10 support – ModeShape 5 fully integrates with Wildfly 10, while still supporting Wildfly 9. Note that starting from this release ModeShape will no longer support Wildfly 8 or earlier.
  • Simplified configuration – Because ModeShape 5 no longer uses Infinispan, the additional cache configuration files that users of ModeShape 3 and ModeShape 4 are familiar with are no longer required. The entire repository configuration is self-contained in either the JSON or Wildfly files (with the exception that in certain clustering cases a separate JGroups configuration is required).
  • Storage options – ModeShape 5 provides out-of-the-box support for storing content either in-memory, relational databases (via JDBC), or the file system. Each store provides different benefits, so be sure to learn about them from our new persistence documentation.
  • Storage SPI – The new storage SPI means you always have the option of implementing additional stores, as long as they are transactional and support a key-value storage model.

You can migrate from ModeShape 3 and ModeShape 4 by following our migration guide. If you’re starting to use ModeShape for the first time, make sure you read the getting started guide.

The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

ModeShape 4.6.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 4.6.0.Final. The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

This minor release contains fixes for 13 issues and 5 enhancements, the most important of which include storing indexes in Elastic Search and compressing binaries in backups. We encourage everyone to update, but if you’re upgrading a 4.2 or earlier Wildfly installation, be sure to see this section of the documentation about the cache configuration changes made back in 4.3.0.Final. Also be sure to look at our release notes for details on other changes.

This is the last planned release of the 4.x series. Development of ModeShape 5 has been progressing steadily and has some excellent new features around better storage. In fact, it’s nearly feature complete, so look for a 5.0 release in the next few weeks.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

ModeShape 4.5.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 4.5.0.Final. The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

This minor release contains fixes for 10 bugs and 7 enhancements, the most important of which include rebuilding indexes incrementally and storing indexes in Lucene. If you’re upgrading a 4.2 or earlier Wildfly installation, be sure to see this section of the documentation about the cache configuration changes made in 4.3.0.Final. Be sure to look at our release notes for details on other changes.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

ModeShape 4.4.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 4.4.0.Final. The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

This minor release contains fixes for 20 bugs and well over a dozen enhancements, the most important of which is that ModeShape’s Wildfly kit can install on both Wildfly 8 and 9. Another significant new feature are several new built-in mixins for use on parent nodes that will have large numbers of children (>500k); see MODE-2109 and our documentation for more information. The Repository Explorer has an improved user interface and a number of bug fixes, too. If you’re upgrading a 4.2 or earlier Wildfly installation, be sure to see this section of the documentation about the cache configuration changes made in 4.3.0.Final. Be sure to look at our release notes for details on other changes.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

ModeShape 4.3.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 4.3.0.Final. The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

This minor release contains fixes for 18 bugs and 11 enhancements, the most important of which is moving from Infinispan 6.0.2 to Infinispan 7.2.0.Final to fix MODE-2280. This is a big jump with several other fixes and improvements, but it does mean that our Wildfly integration kit had to be changed to use Infinispan in library mode rather than via the Wildfly subsystem (which is still 6.0.2). So if you’re using the Wildfly kit, when moving to ModeShape 4.3.0.Final you will have to update your cache configuration as described in this section of the documentation.

In addition to the Infinispan related changes, we’ve enhanced the Backup & Restore API allowing a more fine-grained control over what parts of a repository are backed up and then restored. We’ve also added the ability to perform a full repository backup & restore via the REST Service. Be sure to look at our release notes for details on other changes.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

ModeShape 4.2.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 4.2.0.Final. The JARs and other artifacts are available in the JBoss Maven repository and in our downloads area, and will be soon in Maven Central. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

This minor release contains fixes for 36 issues and 9 enhancements, including several related to our Teiid and DDL sequencers, as well as support for Wildfly 8.2. Our Wildfly kit now integrates directly with Wildfly’s security services meaning that if a security domain is configured to use caching, ModeShape will take full advantage of this whenever session authentication operations are performed. Also, the local index provider now exposes several MapDB configuration options allowing for advanced performance tuning when this provider is used.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

Introduction to ModeShape 4 on WildFly

Francesco Marchioni has written two articles that show just how easy it is to get started with ModeShape 4 on WildFly and to use JavaEE to very quickly create a simple web application to create and access content.

The first article, “NoSQL Data storage with ModeShape 4“, shows exactly how to install ModeShape 4 into an existing (or new) WildFly 8 server.

The second article, “ModeShape 4 in action“, walks through deploying a simple Web application to insert new nodes into Modeshape JCR repository and display them with a nice Primefaces tree view.

Thanks, Francesco, for providing these excellent and very useful articles!

Filed under: features, open source, techniques

ModeShape 4.1.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 4.1.0.Final. The JARs and other artifacts are available in the JBoss Maven repository and in our downloads area, and will be soon in Maven Central. See our Getting Started guide for details.

Thanks to our whole community for the work that’s gone into this release! This minor release contains fixes for 36 issues and 6 enhancements, including several related to the new indexing & query functionality and some significant ones around the Repository Explorer application. Our Teiid VDB sequencer now supports dynamic VDB parsing and Text Extraction uses Apache Tika 1.6.

Give it a try and let us know what you think!

Filed under: features, jcr, news, releases

ModeShape 3.8.1.Final is available

ModeShape 3.8.1.Final is now available, with almost four dozen bug fixes. This release depends upon Infinispan 5.2.10, and the subsystem can be installed into EAP 6.3 Beta. See the release notes for details.

As usual, the artifacts are in the JBoss Maven repository and will soon be pushed into Maven Central. Or you can download a zip file with the libraries.

Give it a whirl and let us know on IRC or in our forums if you have any problems.

Last planned community 3.x release

Please note that this is the last planned community release of ModeShape 3.x. The community has already switched its focus to the 4.x stream: we’ve already released 4.0.0.Final and are hard at work on 4.1. We encourage any 3.x users to give 4.x a try, which contains all of the fixes in 3.8.1.Final and lots of new features and improvements.

JBoss Data Virtualization 6.1

However, if you are not able to move to 4.x but are looking for professional support, please take a look at Red Hat’s JBoss Data Virtualization platform version 6.1, which is nearing release and will include a completely support version of ModeShape that is based upon 3.8.1.Final. Contact Red Hat sales for more information.

 

Filed under: jcr, news, releases

ModeShape is

a lightweight, fast, pluggable, open-source JCR repository that federates and unifies content from multiple systems, including files systems, databases, data grids, other repositories, etc.

Use the JCR API to access the information you already have, or use it like a conventional JCR system (just with more ways to persist your content).

ModeShape used to be 'JBoss DNA'. It's the same project, same community, same license, and same software.

ModeShape

Topics

Follow

Get every new post delivered to your Inbox.