ModeShape

An open-source, federated content repository

ModeShape 5.4.1.Final is available

The ModeShape community is proud to announce the immediately availability of ModeShape 5.4.1.Final.

This is a micro-release which addresses the following issues:

Enhancement

  • MODE-2682 – Enable counter on HTreeMap used by LocalEnumeratedIndex

Bug

  • MODE-2685 – ClassCastException when attempting to reoder SNS with transient changes
  • MODE-2683 – Lucene indexes do not handle UPPER and LOWER case operands together with LIKE constraints correctly
  • MODE-2681 – When ModeShape get killed the Index-Counter is ‘0’

Task

  • MODE-2684 – Remove modeshape-core’s dependency on Apache Tika

Being a micro-release, the artifacts for this version will only be available in Maven via the JBoss or Central repositories and will not be published on the main ModeShape site

Filed under: uncategorized

ModeShape 5.4.0.Final is available

What’s new

This release addresses 19 bugs and 4 enhancements, most notably:

  • adding support for pass-through configuration properties for the Hikari connection pool (see MODE-2674)
  • fixing a number of transaction and clustering related issues
  • upgrading the Wildfly support to Wildfly 10.1.0.Final
  • adding generic Amazon S3 support for binary storage

You can migrate from ModeShape 3 and ModeShape 4 by following our migration guide. If you’re starting to use ModeShape for the first time, make sure you read the getting started guide.

The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: uncategorized

ModeShape 5.3.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 5.3.0.Final.

This minor release addresses 11 bugs and 2 enhancements, most notably:

  • adding replica set support for the Mongo DB binary store to both the JSON and Wildfly configuration (see MODE-2635)
  • adding the ability to update the configuration for external sources “on the fly”, meaning that the repository does not need to be restarted each time
    an external source is added or changed (see MODE-2650)

You can migrate from ModeShape 3 and ModeShape 4 by following our migration guide. If you’re starting to use ModeShape for the first time, make sure you read the getting started guide.

The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: uncategorized

ModeShape 4.6.2.Final is available

The ModeShape community is proud to announce the immediately availability of ModeShape 4.6.2.Final.

This is a micro-release off the 4.x branch which addresses a couple of repository backup issues: MODE-2587 and MODE-2611

Being a micro-release, the artifacts for this version will only be available in Maven via the JBoss or Central repositories and will not be published on the main ModeShape site

Filed under: uncategorized

ModeShape 5.2.0.Final is available

The ModeShape community is proud to announce the immediately availability of our latest stable release, ModeShape 5.2.0.Final.

This minor release addresses 13 bugs and 5 enhancements, most notably:

  • add minimal partition support so that in case of network partitions a cluster of ModeShape repositories won’t corrupt data
  • update CMIS to use OpenCMIS 1.0 and also support ACLs for the CMIS connector
  • store binary files in Amazon S3

You can migrate from ModeShape 3 and ModeShape 4 by following our migration guide. If you’re starting to use ModeShape for the first time, make sure you read the getting started guide.

The JARs and other artifacts are already available in the JBoss Maven repository and in our downloads area, and will make it to Maven Central soon. See our Getting Started guide for details, and the release notes for specifics of the changes in this release.

Thanks to our whole community for the work that’s gone into this release! Give it a try and let us know what you think!

Filed under: uncategorized

ModeShape 4.6.1.Final is available

The ModeShape community is proud to announce the immediately availability of ModeShape 4.6.1.Final.

This is a micro-release off the 4.x branch which addresses only one issue: MODE-2615

Being a micro-release, the artifacts for this version will only be available in Maven via the JBoss or Central repositories and will not be published on the main ModeShape site

Filed under: uncategorized

ModeShape 5’s persistence changes

Starting with ModeShape 3 in early 2012, all repository nodes were internally represented using JSON documents and stored as BSON values in Infinispan. Although we relied upon some Infinispan features, for the most part ModeShape was merely storing its data using a very basic key-value API.

As ModeShape evolved through the 3.x and 4.x versions, we started having some data persistence issues that were largely outside of our control. ModeShape could be deployed inside JBoss AS (eventually known as Wildfly), so we chose our version of Infinispan based upon the version that was shipped with JBoss AS. Unfortunately, when we found bug in Infinispan, those bugs would be fixed in releases that were not yet included in JBoss AS, meaning we couldn’t get the fixes for quite some time. Using Infinispan also made the repository configuration and internals quite complex. Plus, changes in Infinispan’s persistence stores sometimes meant that persisted data could not be read by newer versions of Infinispan.

But most importantly, in certain situations we saw data corruption render a repository’s content largely unusable. This is a complicated issue that we previously outlined in detail this forum post and this issue.

Therefore, our primary goal with ModeShape 5 was to make sure that the repository data is stored in a more durable and strongly consistent manner that avoided the aforementioned corruption issues. This meant that we had to take a more conservative approach to persistence and give up claims of high scalability and performance (which are fine with eventual consistency, but not with strong consistency which is a must-have for ModeShape).

Already having the design of storing BSON documents in a key-value store helped us a lot, since it meant we only had to come up with transactional, strongly consistent, key-value store alternatives.

ModeShape 5’s initial release  comes with three such stores out of the box.

1. RDBMS store

This was the obvious choice, since relational databases provide strong consistency guarantees with good transactional support, at least with READ_COMMITTED isolation level that ModeShape requires. Enterprise users still trust and use relational databases a great deal. Using a relational database store meant users can still cluster multiple repositories together, as long as all those repositories use the same shared database.

ModeShape 5 comes out-of-the-box with support for H2, Oracle, MySql and PostgreSQL. Repository data is persisted in the form of BLOBs using the same internal BSON format we’ve used since ModeShape 3. We’ve also designed the store in such a way so that in the future, we can add specialized storage types that take advantage of the capabilities of different databases (for example PostgresSQL’s JSONB in 9.5 and above).

Configuration is again much simpler than ModeShape 3 and 4 with an equivalent store, as can be seen from the documentation.

2. File system store

This store uses an embedded H2 database to persist information on the local disk. Internally we use its very nice MVStore API, the lower-level key-value engine used within H2’s normal relational and SQL engine. It provides good transactional support and stores/streams binary objects (like our BSON documents) with optional features like compression and encryption.

For users which don’t want to store the repository data in a RDBMS and who also aren’t interested in clustering, this should be the default go-to store in ModeShape 5.

Configuring such a store is trivial and doesn’t require any additional configuration files (see our documentation for examples)

3. Transient in-memory key-value store

This is the default store when nothing is explicitly configured, and data is only persisted in-memory and is lost as soon as the process stops. Therefore, this is not suitable for production but is a very simple and natural option for testing and exploration. Internally, it uses H2’s MVStore API without persistence.

Other key-value stores

We can add support for other key-values stores in the future, provided they:

  • are strongly consistent;
  • support ACID transactions; and
  • run on Java 8 or above

We’re also happy to hear any suggestions or to evaluate any contributions from our community members.

Performance

Our preliminary tests indicate that all the above stores perform at least as well as their previous Infinispan counterparts in local, non-clustered modes. In fact, they should perform better in write-intensive cases while probably performing slightly slower for read-intensive cases, since Infinispan always had an in-memory cache layer on top of every store.

Which should you use?

We recommend using the file store for non-clustered cases. It’s simple, fast, and doesn’t require an external process. A second option to consider is the JDBC store with an embedded H2 database.

When clustering however, the only suitable option is the relational store with a shared JDBC store. As outlined above and as mentioned in the documentation, strict serializability required by the JCR API comes at a cost: all cluster members must coordinate their operations and use a shared persistent store. To help provide this coordination and to avoid write-contention on the same nodes, ModeShape employs global cluster locking (via JGroups) to ensure nodes can only be modified by one cluster member at a time. We believe that this is only way in which we can ensure the JCR consistency  requirements when running in a cluster.

 

 

 

 

 

Filed under: features, performance, releases, repository, uncategorized,

ModeShape 3.7.1.Final is available

Yesterday we released ModeShape 3.7.0.Final, but we discovered a problem in the BOMs that could pose a problem for any application builds that rely upon some of the default dependency versions in these BOMs (see MODE-2133).

So, we fixed that and pushed out 3.7.1.Final that contains corrected BOMs. This is the only change, so please switch to 3.7.1.Final instead of 3.7.0.Final.

Filed under: uncategorized

Coming soon: manually invoke sequencers

ModeShape has had sequencers almost since the very beginning of the project. When you upload files or add other content to a repository, ModeShape sequencers can automatically process that content to extract useful information and represent it as additional content in the repository.

For example, let’s say that you are going to store (among other things) an XSD file in your repository. Sure, ModeShape stores the actual content as well as metadata about the XSD, such as the name, the MIME type, date it was uploaded, date it was saved, and any other properties your application wants to store. But what about the stuff inside the file? You know, the element declarations, attribute declarations, namespaces, complex types, simple types, etc.?

ModeShape sequencers can get at that stuff. ModeShape comes with quite a few sequencers, including one that can parse the XSD file content and build nodes for each of the element definitions, attribute definitions, complex types, simple types, groups, and all the other concepts. We call this derived content, and having that in your repository is very useful. Imagine being able to find all schemas that contain an element declaration that matches some criteria!

All your application has to do was upload the file — the rest is automatic.

Sequencing workflow

What about cases when you don’t want it to be automatic? For example, you might want to upload and sequence the file all before saving it. Or, you might want some files to be sequenced, and others not to be. Until now, doing this required some pretty complicated workarounds.

However, next month’s ModeShape 3.6 will allow you to configure sequencers that should only be able to be invoked explicitly. We call these “manual sequencers” (in contrast to the “automatic sequencers” we’ve always had), and they are exactly the same as the good ol’ automatic sequencers except for one thing: they don’t have path expressions.

If you recall, a path expression is a pattern that tells ModeShape which content should be sequenced by a particular sequencer. ModeShape watches the changes to the repository, and any time it sees a new or changed node at a path that matches a sequencer’s path expression, ModeShape will automatically run that sequencer on that new/changed node. Every automatic sequencer need at least one valid path expression. But when a sequencer is to be invoked manually, there’s no need to configure a path expression!

So configuring manual sequencer is easy enough, but how do you invoke a sequencer? Very easily.

Consider this scenario: we’ve used a session to upload a file at “/files/schemas/Customers.xsd“, and this node has a primary type of “nt:file“. (This means the file’s content is stored in the “jcr:data” property the “jcr:content” child node.) The session has not yet saved any of this information, so it is still in the session’s transient state. The repository has a manual sequencer named “XSD Sequencer”, so all we want to do is manually invoke it on our newly-uploaded file and have the generated content put directly under the “/files/schemas/Customers.xsd” node, adjacent to the “jcr:content” node. The code is simple:

// Find the file content ...
Node fileNode = session.getNode("/files/schemas/Customers.xsd");
Property content = fileNode.getProperty("jcr:content/jcr:data");

// Pick where we want the new content to be placed ...
Node output = fileNode; // could be anywhere!

// Invoke the specific sequencer ...
boolean success = session.sequence("XSD Sequencer", content, output);

The sequence(...) method returns “true” if the sequencer generated output, or “false” if the sequencer couldn’t use the input and instead did nothing. Remember that when sequence(...) does return, any generated output is only in the session’s transient state and “session.save()” must be called to persist this state.

For more detail about how to configure and use manual sequencers in ModeShape 3.6 or later, please see our documentation. Then, watch this space for news about the 3.6 release.

Filed under: uncategorized

Planning for ModeShape 4.0

That’s right – we’re thinking about ModeShape 4.0 and what it might look like. We’ve literally just created an initial plan, so it’s very early. However, please give it a read and give us your input and feedback! Below are answers to some big questions, but if you have others please contact us or add a comment to the planning document.

Why are we talking about 4.0?

There are several reasons. Firstly, we have a few deprecated methods that we’d like to remove, and we’ll almost certainly move to JCR 2.1 (aka, JSR-333) which is nearing approval. Secondly, some of the bigger changes will not be compatible with the 3.x configuration format. In particular, we’re rethinking everything about how we manage and maintain our indexes, including how they’re configured in clustered repositories.  However, please note that we do not plan to change the storage format, making it much easier to upgrade from 3.x to 4.0.

What does this mean for 3.x?

We still plan to fix issues and add minor enhancements while we work on 4.0 features, and plan to release version 3.6 in roughly 6 weeks. It’s likely that we’ll even release 3.7, since 4.0 won’t be stable enough for production use by then.

Will I be able to upgrade from 3.x to 4.0?

Yes! Since the storage format won’t change, it really becomes just a software upgrade. We expect that most users will be able to easily upgrade with no changes to their application (unless they’re using methods deprecated in JCR 2.0 or ModeShape’s public API). However, many people will likely need to change their configuration files, though we’d like to make that as easy as possible. (Ideally, you may just need to remove some parts from your 3.x configuration.)

Filed under: uncategorized

ModeShape is

a lightweight, fast, pluggable, open-source JCR repository that federates and unifies content from multiple systems, including files systems, databases, data grids, other repositories, etc.

Use the JCR API to access the information you already have, or use it like a conventional JCR system (just with more ways to persist your content).

ModeShape used to be 'JBoss DNA'. It's the same project, same community, same license, and same software.

ModeShape

Topics