ModeShape 3.3.0.Final is available

We’re happy to announce that ModeShape 3.3.0.Final is now available.

This release contains most bug fixes, but does include two excellent new features that were contributed by folks from the Fedora Futures team. ModeShape now has a chaining binary store that can sit atop several other binary stores, allowing client applications to create new binary values and specify in which of the binary stores it should be stored. Our connector framework now supports generating events so that changes made by outside systems can be propagated into ModeShape. (We’ve not yet enhanced our connectors to use this new feature.)

Also, if you’re using JBoss EAP, this release works atop EAP 6.1.0.GA (rather than Beta as with 3.2). And we have new quickstarts that show how easy it is to use ModeShape when deployed this way (you can also download them from our downloads page). See this discussion post for background on what deploying on top of EAP means for community users.

The artifacts are in the JBoss Maven repository and on our downloads page. Our getting started guide has instructions, ModeShape 3 documentation and JavaDoc are available, and our code is on GitHub. Join us on our forums or IRC channel to ask questions, and please log any issues in our JIRA.

Once again, we had a lot of help this release from our community members that reported, fixed issues, and contributed fixes and features! Thanks to the whole community for a job well-done!

Filed under: features, jcr, releases, rest

Alpes JUG and Geneva JUG

While in Europe to attend and talk at the NoSQL Matters conference, I was also invited to talk about ModeShape at several Java User Groups. Fortunately, the schedules worked out to talk to two JUGs: the Alpes JUG in Grenoble, France, and the Geneva JUG in Geneva, Switzerland.

My talk was basically a more detailed introduction to ModeShape, and I covered the various high-level features (like consistency, elasticity, transactions, in-memory storage, etc.) as well as spending more time showing how to use the hierarchical nature of ModeShape.

Both groups had lots of great questions. Then, after each meeting we had a great time eating, drinking, and talking at a local restaurant. The food and drink were excellent, and that was when I got to pepper them with questions about their regions, culture, jobs, and all sorts of other things.

Merci beaucoup to all the Alpes JUG and Geneva JUG members for making the evenings very enjoyable and memorable. But I would especially like to thank Emmanuel Hugonnet and Chris Laprun in Grenoble and Xavier Bourguignon and Maxime Nowak in Geneva for inviting me, for coordinating their group’s schedules to make my visit possible, and for all their help and for answering my logistical questions.

If you ever get an opportunity to visit Grenoble or Geneva, please do! I had a great time and hope to go back as soon as I can.

Filed under: appearances, jcr, presentation

NoSQL Matters 2013

Last week I traveled to Cologne, Germany, to attend the NoSQL Matters 2013 conference. There were lots of great talks, including both beginner and intermediate talks about all things NoSQL. It’s also really good to see the audiences have more familiarity with a variety of database technologies, and are more keenly aware that they have to choose wisely from the many different kinds of NoSQL databases out there.

That’s a point we try to make here, too. Sure, there are lots of scenarios where eventually consistent key-value and document databases are a great solution, just like there are still lots of scenarios where relational database technology is still appropriate. But eventually consistent databases can require your application to manage conflicts, and this may or may not be easy. Sometimes, it’s just easier and more natural to write code that relies upon ACID transactions to guarantee strong consistency.

I gave a talk about using ModeShape as a consistent, in-memory, hierarchical database. Again, it was primarily an introductory talk that covered most of the features that distinguishes it from other database technologies, including relational, key-value, document, and even graph databases.

I also attended several really interesting talks. One of them was “Processing a trillion cells per mouse click” by Alex Hall. Alex talked about how Google needs to very quickly process vast amounts of data so that internal tools can enable Google’s employees to interactively drill down into ad click data to look for various usage patterns. The resulting solution uses a variety of interesting techniques to optimize the indexes to support these drill-down queries. You can read more about it in this paper.

Another great talk was given by Simon Willnauer, who showed how easy it is to use ElasticSearch to query and search your data, even when that data might be stored in another system. (We’re considering having ModeShape use ElasticSearch for our indexing and search uses, so this was quite timely.)

Tug Grall gave one of my other favorite talks. He’s from Cloudbase, and this talk focused on how you use Map-Reduce style JavaScript functions to build indexes from your documents. I’ve talked with Tug before, and it was great to see him again.

Kudos to the NoSQL Matters 2013 organizers for doing such a great job organizing and running the conference . The talks were quite good, it was well attended, and the meals and snacks were above par. I also have to say that Cologne is a fantastic city with incredible charm and history. What a great place for a conference!

Filed under: appearances, features, jcr, presentation

JUDCon Brazil 2013

A few weeks ago I was lucky enough to attend the first JBoss Users Conference held in Latin America. I’m not sure there’s a better place than a JUDCon where JBoss users, community members and core developers can get together and talk, work, and socialize.

JUDCon Brazil 2013 was very successful and really well attended — much like the other seven previous JUDCons. I met in person a lot of people that before this I had only known online as part of the JBoss communities or even by reputation. Plus, I was able to meet quite a few of the folks that work at Red Hat in Brazil.

I gave two talks at the conference. The first was about ModeShape 3 and was an introduction to using ModeShape as a consistent, in-memory, hierarchical database. I covered how you can make your content adhere to a schema, be completely schema-less, or something in between. I also talked about sequencing and federation, two features that are unique to ModeShape.

My second talk was an overview of the JBoss Enterprise Data Services (EDS) platform, which one of Red Hat’s supported middleware platforms that allows you to virtualize your existing databases and data services into unified virtual databases. In effect, you can turn the data you have into the information you need, in real-time. Applications see these virtual databases just as regular database, and they use JPA, JDBC, ODBC, or web services (RESTful and SOAP) to query and update the data. But as the applications interact with the virtual database, EDS transparently operates against the actual underlying (and heterogeneous) data stores and services. It really is a very cool piece of technology.

The slides for these talks are not yet online, but hopefully they’ll be available in the next few weeks. When they are, I’ll let you know.

And finally, the big news of the conference was that JBoss announced the new name for JBoss Application Servers: WildFly.

 

Filed under: appearances, features, jcr, presentation

ModeShape 3.2 and JBoss EAP

Beginning with ModeShape 3.2, we now support installing ModeShape on top of an installation of JBoss EAP 6.1, and we no longer support installing on top of a JBoss AS7 installation. When we announced this change, there was some initial confusion about what this means for community users. Now that we’ve released 3.2.0.Final, I want to repeat that this is nothing but great news for all community users, including those already using JBoss AS7. Let’s look at why this is so.

If you have been following ModeShape for a while, you’ll probably remember that for well over a year we were building a kit that installed ModeShape as a subsystem into an AS7.1.1.Final installation, and that ModeShape then reused the Infinispan subsystem that was included in AS7 as well as all of the configuration and management tooling of AS7. Unfortunately, there were no subsequent releases of AS7.1, which meant that the Infinispan version remained unchanged despite the Infinispan community marching on with many more releases. Essentially, ModeShape was stuck on the same older Infinispan version that was included in AS7.1.1.Final.

Why is that? Well, the JBoss AS community project is focused on rapid development and evolution of features. Meanwhile, Red Hat takes certain community releases of AS and then spends a huge amount of time and effort to test, fix, improve, qualify it and then release it as JBoss EAP. Each EAP release is then supported for many years, making it a very stable platform to run in production.

However, things changed significantly in March 2013, when JBoss announced that it would start making available to the community all of the alpha, beta, and candidate releases for EAP, starting with JBoss EAP 6.1.0.Alpha1. This is a big deal because this makes the stable EAP releases (which are by definition more stable than the upstream community releases) available to the community!

Most ModeShape users want a stable platform in which they can run ModeShape and their services and application. And that’s EAP. So with ModeShape 3.2 we changed our kit to install into the EAP 6.1 platform. It was also a plus that the newer EAP 6.1 releases used some of the latest stable versions of Infinispan.

Now, one of the most-often asked questions centered around whether EAP 6.1.0.Alpha could be used for production. The FAQs make it clear that the answer is that, yes, EAP 6.1.0.Alpha1 (and any other released EAP Alphas) can be used without restriction (other than the LGPL). However, if you desire, you can easily upgrade to a free developer or paid production subscription to use later, even more stable releases.

This is a win-win for all involved. ModeShape gets a more up-to-date server platform while all our users get a far more stable server in which they can deploy their ModeShape-based applications and services!

So give ModeShape 3.2 a try.

Filed under: uncategorized

ModeShape 3.2.0.Final is available

I’m happy to announce that ModeShape 3.2.0.Final is now available. This release took us a longer than we’d hoped, but we’ve fixed an incredible 120 issues, including using JBoss EAP 6.1 (instead of JBoss AS 7.1.1) and upgrading the Infinispan, JGroups, Hibernate Search, Lucene, Tika, and other third party libraries. See our  release notes for more specifics. We recommend everyone using 3.1.3 or earlier upgrade as soon as feasible.

Just to reiterate: this version of ModeShape can be installed on top of JBoss EAP 6.1. See this discussion post for background on what this means for community users.

As usual, the artifacts are in the JBoss Maven repository and on our downloads page. Our getting started guide has instructions, ModeShape 3 documentation and JavaDoc are available, and our code is on GitHub. Join us on our forums or IRC channel to ask questions, and please log any issues in our JIRA.

We had a lot more help this release from our community members that reported and fixed issues. Thanks to the whole community for a job well-done!

Filed under: features, jcr, releases, rest

ModeShape 3.1.3.Final is available

I’m happy to announce that ModeShape 3.1.3.Final is available immediately. It contains almost dozen bug fixes, including several clustering-related fixes; see our  release notes for specifics. We recommend everyone using 3.1.2 or earlier upgrade as soon as feasible.

As usual, the artifacts are in the JBoss Maven repository and on our downloads page. Our getting started guide has instructions, ModeShape 3 documentation and JavaDoc are available, and our code is on GitHub. Join us on our forums or IRC channel to ask questions, and please log any issues in our JIRA.

Thanks to the whole community for a job well-done!

Filed under: features, jcr, releases, rest

ModeShape 3.1.2.Final is available

I’m happy to announce that ModeShape 3.1.2.Final is available immediately. It contains two dozen bug fixes; see our  release notes for specifics. We recommend everyone using 3.1.1 or earlier upgrade as soon as feasible.

As usual, the artifacts are in the JBoss Maven repository and on our downloads page. Our getting started guide has instructions, ModeShape 3 documentation and JavaDoc are available, and our code is on GitHub. Join us on our forums or IRC channel to ask questions, and please log any issues in our JIRA.

Thanks to the whole community for a job well-done!

Filed under: features, jcr, releases, rest

ModeShape Tools for Eclipse

ModeShape Tools is a set of Eclipse plugins for working with ModeShape and JCR repositories. The plugins have been available for a little while, but we wanted to make sure you knew about them.

Editor for CND files

The first plugin we’ll talk about is an Eclipse editor for the Compact Node Definition (CND) format defined in the JCR 2.0 (JSR-283) specification. Although a client application can connect to a JCR repository and programmatically register node types, it is usually easier to place the node type definitions inside a CND file and to use that file to register all the node types contained within it. ModeShape Tool’s CND editor makes it easy to edit such files.

(The JCR 2.0 API does not define how an implementation should support registering node types via CND files, but many of the implementations have a proprietary way to do it. ModeShape extends the javax.jcr.nodetype.NodeTypeManager interface with several “registerNodeTypes” methods that each take a java.io.File, a java.net.URL, or an java.io.InputStream containing CND content. We’re hoping the JSR-333 effort for JCR 2.1 adopts our approach.)

The CND editor is a 2-page Eclipse editor used for editing *.cnd files. The first page is form-based and consists of sections for managing namespace mappings, node type definitions, property definitions, and child node definitions. The second page is a readonly view of the file content.

Take a look at the CND editor with the properties and child node sections collapsed:

When you select a node type, the editor shows the information about that node type on the right side. This includes the node type’s name, attributes, supertypes, and property and child node definitions. Add, change, or remove a property definition or child node definition by double-clicking the particular row in the section’s table or by using the section’s toolbar. Here’s a view of the editor with the properties and child node sections expanded:

The CND editor’s second page is a readonly source view of the *.cnd file. You can control the verbosity of the CND notation by a preference. The source page looks like this:

Here’s a few other features of the CND editor:

• automatic validation of the CND file, with errors displayed in the header section
• copy and paste CND elements within the same editor or between CND editors
• automatic URI completion when adding standard (built-in) namespaces
• the node type section has a search filter to quickly find and select a node type, even when there are hundreds of node types in the file
• the ability to show/hide property definitions and child node definitions inherited from supertypes
• attach comments to any CND element
• Ctrl-space completion and “look ahead” typing for available supertypes and required types.
• built-in help

You can read more detail about the CND Editor in our online documentation.

The CND Editor can edit any CND files for ModeShape or any other JCR 2.0 repository implementation. It is also possible to just install just this plugin. So even if you’re not using ModeShape, we think you’ll still find the CND Editor very useful.

Publishing files to ModeShape

The second ModeShape Tools plugin provides a way to upload files from your Eclipse workspace to a ModeShape repository. It can publish entire projects, folders, or just selected files. You select the ModeShape server, repository, workspace, and area within the workspace where the file(s) are to be placed.

To use, simply select the files and/or directories you want to publish, and then choose “ModeShape->Publish” from the context menu. You’ll then see a dialog that confirms the details of the publishing request:

Simply select “Finish” to publish the files to ModeShape.

The publishing tool can also:

• un-publish (or delete) resources from a publish area of a ModeShape server
• indicate which files should also be versioned (with “mix:versionable”) when they are published
• show the output of the publishing operations in the message console view
• show in a Servers view the repositories, workspaces, and publish areas for multiple ModeShape servers
• always ignore certain file types from being included in the publishing operations, configured via preferences
• built-in help

Installation

Each of these features can be installed separately or together by following these steps:

1. Start up Eclipse 3.7 (Juno), then choose “Help > Install New Software…”
2. Enter http://download.jboss.org/jbosstools/updates/stable/juno/soa-tooling/modeshape/ and hit “Enter”.
3. When the site loads, select which feature(s) you want to install, or click the “Select All” button.
4. Click “Next”, agree to the license terms, and install.

You can read more about ModeShape Tools in our documentation

Want to contribute?

These plugins are open source and are on GitHub, licensed under EPL and LGPL. We’re always looking for contributors to help out. If you would like to contribute, find a bug, want to suggest a new feature, or have any questions, please let us know in our discussion forum and we’ll get you started. The JIRA for these plugins is here.

Filed under: features, news, open source, releases, tools, uncategorized

Structured, unstructured, and everything between

Shane gives a good breakdown of the various ways to classify data as structured or unstructured. He points out that very often data is a mixture of both structured and unstructured data, and he gives several examples.

What I find so interesting about this, however, is how well ModeShape can handle these varieties of data.

ModeShape handles structured data really well. Most data structures are very easily mapped to the nodes and properties that ModeShape uses. And when those nodes also say which node types apply to them, ModeShape can enforce the node structure by validating it against those built-in and/or custom node types and prevent invalid data from being stored.

The other end of the spectrum is unstructured data, and ModeShape handles that beautifully, too. You can store unstructured data in a property using a string value or a binary value. Typically you would use a string value when the data is some form of text, and a binary value in any other cases (or when you don’t want to treat it as text).

But the best part is that ModeShape naturally handles combinations of structured and unstructured data. Recall that ModeShape is a hierarchical database, which means that each database consists of a single tree of nodes, and each node has one or more properties. That hierarchy is by definition structured, though it’s up to you whether ModeShape validates and enforces that structure using node types. But the leaves of that tree — that is the properties and their values — typically unstructured (though property value like dates and even some string values could be considered structured).

ModeShape’s query languages can also deal with both structured and unstructured data. Relationships between nodes, specific properties defined by node types, and the definitions of those properties all are addressable within the query language. But ModeShape queries can include full-text search constraints on both string and binary property values!

ModeShape can search those binary values when it can extract text using the Tika library, which supports many formats, including PDF, Microsoft Office™, RTF, HTML, and many others.

There’s one more way that ModeShape can deal with unstructured data: it can sequence unstructured data (string and binary property values) using built-in or custom sequencers to extract structure and save it as more nodes and properties in the repository. This is ideal for getting at that unstructured data that has the implicit structure defined by the format. For example, if an image is loaded into the repository, ModeShape’s image sequencer can extract the EXIF data in the image (e.g., ISO setting, focal length, aperture, shutter, geo-location, etc.) and save it as properties in the repository. ModeShape has a number of built-in sequencers that can extract this implicit structure from a variety of file formats:

• DDL files
• images (JPEG, GIF, BMP, PCX, PNG, IFF, RAS, PBM, PGM, PPM and PSD)
• audio (MP3)
• comma-separated and delimited text files
• Java source and class files
• Microsoft Office™
• ZIP archives
• XML
• XML Schema
• WSDL

In summary, ModeShape deals very naturally and easily with data that is part unstructured and part structured. What else could you want?

Filed under: features, repository, techniques