16 November 2004
Java XPath 1.0 Engine Comparison - Compliance

To address a couple of questions that have come up: All tests used DOM since it is the most widely supported data model. The latest stable binary releases of each library were employed. At the time of testing this was:

  • Resin 3.0.9
  • Saxon 8.0.1
  • Xalan 2.6.0
  • JXpath 1.2
  • JXP 1.3.5
  • Jaxen 1.0
  • J2SE1.5.0

Compliance Notes

  • J2SE 1.5 and Xalan - Excellent compliance to spec. Oddly, perhaps a little too compliant? If I cranked the iterations up I encountered the effects of this known issue.
  • Jaxen - Overall reasonable compliance. I logged this issue w/substring calls here. Seems to be an issue navigating the ancestor-or-self axis as well; it’s picking up 6 nodes instead of 5.
  • JXP - Version 1.3.3 ( the one I started with ) had some issue navigating the following-sibling axis and some glitches in substring handling. Alexandre and I exchanged some emails and he released 1.3.4 and 1.3.5 to address these concerns. We still have to work through this name(node()) failure and see if I’m doing something wrong.
  • JXPath - I began w/version 1.1 and moved to 1.2. This addressed some issues I encountered. Still an odd problem with the //* and //*[contains(string(.),’Capulet’)] test where it seems there’s a node missing from the result sets. I need to log this in bugzilla.
  • Resin - Failed to seemingly very simple and fundamental tests //speech[speaker = ‘Rom.’] and //speech[speaker = ‘Jul.’]. An issue is logged with caucho, but it has received no attention.
  • Saxon - Some small issues with substring / floor / ceiling. Otherwise excellent compliance. I need to log these issues with the Saxon project.

Summary

  • This work turned into more a compliance test than a performance test. I began with performance questions but quickly shifted to concerns about poor compliance to the XPath 1.0 spec. Each project has it’s own way to perform regression checks, but there is little consistency across libraries. There is clearly a need here for a cross library / platform XPath compliance test to reduce the duplicate effort going on in each project and improve overall compliance levels. OASIS has some tests, but they are coupled to XSLT. This also has me curious about XPath 2.0 and XQuery compliance.
  • There’s a clear lesson as well: Beware of swapping out your Xpath engine without some serious regression tests.
← Read More