Only time will show what the future for XML in the browser holds. Right now, it seems that XML has become the unwanted child that nobody wants to play with, because it seems inept when it comes to creating flashy rotating logos.

Or maybe it is not that bad; we will have to wait and see. And in the meantime, the best we can do is to make sure that XML support in the browser evolves and catches up with today’s state of the technology to remain relevant. Browser vendors are clearly not (or so it seems) going to assist with this, so the solution is either to write browser extension plugins (which, in truth, never really worked in practice) or to use JavaScript to implement the missing functionality.

Luckily, and almost paradoxically, browser vendors have given us a helping hand where it comes to JavaScript: With the almost total war for the fastest web browser and the best level of HTML 5 support, we are witnessing tremendous improvements in the JavaScript engines in virtually all relevant web browsers of today. Not only are they able to execute JavaScript much faster than before, but they are also able to handle much larger JavaScript libraries, and even complete applications.

For the first time – and rather remarkably – it is now possible to consider JavaScript to be robust enough for such tasks as, for example, implementing a conformant XSLT 2.0 processor or…

…or an XProc implementation. For the last couple of months, I have been working (when other priorities allow) on creating a self-contained JavaScript version of Calumet, a Java XProc processor that I have been developing at EMC. In August 2010, I presented a live demo at the Balisage XML conference in Montréal and expressed my hope that future versions of Calumet will be distributed as a dual Java/JavaScript package. If you are interested in the technical details, you can find the slides from the presentation, as well as the conference paper (which, coincidentally, seems to have been one of the impulses that have convinced Michael Kay to attempt a similar thing with Saxon) here.

So, how has the JavaScript version of Calumet progressed in the last two or three months? I must say that we are still not there, but we are getting quite close. Take a look for yourself:

http://xmldemo.emc.com:8080/calumet

I still consider the JavaScript processor “alpha” quality, as there are still compatibility or performance issues with various web browsers (yes, I know that the p:validate-with-schematron example does not work in Safari/Chrome) and some features are not (or not fully) supported yet (for instance, the p:http-request implementation is really, really simple and limited). But perhaps the processor is already good enough to be released so that users can play with it, test it – and most importantly, figure out if they can do anything useful with it.

Christmas sounds like a good plan.

Besides the content (articles, videos, blogs), the real highlight for me is that EMC is going to release a whole suite of its XML tools through the site, free for unlimited developer use. And the first of the downloads is nothing less than EMC Documentum xDB, our native XML database (formerly known as X-Hive/DB).

Personally, I have been waiting for this to happen for a long time. I know that I am biased, but I really think that xDB is a killer application, with so much value and potential. In the 10 years of its existence (yes: birthday party!), xDB has grown into a really robust and mature product with excellent support for standards, and it is no wonder that it has become so successful it the commercial sphere. Sadly, it has also successfully stayed out of reach of the wider community. But this is changing now, and I am expecting lots of positive reactions to this; I, for myself, have always found xDB great to work with.

But the EMC XML Community site is not only about promoting EMC software. Its main focus is XML and XML technologies in general, and the goal is to provide a place of dialog between us – EMC’s XML developers – and the developer community, where useful information about XML (tutorials, best practices, support, etc.) can be shared and discussed.

Looking forward to your feedback, either on this blog, or better, on the EMC XML Community website!

The organizers attempted to make the event a bit special this year: they were broadcasting a live feed from the conference, to make it accessible also for people who could not participate in person. Quite a unique touch, and surely something that will attract a wider audience in the following years of the conference, should the organizers keep doing the same thing (and they should!).

Recordings of the presentations are now starting to show up on the conference website. Only slides with audio at the moment, but the organizers are planning to upload full video recordings of the speakers once these are processed and polished (…and let’s hope not – using Robin Berjon’s words – censored :-).

My presentation about XProc was on the second day of the conference (right after another XProc talk by Norm Walsh who showed some ingenious real-world pipelines – see his blog if you are interested in knowing more), and it was kind of motivating for me to see the positive reaction to EMC‘s plans to release its XML tools to the wider public.

The most frequent question after my presentation seemed to be: When? – and at that moment, the only answer I could give was: Soon, because, in all honesty, I didn’t know. Today, I think, I can reveal more: Very soon.

• XProc integrates a whole plethora of XML technologies, ranging from XPath, XSL and XSLT, to XInclude processing, schema validation and XQuery support. The good thing about XProc is that you don’t have to learn the details of the different programming APIs and models. XProc shields you from that. To me, this is the most significant benefit of using XProc – and in fact, it is the very reason why XProc exists after all. Make manipulating XML content simple, transparent, and easy to understand.
• XProc can make application development simpler, and faster. No more tedious XML programming (I guess we all know that how many times did I write this code before? feeling…), no more low-level dances around constructing/navigating/updating the DOM tree. Not any more: Here, this my XProc pipeline, run it and give me the results I want.
• XProc can make applications more reliable and less buggy (once the XProc processor get good enough, of course :). This is related to the previous point. You see, manual XML programming is potentially dangerous, especially in the hands of unexperienced developers who are not aware of all the nifty details. I have seen too many examples of badly written code for performing an XSLT transformation, or for just parsing an XML document… And I am really glad that finally there is a tool that, to put it bluntly, can shield us from crappy programming. And once there are visual tools for building XProc pipelines – and I am sure there will be some soon – we will be even safer.
• XProc is simple. Querying content using XQuery, transforming the results to XSL-FO, and generating a final PDF document has never been easier:

  <p:declare-step>
    <p:input port="source"/>
    <p:input port="parameters" kind="parameter"/>
  
    <p:xquery>
      <p:input port="query">
        <p:data href="stats.xq" content-type="application/xquery"/>
      </p:input>
    </p:xquery>
  
    <p:xslt>
      <p:input port="stylesheet">
        <p:document href="stats2fo.xsl"/>
      </p:input>
    </p:xslt>
  
    <p:xsl-formatter href="out/stats.pdf"/>
  </p:declare-step>
  

• XProc is extensible. XProc comes with a library of standard steps, but one of the core features of the language is that it allows you to declare custom steps that provide more complex (or not supported by default) functionality. You can organize your custom steps in libraries, which you can then import in your XProc pipelines in a way similar to importing stylesheets in XSLT.

  <p:pipeline>
    <p:import href="sq-library.xpl"/>
    ...
    <sq:get-stock-quote ticker="GOOG"
        xmlns:sq="http://www.foo.com/sq/ns/"/>
    ...
  </p:pipeline>
  

• …and finally, XProc is fun!

For those who don’t know what XProc, or XML Pipeline Language, is, I think it’s best to quote the specification:

[XProc is] a language for describing operations to be performed on XML documents.

An XML Pipeline specifies a sequence of operations to be performed on zero or more XML documents. Pipelines generally accept zero or more XML documents as input and produce zero or more XML documents as output. Pipelines are made up of simple steps which perform atomic operations on XML documents and constructs similar to conditionals, iteration, and exception handlers which control which steps are executed.

The specification is maintained by the XML Processing Model Working Group, which I happen to be a member of. The group is chaired by Norman Walsh, who – in parallel to his great editorial work (and I mean it, Norm; it has been a pleasure for me so far!) – is also responsible for the development of Calabash, the reference XProc implementation.

My involvement with XProc (and the WG) began in early 2008, when I started implementing an XProc processor for my employer. Since then, the specification has undergone a number of changes of varying magnitudes, so at times it was quite a challenge to keep our implementation in sync. But I endured (hey, I get paid for that!), and now I am happy to see that both Calabash and our – yet unnamed – processor are getting more mature every day.

At the moment, one of the most important tasks for the WG is to come up with a comprehensive test suite that would cover most of the language. And the sooner we are done with this the better, since the test suite not only helps to guarantee a reasonable level of interoperability among XProc implementations, but the whole process of writing test cases helps us to detect and fix defects or ambiguities in the specification until there is still time. There are quite a number of tests already, and I would say we are somewhere halfway through, so there is definitely still some work to be done… But we are making good progress, I think, and we will get there, soon.

In the meantime, feel encouraged to get in touch with us at the xproc-dev@w3.org mailing list. It is the perfect place to find out more about XProc and to ask questions. You can subscribe by sending an e-mail with the subject subscribe to xproc-dev-request@w3.org. See you there!

The presentations from the Forum are now available on-line, so if you are curious what is this weird S1000D stuff all about, you can view a PDF version of my presentation here.