Meanwhile perhaps you can remind OSoft of the need to publicize the forthcoming OpenReader capabilities—that will make it easier to attract volunteers.

An “OpenReader coming” logo on the home pages of the OSoft and dotReader sites would be most appreciated.

This would help satisfy the agreement that OSoft made originally with Jon (on behalf of OpenReader) and me (on behalf of LIbraryCity).

I realize you’re dealing with technology, not promo, but the two areas interact. If Mark publicizes and talks up OpenReader and positions the current approach as a temporary fix, then it’ll be easier to attract volunteers and clear up the confusion that Tamas and others understandably have.

Thanks,
David

For our timely ability to read OpenReader, I suppose I am primarily to blame. I’ve been rather bogged down in bundling all of the code in such a way that it just works on all three operating systems, chasing various bugs, and hammering out APIs for our plugin systems while trying to keep this 10k lines of Perl reasonably approachable, well-documented, and maintaining about 75% aggregate test coverage. I also build the releases, manage the mailing lists, and pretty much everything else. But, this is open-source software with a very open plugin system, so I’ll pass the blame to “everyone that hasn’t sent me a plugin yet.”

Yes, software production is unpredictable. Note that we’re still planning to support many different formats and putting a lot of effort into the plugin system plus the extra code required to keep us from being married to any particular format.

At present, the only format supported is the (invented by the original java developers) xml of the legacy ThoutReader. This is because we have an obligation to existing users. However, note that this format is a *plugin* and is designed to be cleanly separated from the rest of the system. Also note that we’ll be first to admit that this format has some problems. We didn’t create a new dotReader format, and I’m still hoping that OpenReader can fill my wishlist for things like a reduced memory footprint, consistent user model, reliable cross-book linking, etc.

It might also be worth noting that dotReader is both a GUI program and a standalone Perl API. This means that any book plugin could be used as part of a conversion tool. So, converting existing thout books to OpenReader format has already been made easier by our orthogonal plugin system and modular code. If someone were to dig into this and write the code to make it happen, this would go a long way toward promoting OpenReader by making the singularly-important Content available in that format. After all, books are all about Content. The same goes for any other format-reading plugins we acquire in the future — an “anything to OpenReader” converter in 600 lines of Perl is a very reasonable goal.

I haven’t seen many examples of openreader format. Is there a test suite? With any sort of non-trivial extensible format, one can never prove 100% support of the specification. The only thing to do is test, test, test. Take a look at the thout format samples in http://svn.dotreader.com/svn/dotreader/trunk/test_packages/ for an example of the sort of thing required. Most of these are very small books that flex some specific aspect of the format — then there is a corresponding test program in the t/book directory which loads the book and verifies the plugin’s behavior.

To make it easier for us and others to support OpenReader, it would be extremely useful to have a similar collection of miniature books (most of mine read like “ABCDE”) which express the particulars and edge cases of the format. These will also help in nailing down the specification and serve as examples for authors and developers of content creation/conversion programs. If you send me a username and password on a secure channel (or the output of `htpasswd -nsb yourusername yourpassword` in e-mail), I’ll give you a section of our subversion repository to post the samples in.

And yes, we do have limited time. I assure you that OpenReader is still on the todo list, but I have a very long todo list.

I’ll also go on record here and elsewhere saying that if you send me working code and tests, I’ll ship it. I can’t stress enough how open this open-source project is. Join the developer’s mailing list (http://dotreader.com/mailman/listinfo), send patches, tests, documentation, plugins, examples. As long as clocks only turn clockwise, we’re going to need community to make dotReader the open standards vehicle that it is intended to be.

Thanks,
Eric Wilhelm
dotReader architect / chief developer

Robert, David says he’s inviting him to post to this blog — I haven’t seen anything about acceptance. And given David’s implicit restrictions (he writes, “I’ll encourage Bill to raise fair questions about OpenReader…” [emphasis added]), I wouldn’t hold my breath.

Ironic, that DigitalPulp Publishing was founded with the most independent of spirits… DPPstore, our eBook retailing site is exclusively for eBooks by independent authors and publishers. We are format agnostic, and hugely in favor of the development of a non-proprietary standard that will allow readers to buy eBooks with confidence knowing that their eBook will be readable regardless of their choice of reading platforms (software and hardware). David R. makes us sound like the evil empire… (I’m putting on my Darth-Vader gear now). I found myself reading this article, which expressed Rothman’s doubts of the personal integrity of Jon Noring, his ability to remain independent in light of working for and with DigitalPulp Publishing, thinking about the difference between speculative journalism, investigative journalism, and Op Ed pieces.

So many blogs are based on Op Ed… just one person blasting out their perceptions of the universe, in microscopic detail or vast sweeping statements. David R. has established Teleread with a backbone of journalistic tradition. He has focused on hardware, software, standards and other eBook issues. He has invited contributors with expertise in all of the realms of eBooks to participate in the dialogue whether or not he was in agreement with their point of view. That is why I, like many interested in eBooks, read Teleread.

I was disappointed with the tone and tenor of Rothman’s latest piece… (But admittedly, I was flattered by the notion that DPP was so powerful that anyone who joined our ranks would be swayed from all of their previously held values and beliefs). Seriously though, there is no need to air personal feelings as if they had any basis in objective facts. Also, David R. managed to muddle up the following issues:

1. Corporate interests could be in conflict with standards development.
2. Jon Noring’s affiliation with DPP could bring about a conflict with his continuing responsibility to and participation in the standards’ movement.
3. OSoft has not integrated the capacity to read the OpenReader Format under a timeline that pleases David.

On the first point, I agree completely with David. Corporate interests could be in conflict with standards development, IF:

• The corporation is invested in a competing standard
• Or, the corporation is supporting a given standard and have a vested interest in seeing it emerge (regardless of the quality of development)

DPP meets neither of these conditions. We are format agnostic in our store – we will sell eBooks in whatever format the publishers create and whatever format the consumers will buy. We are also not invested in any particular standard. We do believe that it is essential for the industry to have standards. David’s point in his article about how standards affect the consumer, “As an ordinary e-book user I badly want be able to own digital books for real and not be at the mercy of any particular company,” is a sentiment we share at DPP. We have not built a business that is dependent on the success of a particular format or an untested standard. Our business is to deliver content in the form that the consumers demand.

On the issue of Jon Noring’s honor – David Rothman has leaped overboard on this one…. Not only is Jon entirely capable of separating his vocation from his passionate avocation of standards development, it is shocking that someone so closely affiliated with him would publicly raise these kinds of doubts. In addition, as I stated previously, DigitalPulp Publishing is in favor of the emergence of a nonproprietary standard for eBook publications – leaving no moral conflict for Jon Noring to battle. In fact, standards are an essential component in realizing the full potential of the eBook market. Consumers must be able to buy an eBook reader, and know that when they download eBooks, they will be readable on their device. It’s common sense. I have seen many of the people in the eBook business engage in the folly of trying to bully consumers into a proprietary format through hardware ties or marketing schemes (such as getting a well known sales channel to exclusively sell eBooks utilizing your pet format). I believe that since there are an infinite number of solutions to the eBook puzzle, making a monopoly impossible to attain, focusing on proprietary formats only handicaps the growth of the industry. A unifying standard (which must be a non-proprietary one) will open up exponential expansion of eBook technology.

On the issue of OSoft’s timely implementation – David Rothman obviously has no idea what the process of software development entails. Timelines in software development are usually made of silly putty! The fact that OSoft has a working Beta at this date is impressive and a testament to their dedication and work ethic. OSoft’s dotReader is an open source project with SVN repositories viewable on OSoft’s Website through the Source Code Link. The development of dotReader is going to include facilitating the reading of all possible formats, including OpenReader’s own. Currently, the basic XML format (dotReader format) is just the quick and easy way to showcase the unique features of XML based formats (such as OpenReader) within the dotReader. They are NOT setting up a standard to compete with OpenReader; they have simply included a much less sophisticated XML format to get eBooks to XML quickly. We are all awaiting those creation tools that OpenReader is working on make it easier for all of us to create valid, nuanced XML-based eBooks.

The long and short – Jon Noring is a dedicated member of the standards community (a thankless job if you ask me) who is now also employed in his field of choice. OSoft’s dotReader is a well conceived and brilliantly implemented way to read eBooks (regardless of format). And DigitalPulp Publishing is a group of dedicated entrepreneurs who are grateful to count Jon Noring among our ranks, and who also fully support Mr. Noring’s continuing efforts in the standards community.

And indeed we have put a draft container spec together, but it hasn’t been published yet. (It’s on my “to do soon” list.)

The OpenReader container will be fully compatible with the recently released IDPF Container. For reasons I won’t explain here, it won’t be fully conformant, but that’s not a major issue since even Adobe is using the OCF Container in “compatible” mode for containing its Mars format (PDF recast in XML.)

It is the intent, as I presently understand it, for IDPF and the OpenDocument folk (at OASIS) to settle upon a common and more generic container format. It, in all likelihood, will be almost identical to the IDPF Container because of the significant effort the IDPF Container WG put into its container spec and in developing it for containing multiple publication types. The current IDPF Container requirement that it must contain an OEBPS Publication is completely arbitrary from a technical point-of-view.

I’ll privately email you the draft OR Container spec. That document will explain how it differs from the IDPF Container spec, but for all intents and purposes is completely compatible.

One thing required to make that promise a reality is to specify exactly how to interpret the bits you actually get when you acquire an OpenReader ebook. As you note in your comment, it’s technicaly possible to wrap up files in one format in any packaging desired, whether that’s a DRM-based package, as in the example you give, or some other packaging. But if you leave the packaging unspecified, there’s no guarantee that your promise can be upheld.

Here’s an analogy: suppose I want to somehow specify books that solve the “tower of Babel” problem for my kids: namely, ones that I’m sure they can read. (They’re beginning readers.) I might do things like certify a base vocabulary I’m sure they can understand. That’s what Dr. Seuss followed, for instance, when he wrote _The Cat in the Hat_: limited himself to 250 words chosen in advance by someone else, and wrote a wonderful rhyming book using them. But if I don’t further specify how the book is packaged, namely, rendered as marks on paper, I haven’t solved the problem I set out to address. My kids can’t read versions of _The Cat in the Hat_ that are printed in Braille, for instance, or translated into Chinese (or even Pig Latin), even if those are alternate renditions of the same original easily-read text.

So I’ve been urging the OpenReader folks for a while to make it clearer what an ebook that claims to be OpenReader-compliant should use (or not use) in the way of packaging. They haven’t done this yet, though I recognize that giving a full format or certification description takes some time. Unfortunately, I can’t find anything in the specs that gives *any* packaging format that OpenReader implementations should recognize, let alone specify what *kinds* of formats are allowed.

The link for the one example ebook given, _My Antonia_, has to tell us out of band that it’s a zip file, and it looks like we have to figure out on our own to unzip that file and look for a .orb file in the top-level unzipped directory to access the OpenReader Binder that software can then use to render the book as a whole. I can’t find anything in the actual specs that tips us off to those conventions. As a human reader, I can figure them out, but software shouldn’t be expected to guess about implicit, unstated conventions.

I do see a link given for “OpenReader Container Format Specification”, which I’m guessing will make conventions like this explicit, but unfortunately that is marked as still “in preparation” and only goes to a blank placeholder document.

I haven’t been following whatever arguments have been going on with OSoft, the OpenReader folks, and other interested parties in recent weeks. But I do know that if you want to someone to read files in a particular format, produce creation tools for it, and promote the format as an open standard, it helps to have an open and complete specification for what the format is.

It looks to me that OpenReader is not at that point yet– there are some important parts released, such as Binder and the basic content specs, but important questions like “what bits will (or might) readers get when they download a book as an OpenReader file? and how will they interpret them?” are still unanswered, at least in any public, non-proprietary specification. In the absence of some crucial parts of a format specification, there may be at least some justification for a vendor that’s dependent on a release schedule to stay in business to go their own way without waiting for those crucial details.

I don’t mean this to be an apologia for the parties other than the OpenReader group that are involved in the current dispute. I don’t know the details of what happened and why, but I hope that all of the parties can work out an arrangement that’s mutually satisfying and that upholds the principles espoused by the OpenReader group. And it seems to me that at the very least there’s some more work that can be done by the OR people that might help.

Happy holidays!
David

Standards compliance takes time.

As far as author creation tools, a few good XSL transformation scripts will probably do the trick (and maybe an online transformer). The problem is that companies are trying to come up single source solutions on servers, whereas individuals want the ability to do the creation/conversions on the desktop. Two different mindsets.

Meanwhile, to add to my comments in the above post, I would encourage members of the open source community to work with Jon in areas such as creation tools for OpenReader—and, yes, implementations independent of dotReader. If they can help OSoft make good on its OpenReader promises, then so much the better. As noted, I consider OpenReader to be both valuable and fixable. It just needs to be done right.

Thanks,
David

In case of dotReader I don’t understand the motivation however, since it’s an open source software if they want to divert from the OpenReader format why did they come up with their own format instead of tweaking the OpenReader standard?

I think what OpenReader really needs is to educate the public. What the heck is the OpenReader specification anyway? At the end it’s rendered as XHTML, OSOft is basically embedding a web browser control and adding some nice features around it, like highlighting and commenting.

I’m sorry to say it but currently the direction Adobe is taking with reflowable PDF makes more sense to me.

Re: DRM…

DRM can be iplemented around any digital content (music, photos, ebooks etc), so I don’t see why the OpenReader specification should be concerned with it. If somebody wants to package an OpenReader document into a DRM scheme and finds customers willing to accept it and pay for it… so be it.