22

By Aaron S. Miller, CTO of BookGlutton, a Web-based community of readers

image Andrew Savikas over at the TOC blog has been lamenting the recent AAP letter to the IDPF, and rightly so. Publishers need to stop thinking of digital books as low-fidelity copies of print originals, and they need to stop thinking about reading as an act of solitude. Reading, whether done in solitude or with a group, always ultimately connects us to the larger world, and the ways we make that connection are changing quickly.

Savikas also points out the lack, in the ePub specifications, of frameworks for social connection, and I think this hits on a bigger issue, which is that the ePub specifications are completely lacking as a Web-based framework. In other words, they seem to have been developed, quite ironically since they are based on Web technologies, without a sense that they might one day prove very useful in the context of the Web itself. ePub, for all its merits, is not browser-friendly, search-engine friendly, or at all ready for the next wave of Web-based social utilities such as annotation, presence, instant communication and more granular linking. Why is this?

First, let me review what an ePub is. It’s essentially a directory of images and/or XHTML documents, just like a web site. The directory is called the OPS, or Open Publication Structure. Minimally, it must contain one file with an extension of ‘.opf’–referred to as the OPF file. This OPS directory is also compressed into an archive format (this is like what WinZip or Stuffit do to your folders). Thus it becomes a single file referred to in the ePub spec as the “container.” It’s what we refer to when we say “EPUB file” (the official style is all-cap).

Now that the acronyms are flying, I want to address the two big reasons that ePub is Web-challenged right now:

  1. The recommended container format, OCF
  2. The proposed way of extending the core structure, OPS “XML islands”

I’ll address these one at a time, and try to explain what I mean without getting too technical.

ePub needs a non-opaque container

On the first point, the Open Container Format that defines an “EPUB file” is a format completely incompatible with browsers, opaque to search engines, and unreasonably difficult to link into from other documents. The good news is that in the specs, it’s only a “recommended” method of containing an Open Publication Structure. The bad news is that the tools and apps coming out are just assuming that Open Publication Structures must be contained in the Open Container Format. For an example, a valid Open Publication Structure (which remember is just a directory or a folder on your desktop), can’t be opened in Digital Editions, nor can it be validated with the ePub validation tool called epubcheck. Digital Editions and epubcheck only accept zip files.

The directory that holds the Open Publication Structure is compliant with IDPF specs, it’s just not in the expected container format. It is itself a container, though, and you could drop that container onto your web browser and easily browse all of its content. In its zipped form, you couldn’t do that, but you could read it Digital Editions. So what’s wrong with the picture?

What’s wrong is OCF, the Open Container Format. I don’t think ePub books need a container format at all, they just need a container, and the perfect container is a plain old fashioned directory. An ePub directory would be named with the ‘.epub’ extension, would contain a valid Open Publication Structure, and would appear to users as a single file. The questions are: 1.) How do you transfer these structures, and 2.) how do you protect them from accidental or intentional tampering by end users?

OCF was supposed to answer these questions, but in doing so, it cut the Web out of the picture. And the main alternatives to OCF are so-called “web archive” formats, none of which work across all of the major browsers. So the consensus is starting to seem like: “we’re stuck with OCF.”

We’re not stuck with OCF

The No Container Format approach has many reference implementations. One is the concept of Bundles. I call it a concept, because the implementation of a Bundle is really just an uncompressed directory of files, often a regular directory just like any other directory, but with an extension and structure which positively identify it as a bundle. Browsers can access it, systems can index it easily and links can point into the XML structures it contains, but it appears to users as a single “package.”

The implementations are often vendor-specific, because they’re tied to operating systems. Think of Digital Editions, and how when you install it your OS suddenly puts an ePub icon on files with the ‘.epub’ extension. The methods for associating icons with file extensions are often the same methods used to identify bundles.

As far as transferring bundle containers, the process is often completely transparent to the user. Email a bundle to a publisher and it will be zipped in the transfer without either party knowing it.

A bundle container would not cut out any current developers or manufacturers, and would add the benefit of linking ePub, literally and figuratively, to the Web. It’s worth consideration, and if it’s already been considered and dismissed, at least renewed discussion

ePub needs extension mechanisms more than extensions

I don’t think the second issue can be solved without first tackling the container problem, but assuming the IDPF does propose linkable, searchable, browseable containers, extensibility needs to be tackled next. Extensibility is simply a techy term for the potential of a format to be extended. XHTML’s extensibility, for example, is represented by the ability for anyone to create and use new tags.

The other two specs of the ePub triad, OPF and OPS, are based on XHTML, but they’re not designed to leverage its extensibility. The specs don’t recommend ways to link, annotate or interleave with Web-based social utilities, but more importantly they don’t recommend how we might specify those ways either. This is a much more pressing requirement. The closest the specs get is the concept of “XML islands” which are “non-preferred vocabularies” of tags in EPUB content documents. This is a step in the right direction. It does allow other kinds of XML to be used in ePub books, as long as there’s something basic to fall back on. However, it’s also an open invitation for vendor-specific approaches, and that’s not good at all. Good frameworks are known for their freedom within limitations, not for declaring open season on corporate invention.

A model for extensibility

A great example for the IDPF to follow is the XMPP standards organization, formerly the Jabber Foundation. They have developed an XML-based “document stream” specification and an extension mechanism to go with it (XMPP and XEP). Vendors develop extensions alongside the standards body, with the goal of finalizing them as XMPP Extension Protocols (XEPs). Extended functionality can be implemented gradually, and in ways that are guaranteed to be compatible with other vendors, but it can also leverage corporate innovation.

As a case study of an extension mechanism for ePub that falls outside the spec, take Adobe’s use of XSL-FO. This is an XML-based templating language geared toward paginated publications. It’s a nice and useful extension to ePub, but it’s Adobe’s, not everyone’s. We need to extrapolate from this, imagining dozens of companies applying their own page layout extensions outside of the specs. Maybe some of those extensions become incorporated into the specs, maybe not. Either way, the purpose of the specs has been defeated, because innovation is driving the spec, rather than the other way around. EBabel has spilled into ePub. This is an outcome the IDPF should strive to avoid.

I hope the community will consider these points seriously. My aim is not to attack ePub but improve it and empower it to evolve.

Moderator: Just a reminder—the ePub logo is unofficial. Meanwhile many thanks to Aaron for his thoughtful critique. Whether you agree with him or not, it’s important that this standard benefit from intensive scrutiny and civil debate; that’s what open standards should be about. For a different perspective, see responses from Hadrien Gardeur and Jon Noring. – D.R.

 
22