OpenReader manifesto

Well, Roger, if nothing else, you’re showing that a standards process of the kind that the OpenReader Consortium envisions would have room for a wide range of viewpoints. Thanks for passing on yours.

Jon can address all of those issues in more detail if he wants. But you do seem for the most part to be covering old territory about which he’s written before.

Meanwhile I will make a point or two myself.

You offer skeptical comments about DRM and also say in a different context: “Yes, that means the reader can change or add text, put in different pictures, sample the book and put pieces into other works.”

In appropriate situations I myself would heartily approve of these capabilities you want, but I also believe in giving publishers the option not to allow them.

Let the marketplace sort this out.

My belief, however, is that without publishers being able to maintain some control, many fewer masterpieces from the 20th century would go online from legal sources.

Like you, I dislike long, long copyright terms and have written zillions of posts against them, but that’s reality today, and many publishers and writers alike will want to discourage remixing of copyrighted works.

If an open way is better, as I believe, just like you, then let the marketplace rather than the limits of technology be the determining factor.

Repeal or mitigation ot the Sonny Bono Copyright Terms Extension Act would help, too, as noted in effect. But that just isn’t around the corner. The real utopian approach isn’t in a balanced effort like OpenReader but in thinking that the big publishers will go along with your dreams. I could discuss big publishers vs. small publishers–I myself want everyone to prosper–but that’s a whole different post.

As for proprietary vs. nonproprietary DRM and exactly how the issue is addressed–well, let the standards setters handle that one. We both know that no DRM system is perfect, especially since pirates can merely scan paper books. But if DRM capability is what it takes to get the modern classics online in e-book form, then so be it. We’ll just have to agree to disagree in a friendly way on that one. In the end, the pirates scanning in the paper books may make the whole DRM debate irrelevant. That’s why I’m so eager for the OpenReader approach to help pave the way for reader implentations with strong annotative and deep-linking capabilities with or without DRM; let’s use such features to help make new business models possible for big and small publishers alike.

Simply put, OpenReader is a wondeful alternative to the Adobe/ETI-constricted approach favored by the IDPF. Remember the real goal of Adobe and ETI–not open truly functional open standards, but rather a continuation of the Tower of eBabel via proprietary forms of DRM. If for no other reason, that is why the world needs OEB work and related efforts to happen outside the IDPF.

Thanks,
David

David,

I think the quest for a shared DRM scheme is quixotic, even if I accept all your points. But say I’m wrong and there is an unbreakable, shareable DRM mechanism for e-books. I’m pointing out that there’s really nothing that would prevent its application to any format.

You could use it on an html file, on an OEB package, on the non-proprietarily-DRMed Mobipocket or eReader formats, on OpenDoc files, on FictionBook2 files, on RTF files, and so on. The two necessary pieces to end the Tower of eBabel (as you call it) are:

– shared encoding and decoding (eg, e-readers all understanding a common DRM), and

– acceptance of the same format by the various e-readers.

You are right to rail against the incompatibilities that run rampant in e-books. But the actual markup or structure of the file format is irrelevant to that discussion.

I don’t believe the right response to selfish big companies running IDPF is to start up a competing standards process. If every company who disagreed with a standard did that, we’d have chaos. At least in the e-book world the disputants are proprietary formats, not standards. That’s a mess from which we can recover.

Take the example of validation in the XML world. XML Schemas was developed to provide things (XML format, data types) that DTD’s didn’t provide. Relax NG was another XML-format validation schema that was developed to provide things (readability, better fit with text, separation of validation from default values) that XML Schemas didn’t provide. It didn’t offer its own version of datatypes, however. It wasn’t a case of “beta vs VHS.” These two are “competing” standards only in an abstract way; they fill two different needs, for different audiences, but in the same space.

In the e-book world, OpenReader should be what OEB is not. It shouldn’t be “OEB with slightly different choices.”

Roger

Hi, Roger. Actually, as Jon will tell you, we’re talking about far more than a “slightly different choice” in the end.

OpenReader is a way to start conservatively, in line with the mood of the publishing industry, and take it from there–eventually moving toward many of the very things you’re advocating.

It is a long-term approach.

OpenReader can and should happen in a careful, systematic way that accounts for such complexities as the need for accessibility, an issue with both moral and legal ramifications.

That’s what Jon is up to. He actually wants to treat publishers more responsibly and respectfully than have Adobe and ETI, which care only about their needs of the moment.

In the end, they’ll use DRM gotchas–the very stuff that Jon is trying to pre-empt via OpenReader and a comprehensive solution. Along the way, we’re hoping for an annotations standard and deep interbook linking and other goodies that can more easily happen with the OpenReader approach prevailing.

Roger, moderation could go a long way. No need to be either the Tsar or Lenin. The Kerensky parallel is hardly exact, and I’m not a student of Russian history, but I can’t resist citing him as a metaphor for my philosophy toward e-book standards.

Thanks,
David the K

Roger,

Nice essay.

I think you’re right about the DRM — at present there seems to be no cost-effective way to deploy an open and shared DRM system which will be effective against book pirates. And closed proprietary DRM systems seem to be only mildly more effective… By the way, Microsoft and Visa thefts are immediately apparent, and countable. But what’s the actual loss if someone shares a book with their sibling, and how is it detected?

Binary formats are not long-lived. There must be a way to present all of the information in a single XML (eg, in UTF-8 or UTF-16 text format) file, as does the FictionBook2 format.

How do you include pictures? By encoding them in base64. How do you make it small? By suggesting e-readers be able to read the format when placed in a zip archive.

About the zip file: I’m not a fan of encrypting content into what Michael Hart would consider a binary format. However, every digital form is binary at some level, and this objection can be taken too far. By your argument, you also have to include a copy of the DTD in the document, and a copy of the XML specification in the document (in what format?), and… Who knows who will understand OEB XML 20 years from now? Especially as (you argue elsewhere) Sophie will have long since taken over…

Perhaps the test could be: If I read this into a fairly simpleminded text editor, can I (perhaps with difficulty) make out the text of the book by looking at what the editor displays on my screen. The zip format fails miserably at this test. Any compressed-text format fails miserably at this test. XML-encrypted content is sometimes poor at this, as well, though it generally does better.

As I’ve pointed out in other forums, the MHTML specification provides a useful textual framework for aggregation of content into a single file. Embedded images and other binary content are included as-is; no need to Base64 them. When viewed in an editor, the text of a document in MHTML format is apparent. It can easily be extended to handle spines, tours, etc. It takes advantage of the international standards for email headers and content to handle character set and language issues. Email standards seem to be incredibly long-lived.

Internet Explorer already writes and reads this format for saving Web pages locally to disk. It would be interesting to see it developed as an ebook format framework.

Hi everyone,

since I don’t have much time at the moment, I’ll keep this short.

Roger you say what the e-book world needs is not a formatting standard? Well look around at the mess we’re having right now, and try to repeat that statement with a straight face.

We have pdf, mobipocket, microsoft reader and bbeb only to mention a few examples of commonly used e-book formats and all of them are closed source and each is incompatible with the others.
When I buy a book in pdf format I can’t be sure that it’ll work on my mobile because of some outlandish DRM issue, but I know that the book I was looking forward to read for a long time won’t work at all on my mobile reader, because it is in bbeb format.

If this is not a clash of formats I don’t know what is. As a user (and passionate reader I might add) there is only one solution: a revolution in terms of format, a reset, a reevaluation of everything that has appeared up until now. And if Adobe, Microsoft and Sony want to hold on to their proprietary formats, because they can’t see farther than their noses (or their wallets in many cases), then be damned to the lot of them.

When I buy a book from what soure whatsoever I WANT to be able to read it WHEN and WHERE I like. There is no whoopdidoop or chippidichap with DRM or security that can justify this kind of inconvenience for the reader. Only twenty years ago a product with restrictions like modern e-books whould never have made the shelves in the supermarket. It’s like saying: “Yes you bought the product and your money feels very good in my pocket, but I’m still going to tell you what to do with your property.” You simply can’t treat your customers like this.

This is the only reason why the OpenReader is a necessity and this alone is more than enough. Readers want to have something simple and easy to use and I bet publishers long for a true standard so they will know where to smash their heads in and really start kranking out some major publications. And from what I have seen so far OpenReader is a damned well though out format, since it fulfills everything readers, authors and publishers want, with an additional possibility for expansion in the future. This is what truly designates a format that deserves the title of “standard”.

Thanks for your interest in the practical, Paolo. Nice post!

Planet Earth has waited long enough for a decent e-book standard. OpenReader will hardly be perfect, but it’ll be a major advance over the present mess, and, yes, there will be ways for it to evolve. In fact, that last aspect is no small part of the idea!

I’m tired of outrages such as having two of my three Mobipocket -loaded machines not work with the supposedly compatible server that a local library is using. In fact, I’m downright angry.

While no DRM would simplify things vastly, it’s a fantasy to think that the big publishers will go along. And as a reader and Digital Divide-hater, not just a gadget-lover, I want a solution to reduce the complexities and help spread the e-books around to those who now lack any books.

That is the power we’ll enjoy if the OpenReader vision prevails over, “How many angels can dance on the head of a pin?” and dreams of the instant disapperance of DRM. I’d love the latter to happen, but I’m a realist.

Get used to this, folks. Work in good faith with the publishers on a compromise solution–while leveling with ’em about the most dangerous medium of all: pulped wood, so vulnerable to scanning.

The longer we delay a standard and keep the eBabel Tower up via DRM or otherwise, the more people will accustom themselves to pirated e-books rather than honest purchases.

Thanks,
David

I think that it is possible to design an “open” system for “digital rights management” for ebooks that is comparable in quality to current schemes. One idea would use public-key cryptographic techniques. The primary patents for the Rivest-Shamir-Adleman (RSA) public-key cryptosystem have elapsed, and hence RSA can be used as a basic part of the system.

A simple scheme would embed a “private key” in the ebook reader. The “public key” would be available to the ebook seller so that he or she can encode a copy of an ebook for a purchaser. The ebook device would decode the encoded book and display parts of it for the purchaser, e.g., one page at a time. However, the ebook device would refrain from outputting the unencrypted contents through a data port. Also the ebook reader would attempt to “hide” the private key. Of course, hiding the private key will be extremely difficult. All DRM schemes are vulnerable to crackers. Hiding the key in a tamper-resistant opaque hardware module would help, at least, for a while.

Authors and publishers must recognize that it will always be possible to pirate books. Even if ebooks have perfect DRM protection, piracy will still be possible. A pirate can use a digital camera to photograph the screen of an ebook device. The pages of an ebook can be displayed in sequence and a camera can take a picture of each page. The page images can be distributed on pirate networks. Also, optical-character-recognition (OCR) programs can be applied to extract text.

Apple’s proprietary system for digital rights management (DRM) is being criticized by several governments. Earlier the French government threatened action against Apple’s restrictive DRM policies. Now, a New York Times article entitled Apple faces fresh legal attacks in Europe describes the problems Apple faces in Scandinavia and the UK.

Government consumer protection agencies in Norway and Sweden want Apple to remove restrictions that prevent customers from playing music they bought through iTunes on devices made by other companies. And in Britain, one of the largest digital-music markets, the British recording industry’s trade association, known as BPI, told a Parliamentary committee on Tuesday that iTunes music should be made compatible with other portable music devices.

These rumblings point toward the desirability of a more open DRM system. If the DRM system is not “open” then it should at least be widely licensed under reasonable, inexpensive, and non-discriminatory terms.

Garson Poole writes:

I think that it is possible to design an “open” system for “digital rights management” for ebooks that is comparable in quality to current schemes. One idea would use public-key cryptographic techniques. The primary patents for the Rivest-Shamir-Adleman (RSA) public-key cryptosystem have elapsed, and hence RSA can be used as a basic part of the system.

I knew I should have expanded my DRM comment above…

Construction of an effective well-documented DRM system using today’s cryptography technology will require building a vast and expensive social and technical infrastructure that seems likely to cost many times more than any possible piracy losses to publishers. A workable public-key management system would have to be established, one that allows effective participation by technology-illiterate consumers ages six and up. Purchasers would have to register with some central agency for, in effect, the right to read a book (a recognized, published keyset which could be used for authentication and encryption). Hardware would have to be rebuilt from the ground up, in order to preserve the digital rights after decryption. All systems would have to be certified, a la HDMI/HDCP. Import and sale of non-certified systems would have to be banned. In fact, since book pirates aren’t selling hardware, continuous raids on suspect hardware-building sites would have to be sanctioned, all over the world.

Without these measures, you might be able to use DRM to stop the purchaser of an ebook from simple copyright violation — for example, sharing a book with their child (who might in fact buy their own copy) — but you won’t be able to stop pirates from ripping the content out of the book and posting it on their own. Meanwhile you’d be dramatically lowering the value of ebooks to consumers, and driving ebook vendors out of the market. People who print and sell books on paper would love you :-).

Bill,

An MHTML approach works for me — well, sort of. I would prefer to leave any non-text portions as they stand, rather than encode them as the XML standard requires. So if we’re not going to create a legal XML file, at least it’s something that’s been engineered by a standards body.

OTOH, I’m not certain whether MHTML extends to use with any arbitrary XML. If it doesn’t, are we in no-man’s land then, neither MHTML nor XML?

But, practically speaking, a single file non-transformed (zipped) file addresses a real worry. Semantically, you are right about the binary nature of everything in the computer, but you and I both expect every computer in the world to be able to read a text file in 20 years (the contrary example of EBCDIC notwithstanding). Whereas right now, I can’t even remember the name of the program that created the ARC files I compressed things into 20 years back.

David,

I don’t mind repeating that if OpenReader generates a parallel open-DRM (or shareable DRM) as part of its development, that will be good news. But in no way is such a mechanism dependent on developing a new e-book format. You may think it politically expedient to join the two or you may wish to avoid the IDPF, but an “OpenEbookDRM” technical committee could be formed at OASIS without an OpenReader, and its fruits applied to any e-book format, including OR, OEB and HTML.

In general I’m skeptical about the success of any e-book digital-restrictions-management scheme and I disparage any. But a truly successful e-book format would engender millions of titles being published or generated each year, not just the tens of thousands of commercial titles that would be concerned with the restrict-ability that DRM introduces. So while I won’t do anything to actively thwart a DRM package, it’s tangential to the larger issue to my way of thinking, even if you phrase the larger issue as one of acceptability.

— Roger Sperberg

An MHTML approach works for me — well, sort of. I would prefer to leave any non-text portions as they stand, rather than encode them as the XML standard requires. So if we’re not going to create a legal XML file, at least it’s something that’s been engineered by a standards body.

MHTML is not an XML packaging format — it uses email standards, instead. So any non-text portions of the book can in fact be included “as they stand”, rather than re-encoding them. XML sections of the book are packaged as XML, while JPEG or PNG sections (illustrations) are packaged as JPEG or PNG.

OTOH, I’m not certain whether MHTML extends to use with any arbitrary XML.

Yes, it does. You can include all sort of things in an MHTML bundle.

Thanks for clarifying this. So an MHTML bundle might not even include any HTML? That was one thing I was wondering about.