Label of Perfect 15126-BWhen I was in college I collected 78 RPM phonograph records, primarily jazz records from the late 1920’s and early 1930’s. Either I was good at collecting, or just lucky. I found and acquired several large jazz and blues collections (a total of over 100,000 records, about 25 tons, passed through my fingers), and didn’t lose a dime in the process.

I’ve long since given up massively collecting the “old 78’s”, and today have only kept a few favorites. One favorite I kept, a quite rare classic jazz recording from late 1928, is shown to the right. [note 1] My experience collecting older sound recordings has given me some unique perspectives as it relates to media, e-books, copyright, conversion, archiving, formats, etc.

VHS logoIn a seemingly unrelated story (yes, I will tie all this together to e-books!), last weekend I was looking at my family’s fairly large VHS movie collection, wondering what to do with it. I’ll probably just give it away since who will buy it? I reflected that a lot of money was spent over the years to acquire these analog video tapes, and now they sit on the shelf, gathering dust, taking up space, never to be viewed again by my family.

Nor will I digitize these video tapes (because of the time and cost to do it right, and because they are of limited NTSC-grade fidelity) since we have moved on to DVD and probably soon will switch to high definition. So instead of converting these analog VHS movies, we’ll probably spend more $$$ to simply replace some of them with high definition digital versions – essentially buying the same movies twice.

Sound familiar?

Tie to e-books: Obsolescence

What do these two small tidbits in my life have to do with e-books and digital publishing? And what about ePub? Lots, really.

The common thread that runs through my two stories is media obsolescence:

  1. Obsolescence (as it relates to difficulty of playback and transfer) of the artifact containing the audio/video recordings, and

  2. Obsolescence (lower fidelity) of the underlying audio/video information.

With e-books as digital files which we can download (or can trivially retrieve from CD-ROM and DVD-ROM), we solve the type 1 obsolescence since we don’t have to carry the digital bits on any particular, difficult-to-transfer artifact. Transferring them with “no loss” is as easy as typing “copy” – no need to spend hundreds or thousands on special equipment to “transfer” the publication.

The second type of obsolescence, though, requires a little more discussion, delving into some of the themes I wrote about in a 2003 TeleRead article: OEBPS: The Universal Consumer eBook Format?

“Flexibility” and “High-Fidelity” are important as affirmed by AAP

AAP LogoVery recently the AAP, whose membership includes all the major American publishers, released an open letter with a strong endorsement of IDPF’s ePub. The following two points in AAP’s letter are germane to this article:

  1. AAP sees retailers selling ePub directly to consumers (refer to AAP’s clarification on this point), as well as selling derivative formats converted from ePub. Publishers understand the great flexibility that ePub provides.

  2. AAP uses the phrase “high-fidelity” to describe ePub. This mention means presentation quality is important to AAP, and thus should be important to everyone else in the e-book industry. It also acknowledges that indeed ePub is “high-fidelity.”

It is clear that publishers consider “flexibility” and “high-fidelity” in e-book formats important, for themselves, for the rest of the industry, and for consumers. And ePub is a format that meets these requirements.

The case for ePub as a consumer format, and its similarity to web content

ePub unofficial logoA couple e-book commentators have slammed using ePub as a consumer format, arguing that it is not “digested” enough (my phrase) to be read on limited resource handheld devices (maybe “dumbed down” is a better phrase?) In their view ePub should only be used as a “master” that the consumer never buys nor handles, and from which the publisher/wholesaler/retailer converts to proprietary formats.

What they really are advocating is that the consumer should only purchase a digested “low fidelity” format optimized for their particular handheld device, a device that is most likely here today and gone tomorrow. (Just like low fidelity 78 RPM records and VHS were “digested” and tied to their particular playback devices – where are they today?) If the consumer gets a different handheld device, they will need to buy the book again in a different, proprietary format. [note 2]

As just noted, AAP itself rejects this limited view of ePub and is supportive of repurposeable, high-fidelity ePub being sold directly to, and read by, consumers.

After all, there now exist ePub reading systems (for example, Adobe’s Digital Editions), and since the specs underlying ePub are completely open and unencumbered, anyone can build their own ePub reading system (or “user agent”.) End-user flexibility in using the e-book is important to AAP.

(Not to mention the possibility with DRM-free ePub that end-users could use tools to convert ePub to some other temporary format as they need, whenever they need it. This possibility has not been discussed.)

Furthermore, today’s “limited hardware” is actually quite close to being able to directly render ePub (filtered to lower fidelity as needed by the device) since ePub is essentially close to a packaged web site. The hard work in rendering ePub is parsing XHTML, decorating the “parsed tree” with CSS (techies here know what I mean by this) and splashing the character glyphs on the screen, which is exactly what web browsers have to do.

Many handheld devices today include web browsers. In essence, when web browsers become ubiquitous in most handhelds, then directly rendering ePub on all these handhelds will be very easy to support. That is, if the handheld can display web pages, it has the requisite horsepower to natively display ePub.

With AAP’s strong endorsement, and the release of thousands of books in the ePub format in the near future, I see more interest in modifying web browsers (and it won’t take much) to directly read ePub, as well as specialized ePub readers. Let a thousand flowers bloom!

DRM: The monkey wrench in the gears

Picture of a monkey wrenchObviously, DRM throws a huge monkey wrench into the end-user convertibility of DRM-protected ePub. But then DRM throws a monkey wrench into all format discussions, so what’s new?

It is hard to predict if the e-book industry, as it matures, will fully embrace DRM encryption, or will decide to forego using it (or only use what Bill McCoy at Adobe calls “social DRM”, where the book copy is unencrypted but identifies the individual buyer.) It does appear the music industry may drop heavy-duty DRM. I hope the major publishers will do likewise.

As I wrote about a while back, I believe DRM encryption is not needed by publishers to maximize their profits and maintain sufficient control over the use of their content. “Social DRM”, combined with digital signatures, is probably more than adequate protection for published e-books and other types of digital publications.

Summary

I see ePub (and its compatible successors) being the reflowable, high-fidelity e-book format that consumers need to only buy once, and have confidence the format will not become obsolete anytime soon. And ePub is designed for convertibility – when a new, high-fidelity, open format replaces ePub in the future, conversion of ePub to that new format, without loss of information, will be possible.

Even if today not all handhelds have the required horsepower to directly render ePub, they soon will, mainly because there’s strong consumer demand for handheld devices to include web browsing capability (even if rudimentary.) ePub is essentially close to a “web site” packaged in an ordinary ZIP file, an important fact which a lot of people strangely seem to miss.

If I have the choice of buying a digested, low fidelity, proprietary e-book format which can only be read on a particular device, or a high-fidelity, open, repurposeable format that can be read on all devices (both natively and converted as needed to other formats), now and into the future, which would I choose? Which would you choose?

I do not want to buy the same book over and over again in order to read it in some needed proprietary format, and to always be stuck reading my e-books in “low fidelity” whenever I have a device capable of higher-quality presentation.


Referenced Notes

Note 1

For those few interested in this recording, “Bugle Call Rag” by the Whoopee Makers, it was recorded 23 November 1928, right at the end of the “Roaring 20’s”. The Whoopee Makers was a pseudonym for a studio recording group whose musicians included a very young Benny Goodman (clarinet), and Jack Teagarden (trombone) – both legends in the history of jazz. It is considered a classic “novelty jazz” style (which BG and JT really didn’t like to play.) Despite its cornball style, this recording includes a classic Jack Teagarden trombone solo improvisation.

My copy of this record is near mint, and in general this record is as rare as hen’s teeth, so my copy might be the best shellac copy in the world. I don’t believe the original master survives today (but not certain of this) and if so a lower noise vinyl test pressing is not possible for Sony BMG to produce (Sony BMG owns the copyright to this recording, and a lot of original masters from that era.) Because this recording was poorly made even for its time, and pressed on poor-quality shellac, there is a lot of background noise and it is a challenge to transfer and restore to something listenable.

For your interest, here’s a Fair Use “last minute” excerpt of my copy of this recording, which starts off with a classic Jack Teagarden trombone solo. This transfer was done by a friend, a professional audio transfer expert,who owns state-of-the-art 78 RPM transfer equipment which probably cost him, in total, a few tens of thousands of dollars. I did some rudimentary declicking and decrackling of the raw transfer file, but did not do any filtering or dynamic noise reduction so as to maintain the “brightness” of the recording as much as possible. I know others would prefer I filter out most of the noise, but then the recording would sound a lot more muffled. Those who collect 78 records learn to tune-out the background noise and enjoy the music!

I note all of this to illustrate how difficult it is to properly transfer 78 RPM source material, and that the overall fidelity suffers due to a host of reasons, including the lower fidelity of the master recording itself. The relevance to e-books is given in the main article.

Note 2

These “pundits” also imply that ePub was developed by a few “lone wolf techies” oblivious to the publishing industry needs, and with no input at all by publishers.

The opposite is actually true, since the specs underlying ePub were designed based on an exhaustive list of requirements submitted by the publishers and other stakeholder groups in the digital publishing industry. Technical representatives of various companies and organizations of these groups continued to monitor and contribute almost on a weekly basis to the progress of the underlying specifications. At the end of the process, the publishers vetted the results and overwhelmingly voted for the specifications based on the advice of their in-house technical experts.

How do I know this? Because I was there the whole time…

Now if we look at proprietary, hardware-dependent e-book formats such as that for Amazon’s Kindle, publishers pretty much had zero input into the design of those formats. It’s no surprise that publishers have strongly endorsed the ePub format since they worked closely since the first OEBPS 1.0 spec in 1999 to make sure that format met their requirements.

Even if some service arose which allows me to re-download my book in a different format as needed (publishers certainly will be wary of such a service for reasons I won’t delve into here), I will be wary since it requires trust that this re-download service will indefinitely stay in business. No, give me the high-fidelity format now which I can convert as my needs require, today and into the distant future.


4 COMMENTS

  1. Excellent, well written and informative post. I’d love to see more of these and a lot less “can random device x be used as an ebook reader”? And “ebook perspective on random development x” posts. I’m sorry to say I frequently skip over your articles in my reader, and it would be a real shame to miss gems like this one.

  2. Thanks for your thoughts, Jon!

    Breaking obsolescence into separate playback and fidelity components is helpful. The other point about the use of high-fidelity formats is that it lends itself to a LOCKSS* strategy that could keep individual works from becoming as scarce as your old 78 up there. If the publisher has the only high-definition copy, what happens if it gets lost, damaged, or intentionally destroyed?

    When obsolescence is brought up, I always find myself thinking about the related problem of access, in the sense of open standards vs DRM, but also in the senses of “I can afford access to this information with its associated technology, but you can’t,” and “My human-readable physical photo albums are more accessible to my two-year-old and my non-computer-using mother than any digital format could be.” So maybe there are some similar distinctions to be made in that realm….

    *Lots Of Copies Keep Stuff Safe

The TeleRead community values your civil and thoughtful comments. We use a cache, so expect a delay. Problems? E-mail newteleread@gmail.com.