But Google is likely to carry on its battle with the authors, photographers, and other individual copyright holders. Some authors consider the fight a matter of principle. And even if Google convinced the individual named authors to settle their lawsuit, that wouldn’t prevent other authors from filing lawsuits in the future. So Google may have little choice but to seek a ruling from the courts that its scanning project is legal under copyright’s fair use doctrine.

One of the important questions is whether the authors’ lawyers can get class-action certification so they can claim to represent the interests of all authors. New York Law School copyright scholar James Grimmelmann notes this is an important question, because if they can’t get class-action status some of the plaintiffs’ lawyers will probably decide it’s not worth their while to continue representing so few clients. This is why Google is asking for the Author’s Guild to be dismissed from the suit—it will make it easier to argue against class-action certification.

If the case proceeds at its current pace, Ars Technica suggests, we can expect a decision on the question of fair use in late 2012 or 2013.

Judge Chin had rejected the much-vaunted $125 million previous settlement back in March, feeling that it gave too much power to Google. He expressed the opinion that an opt-in system, in which authors and publishers explicitly had to allow their books to be made available, would be preferable over the opt-out system Google had previously proposed.

After the hearing, a Google spokesman said that the company was looking into “a number of options” to comply with the judge’s suggestions, and an Authors Guild lawyer said they were working toward an opt-in settlement.

If both parties have not come up with a settlement by the September deadline, the judge will set a “relatively tight schedule” to move the case forward to trial. Lest we forget, in six years of legal wrangling, the case has never managed to move past preliminary hearings and settlement talks. It’s hard to blame Judge Chin for starting to get a little impatient.

Still, it’s a little sad to see the opt-out option, which would have made orphan works available by default, go out the window. One of the parts of the original settlement that most appealed to me is that it would have meant vanished rights-holders would no longer default to being dogs in the manger.

As a private sector company, the core aim of Google is to make money. The Google Videos situation shows that in order to lower expenditure and adjust its priorities, Google was willing to delete content entrusted to it by users. Libraries have trusted Google with millions of documents: many of the books scanned by Google are not digitised or OCR-processed anywhere else and, with budgets for university libraries shrinking year after year, may not be digitised again any time in the near future. Google acted admirably by listening to users and working to save the videos but entrusting such vast cultural archives to a body that has no explicit responsibilities to protection, archiving and public cultural welfare is inherently dangerous: as the situation made clear, private sector bodies have the ability to destroy archives at a whim.

He goes on to talk about how cultural institutions and the public sector should be enabling access to digital information, bridging the digital divide, and so on, and that we should have a national digital library. While I can’t argue with that, I think it’s a little bit odd the way that he’s singling Google out.

Aside from the fact that plenty of people already didn’t trust Google with books without needing this provocation (which is why Google got sued by the Authors Guild), this really holds true for any commercial institution that has its hands on lots of user-generated content. If social-networking and blogging sites (for instance, the embattled also-ran MySpace which News Corp has just decided to try to sell to some other poor sucker) shut down, a lot of people would lose their stuff too.

And given how many recent cloud-based institutions have been failing all at once lately (speaking of losing user content, Time reports “Some Of the Data Lost in Amazon’s Cloud Outage is Gone Forever”) or getting badly hacked, it seems more and more like nothing digital is necessarily safe, Google or not.

The first question has to do with the “death of print”. O’Reilly points out that print probably won’t die, but electronic media will transform what a “book” is. He uses an example of electronic maps, such as Google Maps—no longer static things that just sit on the page, they now show you not only where to go but also how to get there, and in some cases what you can do once you are there.

So the question we need to be asking ourselves about e-books is, are there similar transformations that we can expect in what we think of as the book and it becoming electronic. That’s where the really interesting game is going to be played—in making it new. We’re already seeing this in [Rupert Murdoch’s] The Daily. It’s just too close to the current conception of a newspaper. Meanwhile, FlipBoard and news.me did something much more interesting by turning your Twitter feeds into a kind of realtime newspaper. That’s a completely different approach and angle of competition that newspapers didn’t really think about. I look at news.me instead of the New York Times in the morning. The user interface is perfect for using on a tablet, great for scanning interesting news when you want to have your cup of coffee and just see what’s happening in the world. And it’s curated out of my social network. So you know that kind of transformation is going to happen to e-books as well.

He mentions that the technical/self-help book market has changed considerably over the last few years, with reference titles waning as it became possible to find out freely online how to do a lot of the stuff that those books would have formerly covered. As for the books that do still sell, O’Reilly finds that people reading the books as PDFs on the PC are willing to pay higher prices than those who want to read them on their phones. He is still working on finding exactly the right price points.

On the second page of the interview, O’Reilly reiterates his stance on piracy. He finds it is not much of a problem, because most of the pirates wouldn’t have bought it anyway—but the wider-circulated the pirated material is, the more likely it is someone who will buy it will see it.

Let’s say my goal is to sell 10,000 copies of something. And let’s say that if by putting DRM in it I sell 10,000 copies and I make my money, and if by having no DRM 100,000 copies go into circulation and I still sell 10,000 copies. Which of those is the better outcome? I think having 100,000 in circulation and selling 10,000 is way better than having just the 10,000 that are paid for and nobody else benefits.

People who don’t pay you generally wouldn’t have paid you anyway. We’re delighted when people who can’t afford our books don’t pay us for them, if they go out and do something useful with that information.

He also points out (correctly) that DRM interferes with the user experience, and you don’t sell more copies by making it harder for people to use your product.

He also suggests that, by making it possible to read a Kindle e-book on almost every non-e-reader device you own in addition to the Kindle, Amazon is “effectively” dropping DRM—a barrier that isn’t a barrier isn’t a barrier. While I wouldn’t entirely agree with that, and I’m sure a number of TeleRead readers wouldn’t either, I can see his point.

When the interviewer presses him on the issue, asking what if Amazon really ditched DRM, making piracy easier, O’Reilly points out that there are a lot of people who can’t or won’t take free stuff but won’t think of paying a couple of bucks for a legitimate copy of it. He adds that there are things that can be done outside of DRM to reduce piracy—locking down accounts that see suspicious downloading activity, for instance—and that DRM has never itself been proven to work.

But to me the analogy is: yes, there are people who break into your house, and if you live in a really high-crime neighborhood, maybe you have bars on your windows. But if you live in an ordinary neighborhood you don’t put bars on your windows just because somebody could easily break the glass and get into your house, because guess what? Most of the time people don’t. And when they do, we send police after them to check it out. The whole model that says we must somehow lock things up so that no harm is possible permeates a lot of our psyche.

On the third page, he talks about the Google Books lawsuit, and suggests that in the future the lawsuit will be looked upon as a textbook case of stupidity—when Google offered an avenue for publishers to compete against Amazon, the publishers sued them instead of taking advantage of it.

Finally, he suggests reasons why people might still want to submit works to traditional publishers even in a self-publishing age.

There are several things that they’ll offer, and they’re at different levels of value. Publishers overestimate the value of some and underestimate the value of others. First off, they offer a marketing advantage. They don’t offer that today in e-books, but somebody will have to figure that out. Building a brand, having lesser-known authors draft better-known authors, building out a fanbase. It’s different for genre fiction versus professional books or even literary fiction. With genre fiction, the brand of the publisher really matters, and in literary fiction it doesn’t, at least not very much. It’s like being plugged into the network of people who share. One of O’Reilly’s advantages is that we have a network of thousands of user groups to whom we give free books, to whom we advertise our products, and they spread the word. If you don’t have that database it’s hard to get the attention of the market.

Not everybody will agree with O’Reilly’s points, but certainly nobody can accuse him of being a newcomer to the publishing business. He’s been around since the ‘80s, and is just about the only publisher of his era still around as an independent company.

As O’Reilly points out, the publishing industry is going to be changing a lot over the next few years. I can’t wait to see how it all turns out.

Ars Technica has the story on one of these. A group of Harvard researchers created a tool that could be used to trace the usage of words or phrases in books over the last few centuries. And what’s more, Google has made the tool publicly available via a web interface.

You can go to the site, type in words or phrases (several at a time, if you like) and trace their popularity over time and in comparison to each other. It’s a fascinating way to spend an hour or so.

The tool isn’t perfect—for one thing, it’s case-sensitive, and there’s no way to combine queries: I can see all uses of “Urban Fantasy” or “urban fantasy” on the same chart, but I can’t see a combination of the both terms into a single line. And also, it depends on the scan results from Google being reliable, which they are not always: when I query on “ebook” I get a number of erroneous results over the last few centuries including scan typos and, in some cases, Project Gutenberg e-books.

Following up a search on the “f word” brings to light another interesting shortcoming in Google’s optical character recognition. Investigating a peculiar set of peaks in its usage between about 1630 and 1810 brings search results that reveal Google has been translating the “long s” used in those days as a lower-case “f”, which leads to all sorts of amusing example sentences in the search results. (Update: Danny Sullivan of Search Engine Land discovered this, as well.)

And a search oddity that I get, in which a small number of uses are shown for 1900-1910 when I search on “cyberpunk” or “Geek Squad”, makes me wonder whether some of the metadata on their books is not as good as they think it is. (They don’t show up when I search Google Books on that time period, though.)

Still, it’s fairly interesting to look at the usage of words, including dirty ones, to see how often they have appeared in print over time. And further, it’s a great example of the kinds of uses that can come from having so much data together for the first time. With a little more refinement, this class of tool could be extremely valuable to scholarly research—as well as providing amusing ways for laypeople to pass the time.

The story idea I had was inspired in part by a number of works I’d read over the years, including half-remembered stories that just popped into my head at odd moments. I actually had to ask rec.arts.sf.written for help identifying a couple of them, since I realized I wanted to read them again. And I thought the fastest way to find them might be as e-books. Funnily enough, these two stories have ended up being a study in e-book contrasts.

One of them was a really short (only 2059 words) story by Stephen Goldin called “The Last Ghost”. It was a Nebula finalist for Best Short Story of 1971—one of the shorter pieces of SF I’d read in my youth, but one of the more (pardon the pun) haunting ones, to live in my memory all this time. It’s available in a story collection by Goldin that can be found as an e-book at the usual places for $9.99…everywhere. (Except at Sony, apparently. Amusingly enough, the E-Reads buy link for Sony links to an expired domain placeholder page.) So much for bargain-hunting!

Even Fictionwise has it. Adding insult to injury, they list a “club price” of $8.49 for it—but my club membership expired last year and can never be renewed. (But I guess they’re going to have to keep on showing the club price until multi-year memberships expire.)

I didn’t want to pay $9.99 for a whole book when I just wanted to read the one story again. While I expect the other stories are all great, I simply didn’t want them right now. But fortunately, a little more research led me to discover that Smashwords has that short story available by itself for 99 cents. That was cheap enough I could buy two copies and send one to the fellow whose shared universe my story idea was set in. (In contrast to Kobo’s and Amazon’s, Smashword’s “gifting” system is rather straightforward, and works not unlike a “coffee money” jar: they simply trust you to put money in for every copy you’re going to email to someone else.) This really helped me out, and is exemplary of what I want e-books to be.

But on the other hand, there’s a book by renowned SF editor and novelist Donald A. Wollheim, writing under the pseudonym David Grinnell, called Edge of Time. Written in 1958, about a team of scientists who manage to create their own universe in microcosm, it really captured my imagination, and has held onto it to this day. But it’s not available electronically anywhere.

Even Google Books doesn’t have it (or much else by Wollheim either), suggesting that either the libraries they’ve scanned so far haven’t had much in the way of SF, or (more likely) Wollheim’s estate or publisher requested the titles be pulled. Even though it’s written by one of the most famous SF editors, a man who helped shaped the course of modern SF and fantasy fandom to a remarkable extent (including being arguably responsible for the popularity of Lord of the Rings exploding due to his unauthorized paperback republication of the trilogy), the book isn’t available electronically—or even in a new print edition—at all.

Of course, that doesn’t mean it’s unavailable. Not as a pirated e-book (I did look, out of curiosity, but apparently it’s too obscure for any pirate scanners to bother with), but in another Internet marketing innovation: the Internet-ordered, snail-mail-shipped used paperback. I could (and in fact, just did) go into Amazon right now and order a “Used – Very Good” paperback copy for one penny plus $3.99 shipping—about the same amount as I paid for an Amazon mp3 download of Daft Punk’s new Tron: Legacy soundtrack last night. It won’t put it on my iPad, but it will at least let me read it again.

And Wollheim’s estate won’t see a penny of that…penny. Whereas if it were a $9.99 e-book on Amazon, which I would have been willing to buy, they’d have gotten $7. Like so many other orphan or backlist titles, the book is a Manx cat in a “long tail” world.

The fullest potential of e-books is to create a “celestial library” equivalent to the oft-touted “celestial jukebox” of the digital music revolution. That’s what Amazon has been trying to do with its Kindle. And when it works, it works: I was able to find a relatively obscure short story I wanted to read, at a price I found reasonable, with just a few minutes of Googling. But just as often it doesn’t, and Edge of Time is a frustrating reminder of just how far the library’s shelves are from being filled.

Unlike other stores, Google wants to offer a “read-anywhere” model that will let readers access their e-books on almost any platform with a web browser. (It’s not clear how DRM will play into this.) And independent booksellers are going to form a major part of Google’s sales network, selling e-books through their websites and taking a cut of the proceeds. It could be a way to broaden Google’s store’s reach, and also let bookstores take a cut of the money being made in the e-book world.

And Google also plans to make it easier for blogs and other Internet sites to sell books themselves.

"Google is going to turn every Internet space that talks about a book into a place where you can buy that book," says Dominique Raccah, publisher and owner of Sourcebooks Inc., an independent publisher based in Naperville, Ill. "The Google model is going to drive a lot of sales. We think they could get 20% of the e-book market very fast."

PaidContent seems to have its own sources, and is saying that Google Editions will launch with 500,000 titles—which is considerably more than the 350,000 “books, newspapers, magazines, and blogs” Amazon has to offer. However, Google’s number may count the public-domain titles it has already made available on some readers (much the way Barnes & Noble used them to inflate the count of Nook books).

In a possibly-related matter, TechCrunch reports that Google has suddenly engaged in a big push to complete its Chrome Web Store, possibly for a launch of its own by the end of the year. A number of store-related issues in the development forums for Chromium, the development version of the Chrome web browser, have suddenly started seeing significant activity, and related matters are also seeing attention.

The Chrome Web Store was supposed to launch in October, but like Google Editions, has seen some delay. It’s curious to see signs that two related Google projects are going to launch by the end of the year. Perhaps the Chrome Web Store is going to power Google Editions, or at least feature a way to buy Google Editions books?

It’s going to be interesting to see how this all plays out. As the WSJ article notes, by delaying its store this long Google has lost a lot of early-mover advantage. Many e-book consumers are already “locked in” to other preferred e-book stores, and it’s unclear how well Google’s size and reputation can overcome this disadvantage. Will Google be able to dethrone Amazon? We’ll just have to wait and see.

Reviewing my list for last year, I can’t really find a whole lot that I would change on it. Despite the onset of agency pricing, we’ve still got about a zillion DRM-free free and cheap e-books available from other sources. EPUB is still being widely adopted, even if Barnes & Noble and Apple are fragmenting the DRM on it. E-book readers are exploding more than ever, especially with the iPad and iBooks joining the fray.

My iPod Touch is gone, alas, but someday I’ll have another. Google Book Search is still up and coming. Baen’s rescued Meisha Merlin writers are cranking out even more books (including the Liaden novel fans have been awaiting for literally years, and three more following that!).

And e-reader prices are falling faster than ever, with multiple readers available this Black Friday or holiday season for under $100. Even the industry leaders are getting in on the act: Amazon is selling its G2, 3G-wireless only reader for $89 tomorrow, and Barnes & Noble has its wi-fi Nook on eBay for $99. That kind of deal has been unheard of until now.

There are still a lot of annoyances in the e-book market as a whole—agency pricing, territorial restrictions, obnoxious DRM—but we should probably pause for a moment and give thanks for how good we have it now, compared to, say, five years ago. E-books are on a lot of people’s minds now, and the more people read them the better they’ll get. Sooner or later we’re going to have that dirt-cheap “disposable” e-reader the way we now have dirt-cheap “disposable” cell phones.

Industry observers are pleased with the deal, which seems to have produced a similar result to the Google Books settlement without expensive, time-consuming litigation. However, the French Publishers Association is still skeptical, warning that Google "has never respected its commitments as regards intellectual property law".

And French Culture Minister Frédéric Mitterrand has reacted cautiously, expressing regret that Hachette acted unilaterally before the French book business as a whole had reached a consensus strategy, and saying he would continue consulting with the publishing industry toward that end.

Warning that the [agreement between Hachette and Google] must "respect the principles defined within the framework of these consultations", Mitterrand said he had told Google in the last few days that he attaches great importance to the issue and that his priority was for respect of authors’ and publishers’ copyright to be assured before the U.S. concern began working with major institutions like the French National Library. He "will remain attentive to the results of the (legal) procedures underway in the United States", where a settlement between authors and Google is awaiting approval by the courts.

The Bookseller’s FutureBooks reports on the settlement and posts the press release. The Bookseller itself has more backstory, noting that Hachette had filed an objection to the Google Books settlement with the US court in September. Google says that it does not currently plan to replicate the deal in other countries, but is “always talking to our publisher partners around the globe about possible collaborations."

In the last couple of days, Google has rolled out a new tweak called “Rich Results,” which presents one extra-large search result if Google thinks that you’re searching for a specific book title.

Rich Results is the latest in a series of smaller front-end tweaks that have been matched by backend improvements. Now, the book search algorithm takes into account more than 100 "signals," individual data categories that Google statistically integrates to rank your results. When you search for a book, Google Books doesn’t just look at word frequency or how closely your query matches the title of a book. They now take into account web search frequency, recent book sales, the number of libraries that hold the title, and how often an older book has been reprinted.

Google Books is becoming a better and better for finding the knowledge you want within the books that are available. Leaving the copyright controversy aside, there has never been a better way to search for material inside books; the old card catalog system (or even digital card catalog) just pales by comparison.

And perhaps we should leave the copyright controversy aside. At least, that’s the perspective offered by the other article that caught my eye. On The Guardian, Robert McCrum looks at the question of creating national digital libraries. He points to a recent presentation (reprinted in The New York Review of Books) by Robert Darnton, director of the Harvard University Library, which proposes the idea.

What’s remarkable about Darnton’s very short paper – a call to arms, really – is that by placing the "vexed question of copyright" in a national perspective, and by putting the idea of "cultural commons" to the service of the common good, Darnton debates an issue that usually generates heat not light in a way that sounds supremely rational. Neither Britain nor the US has plans for a national digital library but Japan, France and the Netherlands all do and, as Darnton remarks, if they "can do it, why can’t the United States?" I would add: why can’t Britain?

Darnton’s reprinted speech is worth reading in its own right. He points out that freedom of access to information was an important principle to founding fathers Jefferson (who said “Knowledge is the common property of mankind,” and also made the oft-quoted analogy of knowledge to candles) and Adams.

And unlike in the 18th century, the Internet offers the potential of enormous freedom of access to information. So, Darnton suggests, we should take advantage of that freedom and create a national electronic library. For all the copyright controversy surrounding Google, it has at least shown what is possible. If a corporation can do it, why can’t the government, or other organizations that work toward public interests?

I propose that we dismiss the notion that a National Digital Library of America is far-fetched, and that we concentrate instead on what we can learn from others about issues such as: How can we deal with the problem of copyright and of orphan books, i.e., books whose copyright holders can’t be located? How can we cope with the complexities of metadata—that is, catalog-type information necessary to locate digital texts in the ever-changing environment of cyberspace? How can we find funding and develop a business plan that will resolve the long-term difficulties of collection management and preservation?

McCrum suspects that publishers are still too much in shock from Google’s “audacious copyright snatch” to consider the idea, but points out that all that Google has really done is privatize what a national government or culture should be doing to begin with. It makes sense to me—after all, in the US we have a Library of Congress; why not an E-Library of Congress?

Mike Masnick at Techdirt posts a summary of the study, which shows that affected publishers’ profits grew faster on average in the years after the project than the years before. Publishers who did not opt out of the publishing partner agreement also saw large increases in revenues and profits.

This puts me in mind of the VCR, which thirty years ago Jack Valenti famously compared to the Boston Strangler, but subsequently formed the basis for an entirely new business sector in Hollywood that accounts for significantly more revenue than theatrical showings today. Who knows what Google Books will have made of the publishing industry thirty years from now?

Google’s counting method relies entirely on its enormous metadata collection—almost one billion records—which it winnows down by throwing out duplicates and non-book items like CDs. The result is a book count that’s arrived at by a kind of process of elimination. It’s not so much that Google starts with a fixed definition of “book” and then combs its records to identify objects with those characteristics; rather, the GBS algorithm seeks to identify everything that is clearly not a book, and to reject all those entries. It also looks for collections of records that all identify the same edition of the same book, but that are, for whatever reason (often a data entry error), listed differently in the different metadata collections that Google subscribes to.

But the problem with Google’s count, as is clear from the GBS count post itself, is that GBS’s metadata collection is a riddled with errors of every sort. Or, as linguist and GBS critic Goeff Nunberg put it last year in a blog post, Google’s metadata is “train wreck: a mish-mash wrapped in a muddle wrapped in a mess.”

And a couple of libraries are contributing scans of a few hundred older works that are still under copyright—which is what got Google in trouble. For books that are still under copyright, the library will treat a physical book and an electronic scan of that book as the same volume: it will check out one e-copy of a book for each physical version of the book it has, and while the e-copy is out the physical one cannot be checked out, or vice versa.

A checked-out e-copy will be automatically rendered inaccessible at the end of the checkout period by DRM, just as with the Overdrive and Fictionwise lending libraries.

Brewster Kahle, founder of the Internet Archive, has been very critical of Google’s Google Books project and the associated settlement. His point of view is that Google is using the courts to seize to itself the legal ability to do the sorts of things that the Internet Archive has been trying to do. It’s not surprising that Kahle would embark upon a similar plan himself.

Of course, the Authors Guild might well challenge this plan just as it challenged Google’s. Scanning books without permission is arguably against the law, and while tying e-book checkouts to owned copies of printed books is a novel idea it is unclear whether the courts would consider it a fair use. Paul Aiken of the Authors Guild said "it is not clear what the legal basis of distributing these authors’ work would be."

On the other hand, Google went from just wanting to scan books to enable searching and showing limited snippets to having a very ambitious plan to become the next big e-bookstore competing with Amazon, Google, and Apple, just by being sued by and settling with the Authors Guild. Perhaps Kahle could be hoping for the same thing.

Given the facts of mass digitization of titles, free-to-use API’s,  and social sharing of resources, the digital library branch is a reality that can be implemented.  Here’s how….

Every library needs a place to start, so our digital branch will be created on a branch of the current library web site or freely created with resources such as Google Sites or Weebly.  Using graphics from the main library site or recreating them from open-source, public domain photos and artwork, it would take only a short time to get going.

Rounding out the top three resources, we could also implement the ManyBooks catalog, Feedbooks catalog and others.  Highlighting these selections, we bring in additional illustrations and book covers through the use of the Google Book Bar and embed options from the Internet Archive.  If our digital collections have a special focus, then inserting the actual titles in our site through Google Books could help bring to attention special collections such as science fair, genealogy and/or gov. document titles.

But our library is more than historical fiction and bestsellers, we should also implement newspaper and magazine resources.  First up for this would be the Google News Archive.  While the resources are small, there are lots of ways to incorporate this into our branch.  Supplementing this, we could make available singular titles such as the Sports Illustrated Archive (you knew about this right?), People Magazine and even Time.  I didn’t mention the magazines now available on Google Books, but they certainly should be there.

The end results?  We’ve got our digital branch up and running in a matter of a few weeks.  Is this a perfect solution?  Nope…but it’s a start!  Rather than being locked into a particular vendor’s ebook implementation or ILS solution, we have an open-idea, low cost, digital library branch that serves our existing patrons and new patrons worldwide.  Our digital library costs next to nothing, uses little staff and is open 24/7.  Thoughts?  Has this already been done for your local library?  Is this a redundant idea?  Let me know in the comments below……

More resources to consider:

(SlideShare): Managing The Digital Branch

(SlideShare): Considering The Digital Library Branch

American Libraries:  Building A Digital Branch

Image Source: Tagxedo