"We were excited to be involved with acquiring the Twitter archives because it’s a unique record of our time," [LoC digital initiatives program manager Bill] Lefurgy said. "It’s also a unique way of communication. It’s not so much that people are going to be interested in what you or I had for lunch, which some people like to say on Twitter."

Researchers will be able to search and data-mine the archives for interesting data. It will not include privacy-protected postings.

(Found via Slashdot.)

The last go-round resulted in six exemptions, including allowing circumvention for incorporating clips into new works for purpose of criticism or comment, including educational purposes. (Apparently the MPAA’s suggestion that professors should just point a camcorder at the screen didn’t go over very well.) It also permitted jailbreaking iPhones, and cracking e-book DRM if and only if no other method existed to use screen-reading of text-to-speech with the DRM-locked version.

Of course, all those things will have to be argued again this time around, and the MPAA and other content lobbies have historically been quite willing to step up and argue against them—and perhaps they learned something from their failures last time around.

The whole process will take several months; if last time was any example, we won’t know the results until late July—but look for reports on both sides’ antics in months to come if any prove to be worth mentioning.

If you just want to watch the section about how the library uses technology to preserve and study manuscripts, jump to 1:15:45 and watch the part about the Preservation Division.

If you’re even more impatient, jump straight to 1:20:45, which shows the “library’s top treasure” vault and some sci-fi shots of its hyperspectral imaging machine, which has revealed a thumbprint on the Gettysburg Address (they can’t determine whehter it’s Lincoln’s until the technology exists to perform an accurate DNA test), and a fairly dramatic word change in an early draft of the Declaration of Independence.

At 1:24:20, there’s a very brief section on the library’s digitization efforts, but all you’ll learn is that so far the library has about 1.25 million of its 15 million items digitized and online. The Chief of the Prints & Photographs Division says cryptically, “The hardest part of digitization is not getting carried away with it. [...] We’re using this digitizing to try to get as many pictures as possible online.” That sounds to me like she’s hinting at some criticism that’s been conveniently left out of the documentary, but I might be looking for drama where no drama exists. This is C-SPAN, after all.

View more at “C-SPAN – Library of Congress”.

“The new immigrants don’t shoot the old inhabitants when they come in. One technology tends to supplement rather than supplant. How you read is not as important as: will you read? And will you read something that’s a book–the sustained train of thought of one person speaking to another? Search techniques are embedded in e-books that invite people to dabble rather than follow a full train of thought. This is part of a general cultural problem.”

–James H. Billington, Librarian of Congress, in a Newsweek poll of “some literary brains on the future of reading.”

Mr. Billington’s grasp of American history is clearly shaky, either that or he has never heard about what the European immigrants did to the Indians in this country, or the Spanish to the Aztecs. Also, since when is searching a book a distraction? In a print book is it a bad thing to look something up in the index? Doesn’t he know what some print books have things called a glossary and – even worse – some print books have footnotes! Better take them out. Can”t have people dabbling by looking at them.

Very odd, indeed.

In the last couple of days, Google has rolled out a new tweak called “Rich Results,” which presents one extra-large search result if Google thinks that you’re searching for a specific book title.

Rich Results is the latest in a series of smaller front-end tweaks that have been matched by backend improvements. Now, the book search algorithm takes into account more than 100 "signals," individual data categories that Google statistically integrates to rank your results. When you search for a book, Google Books doesn’t just look at word frequency or how closely your query matches the title of a book. They now take into account web search frequency, recent book sales, the number of libraries that hold the title, and how often an older book has been reprinted.

Google Books is becoming a better and better for finding the knowledge you want within the books that are available. Leaving the copyright controversy aside, there has never been a better way to search for material inside books; the old card catalog system (or even digital card catalog) just pales by comparison.

And perhaps we should leave the copyright controversy aside. At least, that’s the perspective offered by the other article that caught my eye. On The Guardian, Robert McCrum looks at the question of creating national digital libraries. He points to a recent presentation (reprinted in The New York Review of Books) by Robert Darnton, director of the Harvard University Library, which proposes the idea.

What’s remarkable about Darnton’s very short paper – a call to arms, really – is that by placing the "vexed question of copyright" in a national perspective, and by putting the idea of "cultural commons" to the service of the common good, Darnton debates an issue that usually generates heat not light in a way that sounds supremely rational. Neither Britain nor the US has plans for a national digital library but Japan, France and the Netherlands all do and, as Darnton remarks, if they "can do it, why can’t the United States?" I would add: why can’t Britain?

Darnton’s reprinted speech is worth reading in its own right. He points out that freedom of access to information was an important principle to founding fathers Jefferson (who said “Knowledge is the common property of mankind,” and also made the oft-quoted analogy of knowledge to candles) and Adams.

And unlike in the 18th century, the Internet offers the potential of enormous freedom of access to information. So, Darnton suggests, we should take advantage of that freedom and create a national electronic library. For all the copyright controversy surrounding Google, it has at least shown what is possible. If a corporation can do it, why can’t the government, or other organizations that work toward public interests?

I propose that we dismiss the notion that a National Digital Library of America is far-fetched, and that we concentrate instead on what we can learn from others about issues such as: How can we deal with the problem of copyright and of orphan books, i.e., books whose copyright holders can’t be located? How can we cope with the complexities of metadata—that is, catalog-type information necessary to locate digital texts in the ever-changing environment of cyberspace? How can we find funding and develop a business plan that will resolve the long-term difficulties of collection management and preservation?

McCrum suspects that publishers are still too much in shock from Google’s “audacious copyright snatch” to consider the idea, but points out that all that Google has really done is privatize what a national government or culture should be doing to begin with. It makes sense to me—after all, in the US we have a Library of Congress; why not an E-Library of Congress?

And soon those archives will include every public tweet ever posted to Twitter since the service’s inception in 2006.

Much as with the Internet Archive’s decision to archive the web, the Library of Congress sees the move to communication on Twitter as an important sea change in the way people communicate—one that should be preserved for the study and understanding of future researchers.

For instance, "one of the things we see emerging is that Twitter is a news distribution medium now," [Martha Anderson, director of the National Digital Information Infrastructure and Preservation Program at the Library] says. Issue a press release, especially on paper, and response is minimal. But when the library tweeted about its plans, feedback was immediate and overwhelming.

Ars Technica has a piece about the Library’s plans which includes an amusing look at what we might learn about 17th century history if Shakespeare had twittered for posterity.

At present, the archives will not be offered to the public, but only made available to approved researchers.

It’s interesting and a little sobering to consider that every random tweet you ever made, every time you felt obligated to tell the world what you just had for lunch or how long you had to wait for the bus, is going to be available for the perusal of future generations.

When we think of history, we think of musty old tomes, all the classic books that are now public domain e-books. But in the grand scheme of things, little day-to-day messages that show what the average person was thinking and doing could be far more important to reconstructing pictures of our daily life.

It’s also kind of funny to think about future generations needing to build a picture of what life was like in the here and now. After all, we know what our life is like, and are recording it digitally in more ways than ever before. It’s hard to conceive of a future where all that could be forgotten.

But time marches on—and institutions like the Library of Congress exist to try to make sure as much history as possible is preserved. Even the history of our Twitter streams.

WorldCat can help and there are a few other index sites as well, but today I thought it would be a great time to introduce one of the best ways to search for titles and topics:  The Library of Congress Online Catalog.

Located at http://catalog.loc.gov, this online catalog can help you find lists of titles, topics, and quite possibly eBooks on your subject of interest.  With 14 million records as well as “romanized” information on foreign language titles you are almost guaranteed to find something.

Not just for eBooks, the catalog contains all types of items and formats, ranging from links to external web sites, PDF copies of books, historic images and regular book information on titles old and new.  Does this make it less of a resource for us eBook users?  I don’t think so, because even if you can’t find an online eBook, you can still collate together a list that you can take to other sites and libraries and hopefully find that elusive electronic version.

The catalog’s search interface is deceptively simple, yet very powerful.  Much like popular search engines, there’s a main entry box to get started with a quick search, yet don’t overlook the advanced options which include phrase searching, Boolean operators (my favorite!), and index searching among others.  Using these options, the true power of the catalog becomes apparent.  It’s not easy, especially compared to Google, but if you can get a handle on how it works, then you can have some measure of success.

Viewing your results can as equally informative but sometimes challenging.  Search results can be skimmed and collated together into a list from which you can download to your computer (or device) or just email them to yourself.   This is especially handy for general subject searching, helping to get a basic list of resources together on a favorite topic.  At the title level, if you want more information, then it’s possible to sift through extended bibliographic data, even down to the MARC tags.

There are other catalogs to search while you are on the site.  While not strictly “eBooks”, these resources can still be part of the experience and resources can be used on your device to some extent.  These include the famous Prints & Photographs Online Catalog containing topics ranging from historic American buildings to cartoons and Depression-era historic photographs as well as the SONIC online catalog of audio resources contained in the Library of Congress.

So this is our quick guide to the Library of Congress online catalog.  Next time you’re searching for something to read, remember this resource and stop jumping from site to site looking for your next eBook!  If you’ve had a chance to use the catalog, let us know in the links below your thoughts.  Good resource, bad resource?  Or do you even have another catalog that would work even better?

Links:

(Wikipedia) Background information on the Library of Congress

(FAQ) Library of Congress Catalog

(Help, etc.) Using the online catalog

Prints & Photographs Online Catalog

SONIC Catalog (LOC audio resources)

Sources:

Screenshot:  LOC catalog

The scanner has a touchscreen that allows a person to view details close-up, and all it takes to scan a page is a touch of a single button. Full-color images can then be easily saved to a USB flash drive. It easily captures text on creases in pages or in places where tight binding prevents traditional copying.

More info and pictures here. Thanks to Resource Shelf for the tip.

Chronicling America

The Library of Congress Chronicling America Project, located at http://chroniclingamerica.loc.gov/, is my all time favorite site for viewing newspapers. With over a million pages and full digital coverage from the late 1800’s up through the early 1920’s, from stories of the pioneer West to the “growing up” of America after World War One, there’s a lot of great stuff to get interested in! Getting started on the site, you need to remember there are two options offered: A regular searchable index of newspapers (the directory), and the digitized portion, which is the option we’re talking about today. It’s important to know that as you conduct your search, that every word on every page is searchable, the pages having been digitized with OCR software.

If you have a specific topic or keyword that you are interested in, then you will probably start with a “phrase” or keyword search to get going. If you’re just browsing, then you can view resources by state, title, and even year of coverage. A nice feature offered by the site is the ability to construct searches with a “proximity” operator, which can help you with very specific keyword and topic searches.

Results can be viewed and sorted by relevance, state, title and even date in a thumbnail or list style format. Individual pieces launch in their own viewer, with controls to zoom in or out as well as look closer at individual stories. Once you’ve found what you’re looking for, the next part is considering how to put it on your reader. There are three options offered: text, PDF, and image.

Plain text is the first and easiest method of getting the stories. Just cut and paste into your conversion program and then change over to your format of choice such as ePub, or you could just leave as plain text if your reader works best with that. The PDF format is the “prettiest” and will preserve how the newspaper looks on your device, but sometimes this can take a lot of memory to process. This can also be troublesome if your device has a small screen or the PDF cannot be rotated for better viewing. The image format is not a common one, but the help screens on the site do show you how to work with that if your device supports images.

For more information on how to search the site as well as the formats that are offered, try the following link: http://chroniclingamerica.loc.gov/help/. I might also put a plug in now for Calibre, located at http://calibre-ebook.com/. It’s supports lots of formats and will even do conversions and connect with many reading devices. So far, it’s been able to convert anything I throw at it.

Resources

While I’ve only had a chance to profile one resource, there are lots more out there. What follows are a few more links to historical newspaper archives that you can view and download to your reader. Have fun!

University Of Pennsylvania Libraries Historical Newspapers Online:

(http://gethelp.library.upenn.edu/guides/hist/onlinenewspapers.html)

Australian Newspapers: (http://newspapers.nla.gov.au/ndp/del/home)

(New Zealand) Papers Past: (http://paperspast.natlib.govt.nz/)

Editor’s Note: Tony Bandy is a librarian with a background in history and writing, he is currently freelancing and active in the field of library technology training via his company Library Knowledge. You can find his writings in Discovering Family History, Internet Genealogy as well as his blog, Adventures In History. You can reach Tony via Twitter (@LibKnowledge) or email: tony at libraryknowledge.com. PB

Project Information Literacy’s recent paper (PDF) says librarians were “rarely” used by eight out of ten college students participating in a survey. And that’s for “course-related assignments,” the very kind of situation where a librarian might make the most difference.

The paper isn’t so much about librarians per se as it as about how students seek out information for assignments, and Karen Schneider and Sara Houghton-Jan very smartly see this as a “must-read.”

Professor-recommended resources count for students. But as usual, “Google and Wikipedia are go-to sites for everyday life research for nearly every respondent.” Librarian-blessed resource methods don’t quite figure as much as they once would have, pre-Net. Of course, I see this is one more reason for a TeleRead-style national digital library system well integrated with local schools and libraries—and certainly academic ones as well.

To flesh things out, the paper’s scary words for librarians are: “Librarians were tremendously underutilized by students. Eight out of 10 of the respondents reported rarely, if ever, turning to librarians for help with course-related research assignments.”

The main point

As for the main point, Karen regards the paper as upending “most conventional wisdom. First, it shows that students’ information-seeking behavior is at odds with how many libraries provide services; second, that students actually have pragmatic, if overly-formulaic, approaches to research; third, the instructors are the first and most important human relationship these students develop in their research processes; fourth, that students value and use the scholarly resources we provide; and finally (something OCLC has reported in another context), that librarians are at the bottom of all resources students use for their research efforts.”

I myself see a future for librarians if they can lean on information providers to blend librarian-blessed resources better, so students don’t have to hop around from database to database—and if librarians otherwise simplify the use of the best resources.

One way for this to happen would be a system where librarians were more in control—TeleRead territory. Information vendors count. But users’ needs count more. Librarians need to be aggressive and pushy user advocates in such areas as DRM-usability (oxymoron?) and e-book standards—both essential to achieve maximum integration of resources.

Academic and public librarians can also survive if they expand their roles and help develop more books and other content—whether this means public domain digitization projects or partnerships with local bookstores to find and develop talented local writers.

No, I don’t see librarians as vanishing entirely if they don’t wake up. But they will see their numbers vastly diminished.

Oh, and in case you’re curious but haven’t clicked on the relevant link, yes, that dodo bird photo comes from Wikipedia.

———-

Update, 5:04: Another librarian, Gary Price, MLIS and ResourceShelf maven, very correctly reminded me there are places for librarians outside traditional library settings, and I’ve modified the post. Still, my big point remains in the context of public and academic libraries. Do we want society at large to lose librarians to the corporate world and other nonlibrary settings?

Via the Reading 2.0 list, here’s an except from Harvard Professor Robert Darnton’s article in the New York Review of Books—about the Google Books controversy:

The most ambitious solution would transform Google’s digital database into a truly public library. That, of course, would require an act of Congress, one that would make a decisive break with the American habit of determining public issues by private lawsuit. The legislation would have to settle ancillary problems—how to adjust copyright, deal with orphan books, and compensate Google for its investment in digitizing—but it would have the advantage of clearing up a messy legal landscape and of giving the American people what they deserve: a national digital library equal to the needs of the twenty-first century. But it is not clear how Google would react to such a buyout.

As a very very small shareholder in Google, I say, “About time!” Google and Amazon at this point are well on the way to pre-empting public libraries. Should we really privatize such services? Google and Amazon, perhaps, could serve as contractors, and I like the idea of there being robust alternatives to government-supported libraries—some serious freedom-of-the-press issues arise. But let’s not let Google and the like displace public libraries. Hello, Obama? Your Google ties mustn’t compromise you.

More from the Darnton piece:

If state intervention is deemed to go too far against the American grain, a minimal solution could be devised for the private sector. Congress would have to intervene with legislation to protect the digitization of orphan works from lawsuits, but it would not need to appropriate funds. Instead, funding could come from a coalition of foundations. The digitizing, open-access distribution, and preservation of orphan works could be done by a nonprofit organization such as the Internet Archive, a nonprofit group that was built as a digital library of texts, images, and archived Web pages. In order to avoid conflict with interests in the current commercial market, the database would include only books in the public domain and orphan works. Its time span would increase as copyrights expired, and it could include an opt-in provision for rightsholders of books that are in copyright but out of print.

The work need not be done in haste. At the rate of a million books a year, we would have a great library, free and accessible to everyone, within a decade. And the job would be done right, with none of the missing pages, botched images, faulty editions, omitted artwork, censoring, and misconceived cataloging that mar Google’s enterprise. Bibliographers—who appear to play little or no part in Google’s enterprise—would direct operations along with computer engineers. Librarians would cooperate with both in order to assure the preservation of the books, another weak point in GBS, because Google is not committed to maintaining its corpus, and digitized texts easily degrade or become inaccessible.

This digitizing process could be subsidized as part of the Obama administration’s economic stimulus, and the overall cost, spread out over ten to twenty years, would be manageable, perhaps $750 million in all. Meanwhile, Google and anyone else would be free to exploit the commercial sector. The national digital library could be composed from the holdings of the Library of Congress alone or, failing that, from research libraries that have not opened all their collections to Google.

Launching today is a new site from the library called Read.gov. I have
an overview post here if you’re interested but that’s really not the
reason I’m writing.

http://www.resourceshelf.com/2009/09/26/all-sorts-of-features-read-gov-launches-today/

Part of the site is the reason. At the moment they have 29 “classic”
books for teens and kids that can be read in their entirety online.

Move through a book page-by-page (forward or backwards) by simply
clicking on the page you’re currently reading or looking at. The LC
Book Reader also allows you to see facing pages, the option to go
directly to a specific page, zoom (in and out), and the ability to
view the book in “scroll” mode. At the moment books are available in
two categories: Teens and Kids.

Some of the titles available today are: The Raven, A Christmas Carol,
A Wonder-Book for Girls & Boys (Teens) and The Baby’s Own Aesop,
Baseball ABC, Denslow’s Humpty Dumpty, Mother Goose Finger Plays, The
Pied Piper of Hamelin, The Story of the Three Little Pigs, The
Wonderful Wizard of Oz, and The Wonders of a Toy Shop (Kids).

Available at: http://www.read.gov/books/

Hopefully, more titles will be added in the near future.