1. Buzzetti D. "Digital Representation and the Text Model." New Literary History 2002; 33.1:61-88.
2. Schmidt, D. "The Inadequacy of Embedded Markup for Cultural Heritage Texts." Literary and LInguistic Computing 2010; 25.3:337-356.

I’m trying to really polish off the edges and provide a slick interface in our Blacklight implementation, to contrast with the very hacky legacy OPAC.

Applied to the display of your items out with due dates… In a display of due dates, the user has to do some arithmatic to figure out how far away a given date is. What they really care about is, is this tomorrow? In a week? In a month?

So why not display it to them?  There’s a nice Rails helper to calculate human readable time deltas, although I customized it just a bit to handle the fact that some of our due dates have specific times attached (like for reserves; I use a ruby Time object), and others are just dates with no time, before close of business (I use ruby Date object). The built-in method alone works with Dates, but doesn’t always do the most sensible thing with them for this case.

A screenshot.

There are other corners that it wasn’t really feasible to polish off based on the functionality and business rules of the underlying ILS.   Like if the item isn’t renewable, I’d rather now show a renew button at all, instead showing the reason it’s not renewable. But the underlying ILS doesn’t really support that, you have to click ‘renew’ first, and then (maybe, sometimes) get a reason if it wasn’t renewable. So I polished off what I could within reasonable constraints — like at least in this interface if you choose ‘renew all’, it specifically marks which items were renewed and which weren’t, instead of the legacy OPAC that just let you look at the new due dates without telling you if they were actually new or the same old ones due to not being renewable. Oh well.

There are some good spam solutions going these days, e.g., filtering, but spam is a complex problem and here’s one more simple idea that might help. Blog software can require a person to be approved before leaving a comment. Why not use the same approach with email?

• Add a function to email programs that requires a sender to be approved.
• The approval request would contain only the sender’s name and email address. In most cases this is sufficient information to validate a desired sender.
• The approval request would deliberately contain zero content. While a spammer could still send an approval request, the absence of any content besides the name and email address removes all incentive for the spammer to expend any effort.
• Note that this approach is different than adding users to your address book as a exception list because it allows new users to be approved without an additional communication through a different route.

Share/Bookmark

 Have you seen the cover of the new issue of American Libraries?

read more

Thank you to Roxanne McHenry for the opportunity to talk about ebooks, EPUB3, book apps, and Ibis Reader in a recent interview with The Unruly Guides.

The following is a guest post by Andrea Giacomelli of BuioMetria Partecipativa (Participatory Sky Quality Monitoring), an Italy-based project that is also producing open data.

Light pollution is a major issue which concerns not only astronomers and stargazers, but has serious impacts on the environment and human health. The BMP project is an initiative founded in early 2008, by Francesco Giubbilini Francesco and Andrea Giacomelli, two environmental engineers with Tuscan roots, aimed at collecting data on light pollution by non-professionals as a form of environmental awareness raising.

Ordinary citizens, families, and schools can participate. The project also has a scientific aspect, as it allows the collection of valuable data, using a tool, called Sky Quality Meter, which has been on the market for a couple of years now. Measurements can be collected with a user’s instrument, or borrowing one from the BMP group, and subsequently uploaded to the BMP web site.

In addition to collecting new measurements, the BMP team takes care of:

• Producing educational material (videos, tutorials, installations, etc.)
• Holding conferences for both children and adults
• Managing the project website and related web resources
• Managing the database and the map where data is loaded (with support by Luca Delucchi)
• Communicate to disseminate our project

The data uploaded on the BMP web site can then be viewed and downloaded freely (data are available under the Open DataBase Licence). Other contents produced by the BMP team are released under a CC BY-NC-SA license. Furthermore, free and open source geospatial technologies are used for the database and the web mapping engine.

The project has generated considerable interest at national level, among other things, winning an award for innovation in early 2009 and receiving a diverse media coverage. The BMP project is interested in:

• Extending our network of contacts and consolidating existing ones
• Proceeding with measurement campaigns in targeted areas of special natural interest
• Improving the usability and quality of the web site and of the database
• Identifying funding for a more structured implementation of some parts of the BMP sytem and process.

Share This

No related posts.

Or we could save our energy and find untapped sources of content created by our local users and work together to create a single publishing platform and rights-management tool to allow easy creation and access to local content.

That’s the excellent ending of Kathryn Greenhill’s answer to her own question: How do we force publishers to give us ebook content that includes works that our users want and that they find easy to download to their chosen device?

This is such a compelling vision of a way forward for libraries. Not only is it more attainable than forcing publishers to do anything (or even compelling them) but it would result in a much more meaningful public library.

I’m looking forward to the rest of the posts in her series!

GetSatisfaction‘s “How does this make you feel?” intrigues me: why do people answer this? Conventional wisdom says that people don’t classify their posts.

Presumably it’s polite to ask people how they’re doing — at least in some situations. And technically there’s no post classification going on here: it’s mood classification, which most of us are trained in from a young age.

Get Satisfaction aggregates the mood on each discussion thread:

This posting describes how I am beginning to visualize co-occurrences with a Javascript library called Protovis. Alternatively, I an trying to answer the question, “What did Henry David Thoreau say in the same breath when he used the word ‘walden’?”

Network diagrams are great ways to illustrate relationships. In such diagrams nodes represent some sort of entity, and lines connecting nodes represent some sort of relationship. Nodes clustered together and sharing many lines denote some kind of similarity. Conversely, nodes whose lines are long and not interconnected represent entities outside the norm or at a distance. Network diagrams are a way of visualizing complex relationships.

Are you familiar with the phrase “in the same breath”? It is usually used to denote the relationship between one or more ideas. “He mentioned both ‘love’ and ‘war’ in the same breath.” This is exactly one of the things I want to do with texts. Concordances provide this sort of functionality. Given a word or phrase, a concordance will find the query in a corpus and display the words on either side of it. A KWIK (key word in context) index, concordances make it easier to read how words or phrases are used in relationship with their surrounding words. The use of network diagrams seem like good idea to see — visualize — how words or phrases are used within the context of surrounding words.

Protovis is a Javascript charting library developed by the Stanford Visualization Group. Using Protovis a developer can create all sorts of traditional graphs (histograms, box plots, line charts, pie charts, scatter plots) through a relatively easy-to-learn API (application programmer interface). One of the graphs Protovis supports is an interactive simulation of network diagrams called “force-directed layouts“. After experiencing some of the work done by a few of my colleagues (“Thank you Michael Clark and Ed Summers“), I wondered whether or not network diagrams could be used to visualize co-occurrences in texts. After discovering Protovis, I decided to try to implement something along these lines.

The implementation of the visualization requires the recursive creation of a term matrix. Given a word (or regular expression), find the query in a text (or corpus). Identify and count the d most frequently used words within b number of characters. Repeat this process d times with each co-occurrence. For example, suppose the text is Walden by Henry David Thoreau, the query is “spring”, d is 5, and b is 50. The implementation finds all the occurrences of the word “spring”, gets the text 50 characters on either side of it, finds the 5 most commonly used words in those characters, and repeats the process for each of those words. The result is the following matrix:

Thus, the most common co-occurrences for the word “spring” are “day”, “morning”, “first”, and “winter”. Each of these co-occurrences are recursively used to find more co-occurrences. In this example, the word “spring” co-occurs with times of day and seasons. These words then co-occur with more times of day and more seasons. Similarities and patterns being to emerge. Depending on the complexity of a writer’s sentence structure, the value of b (“breath”) may need to be increased or decreased. As the value of d (“detail”) is increased or decreased so does the number of co-occurrences to return.

Once this matrix is constructed, Protovis requires it to be converted into a simple JSON (Javascript Object Notation) data structure. In this example, “spring” points to “day”, “morning”, “first”, and “winter”. “Day” points to “days”, “night”, “every”, and “today”. Etc. As terms point to multiples of other terms, a network diagram is manifested, and the magic of Protovis is put to work. See the following illustration:

It is interesting enough to see the co-occurrences of any given word in a text, but it is even more interesting to compare the co-occurrences between texts. Below are a number of visualizations from Thoreau’s Walden. Notice how the word “walden” frequently co-occurs with the words “pond”, “water”, and “woods”. This makes a lot of sense because Walden Pond is a pond located in the woods. Notice how the word “fish” is associated with “pond”, “fish”, and “fishing”. Pretty smart, huh?

Compare these same words with the co-occurrences in a different work by Thoreau, A Week on the Concord and Merrimack Rivers. Given the same inputs the outputs are significantly different. For example, notice the difference in co-occurrences given the word “woodchuck”.

Give it a try for yourself. I have written three CGI scripts implementing the things outlined above:

• Walden by Thoreau
• A Week on the Concord and Merrimack Rivers by Thoreau
• Adventures of Huckleberry Finn by Twain

In each implementation you are given the opportunity to input your own queries, define the “size of the breath”, and the “level of detail”. The result is an interactive network diagram visualizing the most frequent co-occurrences of a given term.

The root of the Perl source code is located at http://infomotions.com/sandbox/network-diagrams/.

The visualization of co-occurrences obviously has implications for text mining and the digital humanities, but it also has implications for the field of librarianship.

Given the current environment where data and information abound in digital form, libraries have found themselves in an increasingly competitive environment. What are libraries to do? Lest they become marginalized, librarians can not rest on their “public good” laurels. Merely providing access to information is not good enough. Everybody feels as if they have plenty of access to information. What is needed are methods and tools for making better use of the data and information they acquire. Implementing text mining and visualization interfaces are one way to accomplish that goal within context of online library services. Do a search in the “online catalog”. Create a subset of interesting content. Click a button to read the content from a distance. Provide ways to analyze and summarize the content thus saving the time of the reader.

Us librarians have to do something differently. Think like an entrepreneur. Take account of your resources. Examine the environment. Innovate and repeat.

• FourWhere FourWhere helps you to find comments, tips for venues left by Foursquare, Gowalla and Yelp users.
• CSS3 Menu Create CSS menu absolutely free! Creating a nice looking menu no longer needs all the code and time it use to, thanks to CSS3. Using this free CSS3 Menu you can create great looking buttons using just CSS3 in a few clicks.
• DownloadTube Download Online Videos direct to PC, iPod, PSP, iPhone, Mobile etc.
• MyClientBase MyClientBase is a simple, intuitive, free and open source web based invoice management system developed with freelancers in mind.
• App Inventor for Android
• Gyazo Seriously Instant Screen-Grabbing for Linux, Windows and Mac
• 20 Things I Learned About Browsers and the Web An intro guide for those who are less tech savvy.

Digest powered by RSS Digest

Related posts:

1. IM for Productivity
2. My Android Apps
3. What I learned on the way home

A last-minute change to my plans for ALA Midwinter came on Tuesday when I was sought out to fill in for a speaker than canceled at the ALCTS Digital Preservation Interest Group meeting. Options for outsourcing storage and services for preserving digital content has been a recent interest, so I volunteered to combine two earlier DLTJ blog posts with some new information and present it to the group for feedback. The reaction was great, and here is the promised slide deck, links to further information, and some thoughts from the audience response.

Slide Deck and References
Options in Storage for Digital Preservation

Slides for 'Options in Storage for Digital Preservation'

The point of showing this scenario is to show the widest range of costs — from a storage-only solution like Amazon S3 to a soup-to-nuts service like OCLC Digital Archive. A word about the redacted costs. Some of the numbers for OCLC’s Digital Archive response (from 2008) came from a confidential quote, so the numbers were removed from the public table. For the numbers that are publicly listed, the values come from Barbara Quint’s article.

The articles and blog posts I referenced in the course of the presentation were:

Iglesias, Edward and Wittawat Meesangnil (2010). Using Amazon S3 in Digital Preservation in a mid sized academic library: A case study of CCSU ERIS digital archive system. The Code4Lib Journal, issue 12, retrieved 5-Jan-2011 from http://journal.code4lib.org/articles/4468

Murray, Peter (2008). Long-term Preservation Storage: OCLC Digital Archive versus Amazon S3. Disruptive Library Technology Jester. Retrieved 5-Jan-2011 from http://dltj.org/article/oclc-digital-archive-vs-amazon-s3/

Murray, Peter (2009). Can We Outsource the Preservation of Digital Bits?. Disruptive Library Technology Jester. Retrieved 5-Jan-2011 from http://dltj.org/article/outsource-digital-bits/

Quint, Barbara (2008). OCLC Introduces High-Priced Digital Archive Service. Information Today. Retrieved 5-Jan-2011 from http://newsbreaks.infotoday.com/nbReader.asp?ArticleId=49018

• Amazon S3. Retrieved 5-Jan-2011 from http://aws.amazon.com/s3/
• Chronopolis. Retrieved 5-Jan-2011 from http://chronopolis.sdsc.edu/
• DAITSS. Retrieved 5-Jan-2011 from http://daitss.fcla.edu/
• DuraCloud. Retrieved 5-Jan-2011 from https://wiki.duraspace.org/display/duracloud/DuraCloud
• Iron Mountain. Retreived 5-Jan-2011 from http://www.ironmountain.com/news/2009/impr02232009.asp
• OCLC Digital Archive. Retrieve 5-Jan-2011 from http://www.oclc.org/us/en/digitalarchive/
• Private LOCKSS Networks. Retrieved 5-Jan-2011 from http://lockss.stanford.edu/lockss/Private_LOCKSS_Networks

Some Thoughts
There was a great deal of discussion after the presentation about how good of a guarantee is good enough. Amazon S3, offers two levels of availability: “Designed to provide 99.999999999% durability and 99.99% availability of objects over a given year.” The question was whether that slight risk of loss is “good enough” for our purposes. Coming to grips with the digital storage, can we (as the librarian profession) get someone from Amazon to talk about what they do to assure that data is available? Can the terms that they use be translated into terms that we use and understand? Can we get a level of familiarity and comfort with their storage about what they do to trust them as a long-term data warehouse? Can we pull out the appropriate questions of the Trusted Repositories Audit & Certification: Criteria and Checklist to see how Amazon S3 measures up?

At the Friday ‘Big Heads’ meeting much of the conversation revolved around Incrementalism vs. Revolution, as have so many conversations, about so many things. Someone quoted David Mamet (I can’t find the quote) that what we need is sledge hammers, not chisels, and I thought it was a notion too good to pass up as a jumping off point to discuss that meeting.

There were a lot of interesting topics discussed at the meetings, but as is my habit I’m going to focus only on the topics of interest to me. As usual there were a number of vendors in the audience, and when a few of the ‘heads’ at the main table voiced the expectation that they would be depending on the vendor community for help as they experienced additional staff reductions and resource constraints in general, the vendors came up to the microphones to respond. A couple of vendors expressed their concern that the library community in general has not been able to articulate what they want from vendors, and this has made it difficult for them to develop business plans. I hear a variation of this line when I stroll the exhibit halls and talk to vendors about what their plans are for RDA implementation. Almost always I hear that they have not heard from their customers about what they want, and they’re waiting for that before making plans. As a result, when I’m presenting to librarians about RDA, I tell them that they should be talking to their vendors, asking when and how they will be implementing, etc., etc. The problem with that approach is that a) most of the time the librarians don’t know what to ask, beyond the when and how; and b) when they get an answer they often don’t know how to interpret it. Maybe I’m slow, but I’m coming to the conclusion that I should stop telling people to talk to their vendors about RDA. I’m not sure it matters.

I went up to the microphone for one of my usual rants, after hearing quite enough of this dancing around. Here’s the reality, as I see it:

1. Libraries are unlikely to agree on what they want (this has been true in the past, and will likely be true in the future)
2. Given the generally low level of understanding of RDA data issues across the library spectrum (and certainly the vendors), it’s unlikely that any articulation of needs to vendors would represent something that vendors could rely on to build a business plan
3. Vendors are still talking about the provision (e.g. sale) of bibliographic records as a basis for their services to libraries.

My rant included all three of those points and more. Little over a year ago, the R2 report on the marketplace for MARC records (upon which I blogged) assumed that there is a marketplace for MARC records which will continue and that a direct return on investment is possible (or desirable) for creators of data. I said then, and still believe, that such a viewpoint is both unrealistic and in fact destructive to the task of moving forward into a world where data is not the coin of the realm but freely available (this is the basis for linked open data) and the investment and return on investment is around data services, not data sale. After my rant to Big Heads, one of the vendors came up to talk to me and offered up some useful nuggets to support my view: a) they provide records, but don’t make much money on them; b) the realm of digital metadata is vastly more complicated than that for physical metadata. It’s a huge challenge for vendors to operate in this world, but clearly the usual answers are no longer working, even as the data revolution is not yet fully upon us. The inevitable conclusion is that vendors who wait for their customers to tell them what they want may not survive the coming revolution. This is no time for chisels.

In this context it’s good to meditate on Henry Ford’s famous statement: “If I had asked people what they wanted, they would have said faster horses.”