Journal of Web Librarianship: Responsive Web Site Development at the Library, Institute of Technology Tallaght: A Case Study
Law, J. (2009). The New Blackwell Companion to Social Theory, chapter Actor Network Theory and Material Semiotics, pages 141–158. Wiley-Blackwell, Oxford.
Law is careful to state up front that ANT is not a theory, and is instead a set of descriptive practices. It can be defined in the abstract, but it is best understood by looking at how it works in practice. Perhaps this is where Nicolini got the idea of the toolkit from, which eschews theory, and places ANT firmly in a practitioner space:
As a form, one of several, of material semiotics, it is best understood as a toolkit for telling interesting stories about, and interfering in, those relations. More profoundly, it is a sensibility to the messy practices of relationality and materiality of the world. (p. 141-142)
Law mentions that key to understanding ANT is the idea of translation, that was first introduced by [Michel Serres] and used by Latour and Callon in their early work on ANT. Translation is making two words equivalent, but since no two words are equivalent translation is about betrayal or shifting and linking words. He is also situates ANT as a scaled down, or empirical version of what Foucault calls discourses or epistemes. He also compares translation to Deleuze’s idea of nomadic philosophy. He also draws a parallel between Delueze’s assemblage or agencements and ANT. Just as an aside it’s interesting to think about how this philosophical work involving networks w was germinating in France in the the 1970s and 1980s and then we see the Web itself being created in the late 1980s.
Here are some features of Actor Network Theory as originally conceived:
Material Durability: social arrangements delegated into non-bodily physical form tend to hold their shape better than those that simply depend on face-to-face interaction
Strategic Durability: actor network conception of strategy can be understood more broadly to include teleologically ordered patterns of relations indifferent to human intentions
Discursive Durability: discourses define conditions of possibility, making some ways of ordering webs of relations easier and others difficult or impossible
And then here are some features of what Law calls the New Material Semiotic a more “polytheistic” version of ANT that he groups under the Deluezian heading Diaspora. Interestingly he cites Star (1990) as offering one of the earliest critiques of ANT, from a feminist perspective.
Performativity: the social is not constructed, it is enacted or performed, and its in these performances that they can be understood and studied.
Multiplicity: a given phenomenon can be understood as a confluence of practices, that aren’t different perspectives on the same phenomenon, but are actually different practices that may be coordinated for some duration Mol (2002). Aside: it’s kind of interesting that Mol has been one of the points of connection between my independent study on practice theory and my ethnographic methods class this semester.
Fluidity: the ability of objects and practices to mutate, change shape, reconfigure and persist.
Realities and Goods: networks create multiple overlapping ethical realities
To describe the real is always an ethically charged act. But, and this is the crucial point, the two are only partially connected: goods and reals cannot be reduced to each other. An act of political will can never, by itself, overturn the endless and partially connected webs that enact the real. Deconstruction is not enough. Indeed, it is trivial (Latour, 2004). The conclusion is inescapable: as we write we have a simultaneous responsibility both to the real and to the good. Such is the challenge faced by this diasporic material semiotics. To create and recreate ways of working in and on the real while simultaneously working well in and on the good.References
Latour, B. (2004). Why has critique run out of steam? From matters of fact to matters of concern. Critical Inquiry, 30(2), 225–248.
Mol, A. (2002). The body multiple: Ontology in medical practice. Duke University Press.
Nicolini, D. (2012). Practice theory, work, and organization: An introduction. Oxford University Press.
Star, S. L. (1990). Power, technology and the phenomenology of conventions: On being allergic to onions. The Sociological Review, 38(S1), 26–56.
Not everyone I've talked to thinks the combination is a good idea. In the publishing world, there is fear that the giants of the internet who dominate the W3C will not be responsive to the idiosyncratic needs of more traditional publishing businesses. On the other side, there is fear that the work of IDPF and Readium on "Lightweight Content Protection" (a.k.a. Digital Rights Management) will be a another step towards "locking down the web". (see the controversy about "Encrypted Media Extensions")
What's more, a peek into the HTML5 development process reveals a complicated history. The HTML5 that we have today derives from a a group of developers (the WHATWG) who got sick of the W3C's processes and dependencies and broke away from W3C. Politics above my pay grade occurred and the breakaway effort was folded back into W3C as a "Community Group". So now we have two, slightly different versions of HTML, the "standard" HTML5 and WHATWG's HTML "Living Standard". That's also why HTML5 omitted much of W3C's Semantic Web development work such as RDFa.
Amazon (not a member of either IDPF or W3C) is the elephant in the room. They take advantage of IDPF's work in a backhanded way. Instead of supporting the EPUB standard in their Kindle devices, they use proprietary formats under their exclusive control. But they accept EPUB files in their content ingest process and thus extract huge benefit from EPUB standardization. This puts the advancement of EPUB in a difficult position. New features added to EPUB have no effect on the majority of ebook user because Amazon just converts everything to a proprietary format.
Last month, the W3C published its vision for eBook standards, in the form on an innocuously titled "Portable Web Publications Use Cases and Requirements". For whatever reason, this got rather limited notice or comment, considering that it could be the basis for the entire digital book industry. Incredibly, the word "ebook" appears not once in the entire document. "EPUB" appears just once, in the phrase "This document is also available in this non-normative format: ePub". But read the document, and it's clear that "Portable Web Publication" is intended to be the new standard for ebooks. For example, the PWP (can we just pronounce that "puup"?) "must provide the possibility to switch to a paginated view" . The PWP (say it, "puup") needs a "default reading order", i.e. a table of contents. And of course the PWP has to support digital rights management: "A PWP should allow for access control and write protections of the resource." Under the oblique requirement that "The distribution of PWPs should conform to the standard processes and expectations of commercial publishing channels." we discover that this means "Alice acquires a PWP through a subscription service and downloads it. When, later on, she decides to unsubscribe from the service, this PWP becomes unavailable to her." So make no mistake, PWP is meant to be EPUB 4 (or maybe ePub4, to use the non-normative capitalization).
There's a lot of unalloyed good stuff there, too. The issues of making web publications work well offline (an essential ingredient for archiving them) are technical, difficult and subtle, and W3C's document does a good job of flushing them out. There's a good start (albeit limited) on archiving issues for web publications. But nowhere in the statement of "use cases and requirements" is there a use case for low cost PWP production or for efficient conversion from other formats, despite the statement that PWPs "should be able to make use of all facilities offered by the [Open Web Platform]".
The proposed merger of IDPF and W3C raises the question: who gets to decide what "the ebook" will become? It's an important question, and the answer eventually has to be open rather than proprietary. If a combined IDPF and W3C can get the support of Amazon in open standards development, then everyone will benefit. But if not, a divergence is inevitable. The publishing industry needs to sustain their business; for that, they need an open standard for content optimized to feed supply chains like Amazon's. I'm not sure that's quite what W3C has in mind.
I think ebooks are more important than just the commercial book publishing industry. The world needs ways to deliver portable content that don't run through the Amazon tollgates. For that we need innovation that's as unconstrained and disruptive as the rest of the internet. The proposed combination of IDPF and W3C needs to be examined for its effects on innovation and competition.
Philip K. Dick's Mr. Robot is
one of the stories in Imagination
Stories of Science and Fantasy,
January 1953. It is available as
an ebook from Project Gutenberg
and from GITenbergMy guess is that Amazon is not going to participate in open ebook standards development. That means that two different standards development efforts are needed. Publishers need a content markup format that plays well with whatever Amazon comes up with. But there also needs to be a way for the industry to innovate and compete with Amazon on ebook UI and features. That's a very different development project, and it needs a group more like WHATWG to nurture it. Maybe the W3C can fold that sort of innovation into its unruly stable of standards efforts.
I worry that by combining with IDPF, the W3C work on portable content will be chained to the supply-chain needs of today's publishing industry, and no one will take up the banner of open innovation for ebooks. But it's also possible that the combined resources of IDPF and W3C will catalyze the development of open alternatives for the ebook of tomorrow.
Is that too much to hope?
Last week I attended Hydra Connect 2016 in Boston, with a team of three others from the Brown University Library. Our team consisted of a Repository Developer, Discovery Systems Developer, Metadata Specialist, and Repository Manager. Here are some notes and thoughts related to the conference from my perspective as a repository programmer.
There was a poster about IPFS, which is a peer-to-peer hypermedia protocol for creating a distributed web. It’s an interesting idea, and I’d like to look into it more.
APIs and Architecture
There was a lot of discussion about the architecture of Hydra, and Tom Cramer mentioned APIs specifically in his keynote address. In the Brown Digital Repository, we use a set of APIs that clients can access and use from any programming language. This architecture lets us define layers in the repository: the innermost layer is Fedora and Solr, the next layer is our set of APIs, and the outer layer is the Studio UI, image/book viewers, and custom websites built on the BDR data. There is some overlap in our layers (eg. the Studio UI does hit Solr directly instead of going through the APIs), but I still think it improves the architecture to think about these layers and try not to cross multiple boundaries. Besides having clients that are written in python, ruby, and php, this API layer may be useful when we migrate to Fedora 4 – we can use our APIs to communicate with both Fedora 3 and Fedora 4, and any client that only hits the APIs wouldn’t need to be changed to be able to handle content in Fedora 4.
I would be interested in seeing a similar architecture in Hydra-land (note: this is an outsider’s perspective – I don’t currently work on CurationConcerns, Sufia, or other Hydra gems). A clear boundary between “business logic” or processing and the User Interface or data presentation seems like good architecture to me.
Data Modeling and PCDM
Monday was workshop day at Hydra Connect 2016, and I went to the Data Modeling workshop in the morning and the PCDM In-depth workshop in the afternoon. In the morning session, someone mentioned that we shouldn’t have data modeling differences without good reason (ie. does a book in one institution really have to be modeled differently from a book at another institution?). I think that’s a good point – if we can model our data the same way, that would help with interoperability. PCDM, as a standard for how our data objects are modeled, might be great way to promote interoperability between applications and institutions. In the BDR, we could start using PCDM vocabulary and modeling techniques, even while our data is in Fedora 3 and our code is written in Python. I also think it would be helpful to define and document what interoperability should look like between institutions, or different applications at the same institution.
It seems like the IIIF community has a good solution to image interoperability. The IIIF community has defined a set of APIs, and then it lists various clients and servers that implement those APIs. I wonder if the Hydra community would benefit from more of a focus on APIs and specifications, and then there could be various “Hydra-compliant” servers and clients. Of course, the Hydra community should continue to work on code as well, but a well-defined specification and API might improve the Hydra code and allow the development of other Hydra-compliant code (eg. code in other programming languages, different UIs using the same API, …).
From Mike Conlon, VIVO project director
From Tim Donohue, DSpace tech lead on behalf of the DSpace developers
Austin, TX DSpace 5.6 is now available providing security fixes to the XMLUI, JSPUI and REST API, along with bug fixes to the DSpace 5.x platform.
We have a data management dilemma, and we hope that you – data smart people of the world – can help us out. We need a versioning and change tracking system for around 50 million XML files, and no existing solutions seem to fit.
About The Project
Our Tracking Task
Like most digitization projects, we generate many page images. The binary image files rarely change and are not difficult to track. However, in addition to images, we create rich XML files containing descriptive/structural metadata and OCR. As we uncover mistakes in the OCR, encounter metadata anomalies, and gather new data through CAP-facilitated research projects, we will need to update these files. Tracking those changes is going to be a bit more difficult.The Files
We are scanning about 37,000 volumes. Each volume contains multiple pages (obviously) and multiple cases. Usually, a case takes up a few pages, but some cases are so small that several can fit on one page, so there’s no direct parent/child relationship between them. Cases never span volumes.
If you’re interested in checking out a case for yourself, you can grab a sample case with all the associated files here.
How we split these things up into files:
For each volume:
- One METS XML file with all volume-level metadata (~ 1 MB avg)
For each page side:
- One lossless jp2 (~2.5 MB avg)
- One 1-bit tiff (~60 KB avg)
- One ALTO v3 XML file (~75 KB avg)
For each case:
- One METS XML file, which includes the text of each case body, and all case-level metadata (~75 KB avg)
- Roughly 37k volumes, so about 37,000 volume XML files
- Roughly 40mil page-sides, so that many jp2s, tiffs, and ALTO XML files
- A bit fewer than 10 million Cases, so that many Case METS XML files
Our key requirements:Data Set Versioning
Ideally this could be done at the corpus or series level (described below.) This would be useful to researchers working with larger sets of data.Sanitizable Change Tracking
As is the case with most change-tracking systems, when recording changes, we usually want to be able to ascertain the state of the data before the change, whether this is by recording the old version and the new version, or the delta between the two versions. However, with some change types, we do require the ability to either delete the delta or the old data state. Ideally, we would be able to do this without removing the entire change history for the file.File Authentication
People should be able to check if the version of the file they have is, or ever has been in our repository.Open Data Format
Even if the change/versioning data isn’t natively stored in an easily human-readable format, it must at least be exportable into a useful open format. No strictly proprietary solutions.Access Control
We have to be able to control access to this data.
Our Wish List
- FOSS (Free Open Source Software) Based Solution
- Diffing — allow downstream databases to fetch deltas between their current version and the latest
- Minimal system management overhead
- Ability to efficiently distribute change history with the data, ideally in a human-readable format
- XML-aware change tracking, so changes can be applied to XML elements with the same identifiers and content, in different files
- Will automatically detect replacement images
What we’ve considered, and their disadvantages
- Dataset is much too large to store in a single repository
- Non-plain-text change history
- Redacting a single file requires rewriting large portions of the tree
- Not geared to handle XML data
- Would require storing in a different format/syncing
- Non-plain-text change history
- Provides sanitizable change tracking but no versioning of larger data sets
- Non-plain-text change history
- Seems to not allow easy sanitization of change history
- P2P Architecture doesn’t give us enough access control for the first phase of the project.
Something we write ourselves
- Reinvents the wheel, at least in part
- Probably not as efficient as more mature tools
Should the data be restructured?
Currently, the repository is fairly flat with each volume in its own directory, but no other hierarchy.
Files could be partitioned by “series.” A series is a numbered sequence of volumes from a particular court, such as the Massachusetts Reporter of Decisions. The largest series so far contains approximately 1k volumes, 750k pages, and 215k cases, but they are rather inconsistently sized, with the smallest containing only one volume, and the average containing 71. There are 635 series in total.
Many data consumers will want only case files, and not per-page or per-volume files. It may make sense to store case XML files and non-case-XML files in separate repositories.
What We Need From You
Ideas. We want to make sure that we get this right the first time. If you have insight into solving problems like this, we’d love to hear from you.
Please reach out to us at firstname.lastname@example.org.
The 2017 conference will be held from March 6 through March 9 in the at the Luskin Conference Center at UCLA. With a very large and modern conference center at our disposal, Code4Lib 2017 will be able to accomodate the growing number of attendees while also retaining that small, tight-knit Code4Lib community feeling of previous years. We hope you can come join us!
More details to come soon; in the meantime, the Keynote Committee is about to close submissions for the conference keynote speaker, so be sure to nominate a keynote speaker on the Code4Lib wiki before Friday, October 14th. Proposals for prepared talks are also currently open and will be accepted until November 7th. This year, there is a new, separate process for panel proposals which we are very excited about. Proposals for pre-conference workshops are also currently open and will be accepted until November 8th.
Also, the Sponsorship Committee is actively looking for sponsors for 2017; more information about how to get in touch with the committee will be forthcoming.
It’s shaping up to be a great conference this year, and there will be lots more opportunities to volunteer. Our team is looking forward to seeing you on March 6!
~ The Code4Lib 2017 Los Angeles Committee
This travelogue documents my experiences and what I learned on a tiny road trip including visits to Indiana University, Purdue University, University of Illinois / Urbana-Champagne, and Washington University In St. Louis between Monday, October 26 and Friday, October 30, 2017. In short, I learned four things: 1) of the places I visited, digital scholarship centers support a predictable set of services, 2) the University Of Notre Dame’s digital scholarship center is perfectly situated in the middle of the road when it comes to the services provided, 3) the Early Print Project is teamed with a set of enthusiastic & animated scholars, and 4) Illinois is very flat.
Four months ago I returned from a pseudo-sabbatical of two academic semesters, and exactly one year ago I was in Tuscany (Italy) painting cornfields & rolling hills. Upon my return I felt a bit out of touch with some of my colleagues in other libraries. At the same time I had been given an opportunity to participate in a grant-sponsored activity (the Early English Print Project) between Northwestern University, Washington University In St. Louis, and the University Of Notre Dame. Since I was encouraged to visit the good folks at Washington University, I decided to stretch a two-day visit into a week-long road trip taking in stops at digital scholarship centers. Consequently, I spent bits of time in Bloomington (Indiana), West Lafayette (Indiana), Urbana (Illinois), as well as St. Louis (Missouri). The whole process afforded me the opportunity to learn more and get re-acquainted.Indiana University / Bloomington
My first stop was in Bloomington where I visited Indiana University, and the first thing that struck me was how much Bloomington exemplified the typical college town. Coffee shops. Boutique clothing stores. Ethnic restaurants. And teaming with students ranging from fraternity & sorority types, hippie wanna be’s, nerds, wide-eyed freshman, young lovers, and yes, fledgling scholars. The energy was positively invigorating.
My first professional visit was with Angela Courtney (Head of Arts & Humanities, Head of Reference Services, Librarian for English & American Literature, and Director of the Scholars’ Commons). Ms. Courtney gave me a tour of the library’s newly renovated digital scholarship center.  It was about the same size at the Hesburgh Libraries’s Center, and it was equipped with much of the same apparatus. There was a scanning lab, plenty of larger & smaller meeting spaces, a video wall, and lots of open seating. One major difference between Indiana and Notre Dame was the “reference desk”. For all intents & purposes, the Indiana University reference desk is situated in the digital scholarship center. Ms. Courtney & I chatted for a long hour, and I learned how Indiana University & the University Of Notre Dame were similar & different. Numbers of students. Types of library collections & services. Digital initiatives. For the most part, both universities have more things in common than differences, but their digital initiatives were by far more mature than the ones here at Notre Dame.
Later in the afternoon I visited with Yu (Marie) Ma who works for the HathiTrust Research Center.  She was relatively new to the HathiTrust, and if I understand her position correctly, then she spends a lot of her time setting up technical workflows and the designing the infrastructure for large-scale text analysis. The hour with Marie was informative on both of our parts. For example, I outlined some of the usability issues with the Center’s interface(s), and she outlined how the “data capsules” work. More specifically, “data capsules” are virtual machines operating in two different modes. In one mode a researcher is enabled to fill up a file system with HathiTrust content. In the other mode, one is enabled to compute against the content and return results. In one or the other of the modes (I’m not sure which), Internet connectivity is turned off to disable the leaking of HathiTrust content. In this way, a HathiTrust data capsule operates much like a traditional special collections room. A person can go into the space, see the book, take notes with a paper & pencil, and then leave sans any of the original materials. “What is old is new again.” Along the way Marie showed me a website — Lapps Grid — which looks as if it functions similar to Voyant Tools and my fledgling EEBO-TCP Workset Browser. [3, 4, 5] Amass a collection. Use the collection as input against many natural language processing tools/applications. Use the output as a means for understanding. I will take a closer look at Lapps Grid.Purdue University
The next morning I left the rolling hills of southern Indiana for the flatlands of central Indiana and Purdue University. There I facilitated a brown-bag lunch discussion on the topic of scalable reading, but the audience seemed more interested in the concept of digital scholarship centers. I described the Center here at Notre Dame, and did my best to compare & contrast it with others as well as draw into the discussion the definition of digital humanities. Afterwards I went to lunch with Micheal Witt and Amanda Visconti. Mr. Witt spends much of his time on institutional repostory efforts, specifically in regards to scientific data. Ms. Visconti works in the realm of the digital humanities and has recently made available her very interesting interactive dissertation — Infinite Ulysses.  After lunch Mr. Witt showed me a new library space scheduled to open before the Fall Semester of 2017. The space will be library-esque during the day, and study-esque during the evening. Through the process of construction, some of their collection needed to be weeded, and I found the weeding process to be very interesting.University of Illinois / Urbana-Champagne
Up again in the morning and a drive to Urbana-Champagne. During this jaunt I became both a ninny and a slave to my computer’s (telephone’s) navigation and functionality. First it directed me to my location, but no parking places. After identifying a parking place on my map (computer), I was not able to get directions on how to get there. Once I finally found parking, I required my telephone to pay. Connect to remote site while located in concrete building. Create account. Supply credit card number. Etc. We are increasingly reliant (dependent) on these gizmos.
My first meeting was with Karen Hogenboom (Associate Professor of Library Administration, Scholarly Commons Librarian and Head, Scholarly Commons). We too discussed digital scholarship centers, and again, there were more things in common with our centers than differences. Her space was a bit smaller than Notre Dame’s, and their space was less about specific services and more about referrals to other services across the library and across the campus. For example, geographic information systems services and digitization services were offered elsewhere.
I then had a date with an old book, but first some back story. Here at Notre Dame Julia Schneider brought to my attention a work written by Erasmus and commenting on Cato which may be a part of a project called The Digital Schoolbook. She told me how there were only three copies of this particular book, and one of them was located in Urbana. Consequently, a long month ago, I found a reference to the book in the library catalog, and I made an appointment to see it in person. The book’s title is Erasmi Roterodami libellus de co[n]structio[n]e octo partiu[m]oratio[n]is ex Britannia nup[er] huc plat[us] : et ex eo pureri bonis in l[ite]ris optio and it was written/published in 1514. [7, 8] The book represented at least a few things: 1) the continued and on-going commentary on Cato, 2) an example of early book printing, and 3) forms of scholarship. Regarding Cato I was only able to read a single word in the entire volume — the word “Cato” — because the whole thing was written in Latin. As an early printed book, I had to page through the entire volume to find the book I wanted. It was the last one. Third, the book was riddled with annotations, made from a number of hands, and with very fine-pointed pens. Again, I could not read a single word, but a number of the annotations were literally drawings of hands pointing to sections of interest. Whoever said writing in books was a bad thing? In this case, the annotations were a definite part of the scholarship.
Washington University In St. Louis
Yet again, I woke up the next morning and continued on my way. Along the road there were billboards touting “foot-high pies” and attractions to Indian burial grounds. There were corn fields being harvested, and many advertisements pointing to Abraham Lincoln stomping locations.
Late that afternoon I was invited to participate in a discussion with Doug Knox, Steve Pentecost, Steven Miles, and Dr. Miles’s graduate students. (Mr. Knox & Mr. Pentecost work in a university space called Arts & Sciences Computing.) They outlined and reported upon a digital project designed to aid researchers & scholars learn about stelae found along the West River Basin in China. I listened. (“Stelae” are markers, usually made of stone, commemorating the construction or re-construction of religious temples.) To implement the project, TEI/XML files were being written and “en masse” used akin to a database application. Reports were to be written agains the XML to create digital maps as well as browsable lists of names of people, names of temples, dates, etc. I got to thinking how timelines might also be apropos.
The bulk of the following day (Friday) was spent getting to know a balance of colleagues and discussing the Early English Print Project. There were many people in the room: Doug Knox & Steve Pentecost from the previous day, Joesph Loewenstein (Professor, Department of English, Director Of the Humanities Digital Workshop and the Interdisciplinary Project in the Humanities) Kate Needham, Andrew Rouner (Digital Library Director), Anupam Basu (Assistant Professor, Department of English), Shannon Davis (Digital Library Services Manager), Keegan Hughes, and myself.
More specifically, we talked about how sets of EEBO/TCP () TEI/XML files can be: 1) corrected, enhanced, & annotated through both automation as well as crowd-sourcing, 2) supplemented & combined with newly minted & copy-right free facsimiles from the original printed documents, 3) analyzed & reported upon through text mining & general natural language processing techniques, and 4) packaged up & redistributed back to the scholarly community. While the discussion did not follow logically, it did surround a number of unspoken questions, such as but not limited to:
- Is METS a desirable re-distribution method?  What about some sort of database system instead?
- To what degree is governance necessary in order for us to make decisions?
- To what degree is it necessary to pour the entire corpus (more than 60,000 XML files with millions of nodes) into a single application for processing, and is the selected application up to the task?
- What form or flavor of TEI would be used as the schema for the XML file output?
- What role will an emerging standard called IIIF play in the process of re-distribution? 
- When is a corrected text “good enough” for re-distribution?
To my mind, none of these questions were answered definitively, but then again, it was an academic discussion. On the other hand, we did walk away with a tangible deliverable — a whiteboard drawing illustrating a possible workflow going something like this:
- cache data from University of Michigan
- correct/annotate the data
- when data is “good enough”, put the data back into the cache
- feed the data back to the University of Michigan
- when data is “good enough”, text mine the data and put the result to back into the cache
- feed the data back to the University of Michigan
- create new facsimiles from the printed works
- combine the facsimiles with the data, and put the result to back into the cache
- feed the data back to the University of Michigan
After driving through the country side, and after two weeks of reflection, I advocate a slightly different workflow:
- cache TEI data from GitHub repository, which was originally derived from the University of Michigan 
- make cache accessible to the scholarly community through a simple HTTP server and sans any intermediary application
- correct/annotate the data
- as corrected data becomes available, replace files in cache with corrected versions
- create copyright-free facsimiles of the originals, combine them with corrected TEI in the form of METS files, and cache the result
- use the METS files to generate IIIF manifests, and make the facsimiles viewable via the IIIF protocol
- as corrected files become available, use text mining & natural language processing to do analysis, combine the results with the original TEI (and/or facsimiles) in the form of METS files, and cache the result
- use the TEI and METS files to create simple & rudimentary catalogs of the collection (author lists, title lists, subject/keyword lists, date lists, etc.), making it easier for scholars to find and download items of interest
The primary point I’d like to make in regard to this workflow is, “The re-distribution of our efforts ought to take place over simple HTTP and in the form of standardized XML, and I do not advocate the use of any sort of middle-ware application for these purposes.” Yes, of course, middle-ware will be used to correct the TEI, create “digital combos” of TEI and images, and do textual analysis, but the output of these processes ought to be files accessible via plain o’ ordinary HTTP. Applications (database systems, operating systems, content-management systems, etc.) require maintenance, and maintenance is done by a few & specialized number of people. Applications are often times “black boxes” understood and operated by a minority. Such things are very fragile, especially compared to stand-alone files. Standardized (XML) files served over HTTP are easily harvestable by anybody. They are easily duplicated. They can be saved on platform-independent media such as CD’s/DVD’s, magnetic tape, or even (gasp) paper. Once the results of our efforts are output as files, then supplementary distribution mechanisms can be put into place, such as IIIF or middleware database applications. XML files (TEI and/or METS) served over simple HTTP ought be the primary distribution mechanism. Such is transparent, sustainable, and system-independent.
Over lunch we discussed Spenser’s Faerie Queene, the Washington University’s Humanities Digital Workshop, and the salient characteristics of digital humanities work.  In the afternoon I visited the St. Louis Art Museum, whose collection was rich.  The next day, on my way home through Illinois, I stopped at the tomb of Abraham Lincoln in order to pay my respects.
In conclusion, I learned a lot, and I believe my Americana road trip was a success. My conception and defintion of digital scholarship centers was re-enforced. My professional network was strengthened. I worked collaboratively with colleagues striving towards a shared goal. And my personal self was enriched. I advocate such road trips for anybody and everybody.Links
 digital scholarship at Indiana University – https://libraries.indiana.edu/services/digital-scholarship
 HathiTrust Research Center – https://analytics.hathitrust.org
 Lapps Grid – http://www.lappsgrid.org
 Voyant Tools – http://voyant-tools.org
 EEBO-TCP Workset Browser – http://blogs.nd.edu/emorgan/2015/06/eebo-browser/
 Infinite Ulysses – http://www.infiniteulysses.com
 old book from the UIUC catalog – https://vufind.carli.illinois.edu/vf-uiu/Record/uiu_5502849
 old book from the Universal Short Title Catalogue – http://ustc.ac.uk/index.php/record/403362
 EEBO/TCP – http://www.textcreationpartnership.org/tcp-eebo/
 METS – http://www.loc.gov/standards/mets/
 IIIF – http://iiif.io
 GitHub repository of texts – https://github.com/textcreationpartnership/Texts
 Humanities Digital Workshop – https://hdw.artsci.wustl.edu
 St. Louis Art Museum – http://www.slam.org
New vacancy listings are posted weekly on Wednesday at approximately 12 noon Central Time. They appear under New This Week and under the appropriate regional listing. Postings remain on the LITA Job Site for a minimum of four weeks.
New This Week
Visit the LITA Job Site for more available jobs and for information on submitting a job posting.
Donald S. Kornfeld and Sandra L. Titus have a comment in Nature entitled Stop ignoring misconduct arguing that the bad incentives for researchers inevitably produce misconduct, but that this is routinely swept under the carpet:
In other words, irreproducibility is the product of two factors: faulty research practices and fraud. Yet, in our view, current initiatives to improve science dismiss the second factor. For example, leaders at the US National Institutes of Health (NIH) stated in 2014: “With rare exceptions, we have no evidence to suggest that irreproducibility is caused by scientific misconduct”. In 2015, a symposium of several UK science-funding agencies convened to address reproducibility, and decided to exclude discussion of deliberate fraud.The scientific powers-that-be are ignoring the science:
Only 10–12 individuals are found guilty by the US Office of Research Integrity (ORI) each year. That number, which the NIH used to dismiss the role of research misconduct, is misleadingly low, as numerous studies show. For instance, a review of 2,047 life-science papers retracted from 1973 to 2012 found that around 43% were attributed to fraud or suspected fraud. A compilation of anonymous surveys suggests that 2% of scientists and trainees admit that they have fabricated, falsified or modified data. And a 1996 study of more than 1,000 postdocs found that more than one-quarter would select or omit data to improve their chances of receiving grant funding.Linked from this piece are several other Nature articles about misconduct:
- Misconduct: Lessons from researcher rehab by James M. DuBois, John T. Chibnall, Raymond Tait and Jillon Vander Wal is an interesting report on a program to which researchers are referred after misconduct has been detected. It identifies the pressure for "more not better" as a factor:
By the metrics that institutions use to reward success, our programme participants were highly successful researchers; they had received many grants and published many papers. Yet, becoming overextended was a common reason why they failed to adequately oversee research. It may also have led them to make compliance a low priority. ... Scientists become overextended in part because their institutions value large numbers of projects.
- Robust research: Institutions must do their part for reproducibility by C. Glenn Begley, Alastair M. Buchan and Ulrich Dirnagl argues for compliance processes for general research analogous to those governing animal research. They too flag "more not better":
Institutions must support and reward researchers who do solid — not just flashy — science and hold to account those whose methods are questionable. ... Although researchers want to produce work of long-term value, multiple pressures and prejudices discourage good scientific practices. In many laboratories, the incentives to be first can be stronger than the incentives to be right.
- Workplace climate: Metrics for ethics by Monya Baker reports on institutions using a survey of researcher's workplace climate issues in areas such as "integrity norms (such as giving due credit to others' ideas), integrity inhibitors (such as inadequate access to material resources) and adviser–advisee relations".
For a portrait of income inequality in science, look no further than the labs of the University of California. Twenty-nine medical researchers there earned more than US$1 million in 2015 and at least ten non-clinical researchers took home more than $400,000 each. Meanwhile, thousands of postdocs at those universities received less than $50,000. Young professors did better, but many still collected less than one-quarter of the earnings of top researchers.The work of Richard Wilkinson, Kate Pickett and others shows that increasing inequality is correlated with misconduct, among other social ills. The finance industry is the poster-child of inequality, their machinations in the 2008 financial crisis, such as robosigning and synthetic CDOs, should be evidence enough. Which way round the chain of causation runs is not clear.
Even if there is no actual misconduct, the bad incentives will still cause bad science to proliferate via natural selection, or the scientific equivalent of Gresham's Law that "bad money drives out good". The Economist's Incentive Malus, subtitled Poor scientific methods may be hereditary, is based on The natural selection of bad science by Paul E. Smaldino and Richard McElreath, which starts:
Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for improvement, suggesting that they result from something more than just misunderstanding. The persistence of poor methods results partly from incentives that favour them, leading to the natural selection of bad science. This dynamic requires no conscious strategizing—no deliberate cheating nor loafing—by scientists, only that publication is a principal factor for career advancement.The Economist writes that Smaldino and McElreath:
decided to apply the methods of science to the question of why this was the case, by modelling the way scientific institutions and practices reproduce and spread, to see if they could nail down what is going on.
They focused in particular on incentives within science that might lead even honest researchers to produce poor work unintentionally. To this end, they built an evolutionary computer model in which 100 laboratories competed for “pay-offs” representing prestige or funding that result from publications. ... Labs that garnered more pay-offs were more likely to pass on their methods to other, newer labs (their “progeny”).
Some labs were better able to spot new results (and thus garner pay-offs) than others. Yet these labs also tended to produce more false positives—their methods were good at detecting signals in noisy data but also, as Cohen suggested, often mistook noise for a signal. More thorough labs took time to rule these false positives out, but that slowed down the rate at which they could test new hypotheses. This, in turn, meant they published fewer papers.
In each cycle of “reproduction”, all the laboratories in the model performed and published their experiments. Then one—the oldest of a randomly selected subset—“died” and was removed from the model. Next, the lab with the highest pay-off score from another randomly selected group was allowed to reproduce, creating a new lab with a similar aptitude for creating real or bogus science. ... they found that labs which expended the least effort to eliminate junk science prospered and spread their methods throughout the virtual scientific community.Worse, they found that replication was did not suppress this selection process:
Replication has recently become all the rage in psychology. In 2015, for example, over 200 researchers in the field repeated 100 published studies to see if the results of these could be reproduced (only 36% could). Dr Smaldino and Dr McElreath therefore modified their model to simulate the effects of replication, by randomly selecting experiments from the “published” literature to be repeated.
A successful replication would boost the reputation of the lab that published the original result. Failure to replicate would result in a penalty. Worryingly, poor methods still won—albeit more slowly. This was true in even the most punitive version of the model, in which labs received a penalty 100 times the value of the original “pay-off” for a result that failed to replicate, and replication rates were high (half of all results were subject to replication efforts).The Economist reports Smaldino and McElreath's conclusion is bleak:
that when the ability to publish copiously in journals determines a lab’s success, then “top-performing laboratories will always be those who are able to cut corners”—and that is regardless of the supposedly corrective process of replication.
Ultimately, therefore, the way to end the proliferation of bad science is not to nag people to behave better, or even to encourage replication, but for universities and funding agencies to stop rewarding researchers who publish copiously over those who publish fewer, but perhaps higher-quality papers.Alas, the people in a position to make this change are reached this exalted state by publishing copiously, so The Economist's is a utopian suggestion. In Bad incentives in peer-reviewed science I wrote:
Fixing these problems of science is a collective action problem; it requires all actors to take actions that are against their immediate interests roughly simultaneously. So nothing happens, and the long-term result is, as Arthur Caplan (of the Division of Medical Ethics at NYU's Langone Medical Center) pointed out, a total loss of science's credibility:
The time for a serious, sustained international effort to halt publication pollution is now. Otherwise scientists and physicians will not have to argue about any issue—no one will believe them anyway.(see also John Michael Greer).This loss of credibility is the subject of Andrea Saltelli's Science in crisis: from the sugar scam to Brexit, our faith in experts is fading which starts:
Worldwide, we are facing a joint crisis in science and expertise. This has led some observers to speak of a post-factual democracy – with Brexit and the rise of Donald Trump the results.
Today, the scientific enterprise produces somewhere in the order of 2m papers a year, published in roughly 30,000 different journals. A blunt assessment has been made that perhaps half or more of all this production “will not stand the test of time”.
Meanwhile, science has been challenged as an authoritative source of knowledge for both policy and everyday life, with noted major misdiagnoses in fields as disparate as forensics, preclinical and clinical medicine, chemistry, psychology and economics.Like I did above, Saltelli uses the finance analogy point out the deleterious effect of simplistic metrics - you get what you reward:
One can see in the present critique of finance – as something having outgrown its original function into a self-serving entity – the same ingredients of the social critique of science.
Thus the ethos of “little science” reminds us of the local banker of old times. Scientists in a given field knew one another, just as local bankers had lunch and played golf with their most important customers. The ethos of techno-science or mega-science is similar to that of the modern Lehman bankers, where the key actors know one another only through performance metrics.But I think in this case the analogy is misleading. The balkanization of science into many sub-fields leads to cliques and the kind of group-think illustrated in William A. Wilson's Scientific Regress:
once an entire field has been created—with careers, funding, appointments, and prestige all premised upon an experimental result which was utterly false due either to fraud or to plain bad luck—pointing this fact out is not likely to be very popular. Peer review switches from merely useless to actively harmful. It may be ineffective at keeping papers with analytic or methodological flaws from being published, but it can be deadly effective at suppressing criticism of a dominant research paradigm.Charles Seife's How the FDA Manipulates the Media shows how defensive scientific institutions are becoming in the face of these problems. They are so desperate to control how the press reports science and science-based policy that they are using "close-hold embargos":
The deal was this: NPR, along with a select group of media outlets, would get a briefing about an upcoming announcement by the U.S. Food and Drug Administration a day before anyone else. But in exchange for the scoop, NPR would have to abandon its reportorial independence. The FDA would dictate whom NPR's reporter could and couldn't interview. The FDA isn't the only institution doing this:
This January the California Institute of Technology was sitting on a great story: researchers there had evidence of a new giant planet—Planet Nine—in the outer reaches of our solar system. The Caltech press office decided to give only a dozen reporters, including Scientific American's Michael Lemonick, early access to the scientists and their study. When the news broke, the rest of the scientific journalism community was left scrambling. “Apart from the chosen 12, those working to news deadlines were denied the opportunity to speak to the researchers, obtain independent viewpoints or have time to properly digest the published research paper,” complained BBC reporter Pallab Ghosh about Caltech's “inappropriate” favoritism in an open letter to the World Federation of Science Journalists.But it may be the only one doing it in violation of their stated policy:
in June 2011, the FDA's new media policy officially killed the close-hold embargo: “A journalist may share embargoed material provided by the FDA with nonjournalists or third parties to obtain quotes or opinions prior to an embargo lift provided that the reporter secures agreement from the third party to uphold the embargo.”The downside of the close-hold embargo is obvious from this example:
in 2014 the Harvard-Smithsonian Center for Astrophysics (CfA) used a close-hold embargo when it announced to a dozen reporters that researchers had discovered subtle signals of gravitational waves from the early universe. “You could only talk to other scientists who had seen the papers already; we didn't want them shared unduly,” says Christine Pulliam, the media relations manager for CfA. Unfortunately, the list of approved scientists provided by CfA listed only theoreticians, not experimentalists—and only an experimentalist was likely to see the flaw that doomed the study. (The team was seeing the signature of cosmic dust, not gravitational waves.)Defensiveness is rampant. Cory Doctorow's Psychology's reproducibility crisis: why statisticians are publicly calling out social scientists reports on a response by Andrew Gelman, a professor of statistics and political science, and director of Columbia's Applied Statistics Center, to a screed by Princeton University psychology professor Susan Fiske, a past president of the Association for Psychological Science. Fiske is unhappy that "self-appointed data police" are using blogs and other social media to criticize published research via "methodological terrorism", instead of using the properly peer-reviewed and "monitored channels". Gelman's long and detailed blog post starts:
Fiske doesn’t like when people use social media to publish negative comments on published research. She’s implicitly following what I’ve sometimes called the research incumbency rule: that, once an article is published in some approved venue, it should be taken as truth. I’ve written elsewhere on my problems with this attitude—in short, (a) many published papers are clearly in error, which can often be seen just by internal examination of the claims and which becomes even clearer following unsuccessful replication, and (b) publication itself is such a crapshoot that it’s a statistical error to draw a bright line between published and unpublished work.Gelman's post connects to the work of Smaldino and McElreath:
Fiske expresses concerns for the careers of her friends, careers that may have been damaged by public airing of their research mistakes. Just remember that, for each of these people, there may well be three other young researchers who were doing careful, serious work but then didn’t get picked for a plum job or promotion because it was too hard to compete with other candidates who did sloppy but flashy work that got published in Psych Science or PPNAS. It goes both ways.
Open Knowledge Foundation: Hurdles and joys of introducing Data Journalism in post-Soviet universities
At the Data Journalism Bootcamp organized by the United Nations Development Programme (UNDP), a large part of the participants came from the Commonwealth of Independent States: Belarus, Ukraine, Moldova, Uzbekistan and Kyrgyzstan. Interestingly, the students were deans and professors of journalism departments who had to dive into a new field. The idea of the bootcamp was to understand how to start teaching data journalism at their respective universities. As an assistant of data journalism trainer Eva Constantaras, I was responsible for helping the Russian-speaking part of the class follow the densely packed schedule. Here are some of my thoughts and observations on this challenge.Data journalism as the new punk for traditional societies
‘Data journalism is the new punk’, a metaphor used by Simon Rogers to express that anyone can do it, can take on new meanings when applied to conservative societies. As much as rock music in the 1970s, data journalism seems revolutionary for conservative media and academia. Going against ‘he said-she said’ narratives, it questions the very power of word that journalists have traditionally been so proud of. Moreover, the whole system of journalism education in many post-Soviet countries is still oriented towards literature and humanities rather than math and statistics. Overcoming this barrier is seen as a major challenge by Irina Sidorskaya, Professor in Journalism, Mass Media and Gender at Belarusian State University.Data journalism as the new toolset for investigative reporters
The good news is that, it seems, some countries have been advancing on data journalism without recognising it as such. One of the participants of the training was Cornelia Cozonac, president of the Center for Investigative Journalism in Moldova and senior lecturer at the Free International University. Take a look at this investigation of matching contractors of public procurement and political party donors that she supervised – data journalism in its pure form. Or check this ‘Guess the salary’ news app, production of Ukrainian Texty.org.ua and short-lister of Data Journalism Awards 2016. In countries where investigative journalism is strong, and there is available data to work with which sheds light on areas of public interest, there is only one step left for data journalism to flourish. And this step is training journalism students to be data-driven.Data Journalism as the new discipline for journalism students
In western societies, data journalism has ventured from newsrooms to classrooms, brought mostly by people from other backgrounds, like bioengineering or statistics. UNDP challenged this at the Bootcamp by addressing directly academics of the Commonwealth of Independent States and Balkans.
Is it too early to expect data journalism to be routinely taught in journalism schools?? In my personal view, as a graduate of a classic journalism school, this move is very much needed. By letting data analysis only be performed by great minds and outliers like Nate Silver, we forget that a journalist’s duty is to think critically and orient their stories for the public interest. To live up to this duty, journalists must be equipped with the training and skills required to understand the data, too.Top tips on how to embed data journalism in university programmes (crowdsourced from participants at the Data Journalism Bootcamp)
- Get computer classes to close the skills gap
- Make public awareness campaign, start with your own professors
- invite experts to evangelize the matter
- create pilot classes, winter/summer schools, bootcamps
- harvest or create opportunities for non-formal education like this resource dedicated to data driven journalism or this training (disclaimer: author of this blogpost is involved in both)
- get governments interested and accountable as open data providers
- partner up with leading newsrooms to create a job demand
- cultivate relationships with your own experts
The LITA blog is seeking regular contributors interested in writing easily digestible, thought-provoking blog posts that are fun to read.
If you’re doing innovative work, finding ingenious solutions, or could conceivably be described as a “Library Technology MacGyver” we want you.
We are big on creativity and embracing new post formats, and 2016 saw the first LITA Vlog series, Begin Transmission. Whether you want to write a traditional blog post, create a podcast, host a Twitter chat, or stream a post with Periscope, we are eager to see what you make. Possible post formats could include interviews, how-tos, hacks, and beyond. Your posts can have an immediate impact, and you’ll find an audience of readers in the thousands.
We embrace diverse formats and diverse voices. Library students and members of underrepresented groups are particularly encouraged to apply.
Writers contribute one post per month. A buddy system means you’ll be paired with an experienced blogger to help you draft and review your posts. You’ll join an active, supportive, and talented cadre of writers on whom you can rely for feedback. The average time commitment is between one and two hours a month.
If that sounds like too much of a time commitment, our guest contributor option may be a good option for you.
To apply, send an email to lindsay dot cronk at gmail dot com by Friday, November 11th. Please include the following information:
- A brief bio
- Your professional interests, including 2-3 example topics you would be interested in writing about
- If possible, links to writing samples, professional or personal, to get a feel for your writing style
Send any and all questions my way!
Lindsay Cronk, LITA blog editor
This is my third post about Reveal Digital and On Our Backs. The first post in March outlines my objections with this content being put online. The second post has some contributor agreements I found in the Cornell’s Rare Books and Manuscripts collection and the notes from my talk at code4libNYS.
About a month ago Reveal Digital decided to temporarily take down the On Our Backs (OOB) content. I was happy to hear about this. However I’ve got several concerns about their public statement (PDF). First, I’m concerned that citing Greenberg v. National Geographic Society foreshadows that they are going to disregard contributor agreements and concerns and put the whole collection online. Second, I’m concerned that minors accessing porn is listed ahead of contributor privacy issues and that reflects Reveal Digital’s priorities. Finally, I’m glad that Reveal Digital has broadened their idea of community consultation from financial stakeholders to include publishers, contributors, libraries, archives, researchers, and others, however I’m still worried about whose voices will be centered in these discussions.Copyright
According to Reveal Digital the Greenberg v. National Geographic Society ruling says gives them “the legal right to create a faithful digital reproduction of the publication, without the need to obtain permissions from individual contributors”. ARL has a summary of this case and a 5 page brief written by Ben Grilliot, who was a legal intern for ARL at the time. I’m far from being an expert on US Copyright Law but I understand this to mean that if Reveal Digital digitizes the entire run of OOB without making any changes it doesn’t matter that contributor agreements has limitations. Even if this is legal, it is not ethical.
The ARL summary says “The Copyright Act is “media-neutral,” and libraries believe that it should allow publishers to take advantage of new technologies to preserve and distribute creative works to the public.” I spoke to 3 people who modelled for OOB and none of them consented to have their photos appear online (PDF). As librarians we can’t uncritically fight for access to information, we need to take a more nuanced approach.Porn
I’m puzzled by “minors accessing sexually explicit content” as the first reason Reveal Digital listed. I can understand that this might be a liability issue, but it’s not difficult to find porn on the internet, especially porn that is more explicit and hard core than the images in OOB. I’m confused by this. Reveal Digital describes OOB as filling “an important hole in the feminist digital canon and is an essential artifact of the ‘feminist sex wars'” so for me this is an unexpected reason. Their statement says that they need a window of time to make the necessary software upgrades to solve this issue. I’m disappointed that this reason is given ahead of contributor privacy.Privacy
I was really happy to read how Reveal Digital articulates the importance of contributor privacy:
On the more complex issue of contributor privacy, Reveal Digital has come to share the concerns expressed by a few contributors and others around the digitization of OOB and the potential impact it might have on contributor privacy. While we feel that OOB carries an important voice that should be preserved and studied, we also feel that the privacy wishes of individual contributors should have an opportunity to be voiced and honored.
I feel like the above statement shows that they really heard and understood the concerns that many of the contributors and I had.Community consultation
I’m thrilled to read that Reveal Digital intends to consult with various communities including “publishers, contributors, libraries, archives, researchers, and others”.
Often when people talk about consultations they mention a need to balance interests. We reject that libraries are neutral, so we need to extend that understanding to community consultation processes like these. Contributors, especially many models, could have their lives damaged by this. Researchers seek to gain prestige, grants, tenure and promotion from access to this collection and don’t stand to lose much, if anything. Different communities have a different stake in these decisions. Also, these groups aren’t homogeneous–it’s likely that some contributors will want this content online, some will be OK with some parts, and others will not any of it online. I hope that centering contributor voices is something that Reveal Digital will build into their consultation plan.
This isn’t the first digitization process that has needed community consultation. We can learn from the consultation process that took place around the digitization of the book Moko: or Maori tattooing or around the digitization of the second wave feminist periodical Spare Rib in the UK (thanks Michelle Moravec for telling me about this). Academic libraries can also learn from how public libraries build relationships with communities.
I presented about our Web Committee’s redesign project at Access 2016 in Fredericton, NB on October 5, 2016. We started doing user research for the project in October 2015 and launched the new guides in June 2016 so it took a while, but I’m really proud of the process we followed. Below is a reasonable facsimile of what I said at Access. (I’ll link to the video of the session when it’s posted.)
Our existing subject guides were built in 2011 as a custom content type in Drupal and they were based on the tabbed approach of LibGuides. Unlike LibGuides, tab labels were hard-coded; you didn’t have to use all of them but you could only choose from this specific set of tabs. And requests for more tabs kept coming. It felt a bit arbitrary to say no to tab 16 after agreeing to tab 15.
We knew the guides weren’t very mobile-friendly but they really were no longer desktop-friendly either. So we decided we needed a redesign.
Rather than figure out how to shoe-horn this existing content into a new design, we decided we’d take a step back and do some user research to see what the user needs were for subject guides. We do user testing fairly regularly, but this ended up being the biggest user research project we’ve done.
- Student user research:
- We did some guerrilla-style user research in the library lobby with 11 students: we showed them our existing guide and a model used at another library and asked a couple of quick questions to give us a sense of what we needed to explore further
- I did 10 in-depth interviews with undergraduate students and 7 in-depth interviews with grad students. There were some questions related to subject guides, but also general questions about their research process: how they got started, what they do when they get stuck. When I talked to the grad students, I asked if they were TAs and if they were, I asked some extra questions about their perspectives on their students’ research and needs around things like subject guides.
- One of the big takeaways from the research with students is likely what you would expect: they want to be able to find what they need quickly. Below is all of the content from a single subject guide and the highlighted bits are what students are mostly looking for in a guide: databases, citation information, and contact information for a librarian or subject specialist. It’s a tiny amount in a sea of content.
I assumed that staff made guides like this for students; they put all that information in, even though there’s no way students are going to read it all. That assumption comes with a bit of an obnoxious eye roll: staff clearly don’t understand users like I understand users or they wouldn’t create all this content. Well, we did some user research with our staff, and turns out I didn’t really understand staff as a user group.
- Staff user research
- We did a survey of staff to get a sense of how they use guides, what’s important to them, target audience, pain points – all at a high level
- Then we did focus groups to probe some of these things more deeply
- Biggest takeaway from the research with staff is that guides are most important for their teaching and for helping their colleagues on the reference desk when students have questions. Students themselves are not the primary target audience. I found this surprising.
We analyzed all of the user research, looked at our web analytics and came up with a set of design criteria based on everything we’d learned. But we still had this issue that staff wanted all the things, preferably on one page and students wanted quick access to a small number of resources. We were definitely tempted to focus exclusively on students but about 14% of subject guide use comes from staff computers, so they’re a significant user group. We felt it was important to come up with a design that would also be useful for them. In Web Committee, we try to make things “intuitive for students and learn-able for staff.” Student-first but staff-friendly.
Since the guides seemed to have these two distinct user groups, we thought maybe we need two versions of subject guides. And that’s what we did; we made a quick guide primarily for students, and a detailed guide primarily for staff.
We created mockups of two kinds of guides based on our design criteria. Then we did user tests of the mockups with students, iterating the designs a few times as we saw things that didn’t work. We ended up testing with a total of 17 students.
Once we felt confident that the guides worked well for students, we presented the designs to staff and again met with them in small groups to discuss. Reaction was quite positive. We had included a lot of direct quotations from students in our presentation and staff seemed to appreciate that we’d based our design decisions on what students had told us. No design changes came out of our consultations with staff; they had a lot of questions about how they would fit their content into the design, but they didn’t have any issues with the design itself. So we built the new guide content types in Drupal and created documentation with how-tos and best practices based on our research. We opened the new guides for editing on June 13, which was great because it gave staff most of the summer to work on their new guides.Quick Guide
The first of the two guides is the Quick Guide, aimed at students. I described it to staff as the guide that would help a student who has a paper due tomorrow and is starting after the reference desk has closed for the day.
- Hard limit of 5 Key Resources
- Can have fewer than 5, but you can’t have more.
- One of the students we talked to said: “When you have less information you focus more on something that you want to find; when you have a lot of information you start to panic: “Which one should I do? This one? Oh wait.” And then you start to forget what you’re looking for.” She’s describing basic information overload, but it’s nice to hear it in a student’s own words.
- Some students still found this overwhelming, so we put a 160-character limit on annotations.
- We recommend that databases feature prominently on this list, based on what students told us and our web analytics: Databases are selected 3x more than any other resource in subject guides
- We also recommend not linking to encyclopedias and dictionaries. Encyclopedias and Dictionaries were very prominent on the tabbed Subject Guides but they really aren’t big draws for students (student quotations from user research: “If someone was to give this to me, I’d be like, yeah, I see encyclopedias, I see dictionaries… I’m not really interested in doing any of these, or looking through this, uh, I’m outta here.”)
- Related Subject Guides and General Research Help Guides
- Link to Detailed Guide if people want more information on the same subject. THERE DOES NOT HAVE TO BE A DETAILED GUIDE.
- Added benefit of the 2-version approach is that staff can use existing tabbed guides as the “Detailed Guides” until they are removed in Sept.2017. I think part of the reason we didn’t feel much pushback was that people didn’t have to redo all of their guides right away; there was this transition time.
- From a design point of view, the Detailed Guide is simpler than the Quick Guide. Accordions instead of tabs
- Students all saw all the accordions. Not all students saw the tabs (that’s a problem people have found in usability testing of LibGuides too)
- Default of 5 accordions for the same reasons that Key Resources were limited to 5 – trying to avoid information overload – but because target audience is staff and not students, they can ask for additional accordions. We wanted there to be a small barrier to filling up the page, so here’s someone adding the 5th accordion, and once they add that 5th section the “Add another item” button is disabled and they have to ask us to create additional accordions.
- There’s now flexibility in both the labels and the content. Staff can put as much content as they want within the accordion – text, images, video, whatever – but we do ask them to be concise and keep in mind that students have limited time. I really like this student’s take and made sure to include this quotation in our presentation to staff as well as in our documentation:
- When I come across something… I’ll skim through it and if I don’t see anything there that’s immediately helpful to me, it’s a waste of my time and I need to go do something else that is actually going to be helpful to me .
And speaking of time, thank you for yours.
In the Library, With the Lead Pipe: Putting Critical Information Literacy into Context: How and Why Librarians Adopt Critical Practices in their Teaching
Critical information literacy asks librarians to work with their patrons and communities to co-investigate the political, social, and economic dimensions of information, including its creation, access, and use. This approach to information literacy seeks to involve learners in better understanding systems of oppression while also identifying opportunities to take action upon them. An increasing number of librarians appear to be taking up critical information literacy ideas and practices in various ways, from cataloging to reference. But what does it mean to make critical information literacy part of one’s work? To learn more about how and why academic librarians incorporate critical information literacy into their classroom instruction, I interviewed thirteen librarians living in the United States. This article describes why and how these individuals take a critical approach to teaching about libraries and information, including the methods they use, the barriers they face, and the positive influences that keep them going.
In these days of mass surveillance and the massive transfer of public goods into private hands, citizens need to know much more about how information works. They need to understand the moral, economic, and political context of knowledge. They need to know how to create their own, so that they make the world a better, more just place.
– Barbara Fister, “Practicing Freedom in the Digital Library: Reinventing Libraries” (2013)
Like many other librarians who teach, I stumbled upon my newfound job duties having no formal training or experience as a teacher. I led the students brought in by their professor through a maze of databases, books, and services within the space of one hour as best I could, and students were often sympathetic but uninterested. Even at this early stage in my career, while students quickly packed up their belongings and filed out of the library classroom, I couldn’t help but wonder if something crucial was missing. I realized I was having difficulty squaring the big reasons I became a librarian–to advocate for and widen people’s access to information, to find ways to contribute to the well-being of communities through a commitment to social responsibility, to fight as one of the last holdouts in a society where “sharing” and “free” are becoming endangered terms–with my primary responsibility as a teacher of, as one student put it, “how to do the library.”
I eventually found what I was searching for, but it took some time. I started to learn about the work that had been taking place in critical information literacy, and critical librarianship more broadly, that had been occurring for decades. I looked into examples of radical librarianship and activism that were sometimes mentioned fleetingly in my MLS program but more often absent altogether, like the efforts of Sanford Berman to identify and update or remove derogatory and harmful Library of Congress subject headings (1971), the Radical Reference collective (Morrone & Friedman 2009), and the Progressive Librarians Guild (PLG), founded in 1990. This long tradition of social justice work being done in and outside of libraries by all types of librarians led to my discovery of critical information literacy. I have briefly addressed some of the many inspirational books and articles of critical librarianship and critical information literacy (Tewell 2015), but the most impactful works for me personally were by Maria Accardi, Emily Drabinski & Alana Kumbier (2010), James Elmborg (2006), Heidi Jacobs (2008), Maria Accardi (2013), and Robert Schroeder (2014). Many of these are from Library Juice Press, a publisher specializing in library issues addressed from a critical perspective.
Critical information literacy aims to understand how libraries participate in systems of oppression and find ways for librarians and students to intervene upon these systems. To do so, it examines information, libraries, and the work of librarians using critical theories and most often the ideas of critical pedagogy. As stated by Lua Gregory and Shana Higgins, critical information literacy “takes into consideration the social, political, economic, and corporate systems that have power and influence over information production, dissemination, access, and consumption” (2013). Inspired by the books and articles I had been reading, I started using different critical information literacy approaches in my classes. As I tried these methods I became increasingly interested in how other librarians made critical information literacy part of their teaching. Having firsthand experience making critical information literacy “work” as a teacher and seeing students truly engaged with their learning and topics that matter to them was very transformative for me. Seeing the change that critical information literacy can make, and the ways it deepened how I approached my interactions with students, led to me being passionate about its potential.
I interviewed 13 librarians working in a variety of academic institutions, living in different regions within the United States, and with varied ages, ethnicities, genders, and ablednesses, to see how they practice critical information literacy within library instruction. I spoke with them by email and via Skype, and asked questions about how their teaching is influenced by critical information literacy, what barriers they faced, what benefits they saw, and what factors allowed their teaching to continue or even flourish. In her recent book Critical Information Literacy: Foundations, Inspiration, and Ideas, Annie Downey argues, “Critical information literacy looks beyond the strictly functional, competency-based role of information discovery and use, going deeper than the traditional conceptions of information literacy that focus almost wholly on mainstream sources and views” (2016). At its core, critical information literacy is an attempt to render visible the complex workings of information so that we may identify and act upon the power structures that shape our lives. How this may actually be done within our libraries is what I wished to investigate in this project.How librarians learned about critical information literacy
Critical information literacy has become better known due to the efforts of many that started writing and thinking about it years ago. This shift from the margins toward the center that critical approaches to librarianship have made is observed by librarian Emily Drabinski in a recent keynote (2016). The interviewees I spoke with learned about critical information literacy in a variety of ways. How you learn about something can in many ways shape how you come to understand it, so I wanted to address this first to set the stage for other questions.
Most interviewees learned of critical information literacy from a colleague either in their workplace or at another library, coupled with an article or book related to the subject. This is described by one librarian who was searching for readings that discussed the cultural aspects of information literacy. Upon reading an article by Cushla Kapitztke recommended by a colleague, this librarian said, “the whole world stopped around me. And I just was just blown away, and I’d never read anything like it…it really spoke to me.” Indicative of critical information literacy’s growing popularity, two interviewees learned about the topic at conferences and unconferences, such as the first #critlib unconference. Though three interviewees discovered critical information literacy during their MLIS programs only one did so through a formal class, while one was through a self-directed research paper and another was by preparing a bibliography while working as a graduate assistant. One librarian’s critical practice at her diverse public university was informed by her background in anthropology, and she identifies the Ferguson uprising as the point where she began discussing social justice issues in relation to information in her classes. She soon after found that library workers were discussing these issues on Twitter at the #critlib hashtag and “realized after the fact that what I was doing fit into this CIL [critical information literacy] approach that was already in place.”
Coursework in areas other than librarianship was key for some interviewees, who learned about critical pedagogy or critical theory before finding an article or book related to critical information literacy. One librarian learned about critical information literacy while doing research for doctoral coursework after having read Paulo Freire and in particular Myles Horton, whose work with poor and undereducated people in rural Tennessee caused her to draw connections with her own work with underprepared students: “Importantly, the community (i.e. students) identified the skills they needed to learn. I began to see information literacy as one of those skills that is truly fundamental to living and working today. This idea really pushed me past what I had previously thought information literacy was.” Other librarians related their educational backgrounds in English, Women’s Studies, and Social Studies as priming them for the ideas of critical pedagogy as applied to information literacy.
A majority of interviewees learned about critical information literacy relatively recently, between 2011 and 2014. Yet three librarians, at a community college, four year university, and liberal arts college, mentioned they had already been practicing these same types of ideas before learning about the term, showing that one may very well use critical information literacy approaches without being aware of the name: “CIL felt like a natural extension of what I had already been doing, and I imagine I’d be practicing critical librarianship/IL even if it weren’t something of an established subfield of information literacy.”How critical information literacy can be incorporated into classes
To gather a sampling of ways these librarians brought critical information literacy into their teaching, I asked them about a time when they incorporated critical information literacy into a class. The interviewees shared a wealth of examples for single sessions and credit-bearing courses, and a few of my favorite examples follow, which I appreciate for their creativity, applicability to a variety of settings, and potential for involving learners with critical concepts. It is important to note that librarians’ identities shape the ways they are able to pursue their work. What one librarian with marginalized identities may be able to discuss in terms of politically-oriented topics with students or negotiate with course instructors in terms of class content will differ from librarians with privileged backgrounds. Librarians with marginalized identities are more likely to face challenges in actualizing their critical information literacy practice.
One interviewee at a regional public university campus described an activity that asks students to explore library databases and present them to the class. This easy-to-implement idea turns the tables on lecture-dominated library instruction, and asks students to not just be involved, but to share their knowledge. As this librarian described:
When I do this activity, I don’t even turn on my projector to show [students] stuff on the screen to get them started–I just have them jump right in, even if they don’t really know what they’re doing. Relinquishing control of the demonstration disrupts the teacher/learner hierarchy of power and places power in the learner’s hands…it shows [students] that they have knowledge that is worth sharing and that they, too, can have power to speak and guide and teach. Their voice matters. The idea, of course, as it that this leaks out of the walls of the classroom and into their lives and worlds.
Another librarian at a small college discusses power with students in terms of viewpoints represented within a database. “I love to talk about the role of power in information structures with students. One of the best ways to do this is to talk about what and who is and is not represented and why.” This librarian continues, “One way I’ve done this is to do a pre-search in a database on a topic with a bit of controversy and see if I can get a results list that is eye-opening…I had students look at the results list and evaluate the first 3-5 results and then we discussed their evaluation process…we talked about how information is created and who does the creating, including looking at who was funding the research in the peer review journals and who had ads in the trade journals.” This idea uses a database and the sources within to generate conversation and dialogue, and relates the evaluation process to the students instead of an external checklist.
One interviewee began a class discussion with a role-playing scenario: “Instead of simply demo-ing a database, I facilitated a role-playing activity in which [students] assumed the roles of scholars, and we then had a discussion about who gets to be a scholar and thus who has a voice in the literature. This was all new to them, and I think they were able to both understand what ‘the literature’ is and problematize academia in ways they hadn’t before.” She further explained, “when I did show them how to use a database, I was able to bring to their attention to the ways in which information organization (subject headings) are also problematic, particularly when it comes to gender identity and sexuality…I think this one-shot was critical in that is not only allowed students to peek ‘behind the scenes’ (as far as how information is produced and organized in academia), but it also troubled these processes.”
These ideas sometimes come in flashes of brilliance, but more often they are the result of trying something small, revising it, and trying again. One librarian at a large research university began by carefully considering the language she used and how she applied her authority as a teacher. She followed these reflective practices by asking discussion questions regarding whose voices are missing from discovery systems, whether Google or a library catalog, and found that students responded to these questions with interest. This then led to the more intentional creation of instruction sessions centered around critical topics and discussions. Several interviewees mentioned they had been unsure whether they were “allowed” to do this type of instruction, particularly those librarians who began thinking about critical information literacy a few years ago when literature and conversations regarding the topic were scarce. Starting out small may help with these feelings of uncertainty, even though, as two interviewees pointed out by relating examples of when their classes did not go as planned, all teaching is difficult and critical information literacy instruction can be particularly demanding due to the emotional investment it often requires.How classroom methods are used to practice critical information literacy
With an understanding of the ways critical librarians taught classes using a critical information literacy approach, I was curious whether they found that particular teaching methods were conducive for doing so. Their responses revealed a number of commonalities as well as some unique ideas. Looking over these different methods, they demonstrate that critical information literacy has the potential to uncover and question some very big issues and norms while simultaneously being something that is very do-able.
Creating opportunities for interaction between the librarian and students was a frequent goal. Nine interviewees mentioned class discussions as their teaching method of choice. One librarian spends a great deal of effort fostering discussions, stating, “I spend more time now developing the questions I am going to ask than any other part of my planning because if you don’t ask the right questions, the conversation never reaches the level it needs to.” Another librarian at a comprehensive public university carefully centers student questions: “One method I use when I teach many of my classes for graduate students or doctoral students is I base the class on their questions…I give them time to talk amongst themselves about what they want to know, then I ask them. I write their questions on the board and tell them I’ll base the class on these questions, and that they should ask more if they have them.” This method “shows the students I want to try to answer their questions – they are the most important. It also parallels the kind of work they will see the librarians do with them at the reference desk or in individual research consultations.”
Certain activities and teaching techniques were shared by interviewees, including the Jigsaw Method which one librarian has “experimented over the past several years…using small groups that then convene into a larger group to guide conversations about exploring databases and evaluating sources–and not just evaluating, but collaboratively developing criteria to evaluate,” and activities that range from “group work exploring a variety of sources surrounding the murder of Trayvon Martin to acting out a scholarly debate on the coming out process.” Others had success using search examples to introduce critical ideas, such as prison abolition or Black Lives Matter. “I always try to use a search example/keywords/ideas that, hopefully, will expose students to a set of results that gets them thinking about an important topic,” one interviewee stated. For example, in an online tutorial one librarian used the research question, “How does air quality affect women’s health?” which is relevant to their student population in terms of geography, health, and economic disparity, but also draws attention to the gendered dimensions of environmental racism.
Another method for teaching critical information literacy is adapted from Paulo Freire’s concept of problem-posing, wherein teachers and learners co-investigate an issue or question of importance to them. One librarian at a small college described a successful example of problem-posing in library instruction, noting that she has “started asking the faculty to help me think of a problem the class could work on together, which I think is the best thing to have happened for my teaching in a long time.” Noting that it took her a great deal of time to reach the point where she is confident in asking faculty for something in exchange for helping their students, she describes an example:
[We looked into] when a specific law was passed and who the primary players were in passing the law. This sounds simple, but there was misinformation all over the place about this law. The Wikipedia entry was wrong and had been cited over and over so the wrong date was starting to appear as the “official date.” This was wonderful for our purposes because we had conversations about source type, government documents, how information gets perpetuated, sourcing and evaluation, etc. Essentially, we are able to problematize information consumption and dissemination with this one little question. The students were very into it.
A small but meaningful change to a common part of library instruction–an overview of the services a library provides–was described by one interviewee: “Instead of telling students about the services that we have, I might actually have the students find one or two things that they didn’t know about the library, and share with each other.” This is a way to “change the expectation that I’m going to be the person standing there and telling them what they need to know. That they can also, you know, construct their own knowledge. And perhaps learn, with maybe my guidance, learn from each other.” More than a third of the librarians I spoke with found that reflection was key to their instruction. “Allowing students time to reflect or posing questions that ask them to consider how/if the lesson is meaningful to them is an important part of the classroom experience for me,” one person affirmed. “Ideally, it adds a small jolt to their experience and communicates to students that I’m here for them, that I want to be useful and a purposeful addition to their classroom, not some intruder with my own agenda.”
The difference in critical approaches to IL does not always relate to the method, and is instead more likely to be based upon the social and political orientation of one’s instructional aims: “My set of methods haven’t changed that much since I started to shift to a more critical focus, it’s the topics that we are discussing that have shifted.” One interviewee describes what worked and what they would like to further pursue in a credit-bearing course, which corresponds to the demographics of their institution:
[A] discussion that went fairly well but that I would like to explore more next year revolved around the idea that publishing academic stuff is a mechanism for someone or a group to gain/earn authority or academic legitimacy. I had walked students through the peer review process, why it is important yet flawed, who engages in it and why, etc. Then, because another course outcome dealt with understanding “the disciplines,” we moved to the history of Chicano/a studies and its struggle for legitimacy in the academy. One of the ways in which it contributed to academic discussions/developed a canon and thus gained legitimacy in the world of higher education, was by establishing its own scholarly journals. My class discussed this a bit, but I’d really like to make this the focal point of my course next year.How theoretical understandings inform the practice of critical information literacy
In order to better understand the various ways the librarians I interviewed thought of critical information literacy, I asked if there were theoretical or conceptual understandings that influenced their work. I clarified that these could be theories, ideas, or writings related to education, social justice, libraries, or other things meaningful to them. Many interviewees conceived of their teaching aims in terms of critical pedagogy. As described by Lauren Smith, critical pedagogy argues that “learners can only truly learn to think critically if they are also able to challenge the problems within power and knowledge structures in their educational environment as well as the wider world” (2013, p. 19). For Alana Kumbier, “Critical pedagogy offers tools we can use to denaturalize and evaluate phenomena that are often understood as inevitable, like economic or cultural globalization, or natural, like a binary sex/gender system, or just accepted, like the authority of an encyclopedia entry” (2014, p. 161).
Many interviewees cited readings that influenced how they thought about the goals and realities of formal education–what one person referred to as the “classics” of critical pedagogy. “While I was in grad school we read Freire, and Giroux, all those sort of classics. bell hooks, Teaching to Transgress, I loved too…If someone else were to say to me, ‘Oh, I’m sort of interested in this critical pedagogy, what is this about?’ those would be the things I would pull off my shelf with excitement and say, ‘You have to read these!’” This same interviewee was also influenced by critical race theory, finding it extremely useful for ideas about building inclusive classroom environments. Critical race theory was discussed by four interviewees, and the Handbook of Critical Race Theory in Education (Lynn and Dixson, 2013) was mentioned as one key resource in this area.
Feminist pedagogy was another area of theory and practice that inspired critical librarians. “For critical pedagogy and theory, I really respond to Freire, Mezirow, Shor, and hooks,” one interviewee replied. “And then feminist pedagogues who have problematized critical pedagogy like Jennifer Gore, Elizabeth Ellsworth, and Patti Lather. I like the critiques of critical pedagogy by these authors because they address the fact that it is really hard to pull critical pedagogy off in our institutions.” Maria Accardi’s 2013 book Feminist Pedagogy for Library Instruction was mentioned by four interviewees as directly informing their practice, particularly feminist pedagogy as an educational approach that “honors student experience and voice, has social justice aims, and is attentive to power dynamics in the classroom.”
One librarian’s upbringing and personal sense of social justice fundamentally informs her work: “I think I am primarily motivated by a strong inner sense of social justice more than anything. I learned about Paulo Freire and the banking method and conscientização [critical consciousness] and all of that…which helped to align my already innate social justice framework with the educational environment. But that sense that I am here on this planet to help make it better was already inside of me…my social justice framework is profoundly informed by my Catholic upbringing and education.” Another interviewee found student rallies and local issues on their campus, such as the cutting of positions in the university’s Ethnic Studies department, to be impactful on a personal level. The academic disciplines interviewees studied as undergraduate or graduate students were also highly influential, whether literature, cultural studies, anthropology, or journalism.How critical information literacy is beneficial
As bell hooks writes, because “our educational institutions are so deeply invested in a banking system, teachers are more rewarded when we do not teach against the grain” (1994, p. 203). Why do librarians teaching against the grain choose to do so? What impact might this approach to teaching make? One question I asked interviewees was whether they find critical information literacy beneficial. The reasons they gave were largely related to the engagement they saw with students as well as their own newly discovered or rediscovered commitment to their work as librarians.
Brian Kopp and Kim Olson-Kopp argue that “the development of critical consciousness in a library setting depends first and foremost on humanizing, or putting a face on, research, and grounding it in the realities which shape it” (2010, p. 57). For the librarians I spoke with, having a basis in critical information literacy enabled them to be more engaged teachers who were able to bring their whole selves into the classroom. “I don’t think I’d still be doing what I’m doing if I hadn’t learned or figured out that I could use critical information literacy in the classroom, because I would be so burned out and bored by point-here-click-here teaching,” one interviewee states. Another librarian found the demands of this type of teaching has made them a better instructor: “It forces me to be self-reflective and challenges me to go outside of my comfort zone of knowing exactly what to do in front of a class. I’m definitely uncomfortable when I don’t have a concrete plan, but I believe that it really does benefit students more when I’m not following a rote plan and can instead allow for diversions.” These benefits of critical information literacy for instruction librarians are related to those for students, in particular fostering a sense of purpose and meaning.
Critical information literacy’s “student-centered emphasis” was influential for one interviewee, which has “meant moving beyond simply discussion or inquiry-based learning, and really bringing the students’ needs or knowledge or perspective to the fore, as much as possible.” Sincerely valuing student knowledge and the perspectives they bring, as well as finding ways to make this knowledge a meaningful part of classes, was discussed by several librarians. This focus on student-centeredness and its effects is described by one interviewee:
When classes are conducted in critical ways I think students get to hear their own voice and hear their own experiences validated. They see themselves, their whole selves, as part of the academic enterprise, an enterprise that they can change for the better. They can realize that the questions that really matter to them, many of which relate to power, are valid academic questions. It can help them make sense of the strange new place (to many) called the university.
Annie Downey notes that one “issue that arises from librarians’ lack of teacher training is that they struggle with finding ways to make their instruction meaningful to students. They often confront the problem of students being unaware to relate the information they are supposed to learn in library instruction sessions to what they may be doing in their classes or to their lives in any meaningful way” (2016). For critical librarians and their students alike, it appears that critical information literacy is one way to provide this meaningfulness.
One unexpected finding was that critical information literacy’s associated ideas and approaches to teaching was a way for librarians to connect with faculty and course instructors in a wide range of institutional settings. Many of these connections were related to shared pedagogical approaches or interests. One interviewee in a large research university said critical information literacy helped her “build a bridge” with teaching faculty and instructors in that these critical approaches to information act as “a shared language that we now have,” and as such, is a positive in developing collaborations such as alternative research assignments. For another librarian in a small liberal arts college, critical information literacy is a way to begin the conversation of going past the one-shot instruction model. “When someone comes to you and says, ‘I want you to come to my class and do a demo of JSTOR, can we have fifteen minutes of your time?’…I can comment back and say, ‘How about, if instead, we do this?’…Critical information literacy is really helpful to open up that dialogue again in a more meaningful way. And talk about the goals of instruction, and not just sort of fall back to that familiar routine and model.” Another interviewee found it a way to connect with faculty regarding shared interests in social justice issues: “As I’ve been exploring CIL I’ve been continually amazed at how many other faculty on campus come from a critical or social justice background! CIL helps me make immediate and deep connections with the faculty I relate to and with whom I work with as I teach IL sessions.”
One librarian at an urban public university observed how the campus setting impacted their teaching and the interests of students and faculty: “We’re a campus that’s really not far from the center of the city, where there are huge protests that happening downtown…even if I am not talking about those things, because…I don’t teach a class on my own, students might be thinking about these issues anyways in one their other classes, or maybe in the work that they do outside of class. So I think all of that makes it more easier for me to incorporate critical information literacy. Because, I feel like they also get it…they understand.” She added that instructors at her institution also generally think about the same types of political issues happening, such as protests and social movements occurring both on-campus and off. Another librarian in a different setting found inspiration for her classes at a small private college in a rural area through an event that transpired near their campus. The U.S. Federal Government conducted a raid of immigrant workers at a meatpacking plant, which was the largest at that point in time. In discussing this raid, the librarian asked students to gather materials from a variety of sources, including the College Archives, that documented or addressed the raid and its implications, and to evaluate what arguments each source was making as well as how it might be useful to or not useful to students’ research. For these librarians, a great deal of inspiration was found in events that impacted their nearby community and were important to students.How barriers shape the practice of critical information literacy
The challenges faced by critical information literacy practitioners are important to consider because they identify obstacles that may then be more easily recognized and addressed. To learn more about challenges critical librarians face, I asked about barriers they experienced in making critical information literacy part of their classroom practice.
Far and away, the one-shot instruction model and a lack of time were the primary barriers. This is due in part because critical information literacy requires a significant time investment: “It takes more time to enact critical information literacy instruction–time to plan, time to reflect. This is not the kind of teaching you can do on autopilot.” Another interviewee makes an important distinction between finding time and making time: “I don’t think we ever ‘find’ the time, we can only make the time for things we think important. So making the time to make critical information literacy important is the key.” Although prioritizing what one finds important in their work is important, a related challenge is that of the single instruction session that many academic librarians make do with. “Obviously, if that’s the best you have to work with, I fully encourage anyone to do what they can,” one person says. “But it takes a certain degree of trust for students to take risks and challenge hegemonic assumptions – and showing up to one class session is not enough time to develop that trust.” Five other interviewees made similar remarks about the limitations of this common teaching scenario.
Having the confidence or courage necessary for a critical approach to teaching as well as the support essential to do so was also discussed. This was sometimes scary for one librarian: “When you’re committing to a way of teaching and learning that is mostly outside the norm, or outside what you’re most familiar with, it can feel scary,” while another interviewee noted how colleagues’ perspectives and their newness to a job impacted their ability to practice critical information literacy: “I struggle to feel confident in my professional actions in general, particularly when they differ from what my co-workers are doing or from what I’ve done in the past. Add to that the fact that I’m still new to my position…and that can lead to a lot of questioning and second-guessing on my part.” Others described being the only one at their institution who teach using critical information literacy as lonely, since “People don’t always get what you’re trying to do, even if you try to explain it to them.” An interviewee at a large institution felt confident in her teaching practice but misunderstood by colleagues, explaining that they “think that I’m wasting my time, or that I’m just being a bit too much of like a warrior, and take myself too seriously.”
Critical librarians also faced resistance from students, for a variety of reasons. Most often was because critical information literacy is not always comfortable or enjoyable for students who are accustomed to lectures and passive means of education:
Students don’t always recognize critical pedagogy as teaching, because it doesn’t look like most of the teaching they’ve experienced before. And maybe they don’t want to be actively engaged; maybe they just want to be lectured to, to be passive. Similarly, the teaching faculty member also may not recognize or understand what you’re doing as real actual teaching and may try to interfere or undermine you while you’re in the middle of teaching.
While interviewees typically found a few students in their classes willing to engage, getting all students on board and changing their expectations, especially within a 50 to 75-minute session, proved difficult. Another concern was balancing critical information literacy topics and methods with immediate student needs and course instructors’ expectations: “This is a difficult balance because not only do I feel pressure to give the instructor and students what they want and expect (database demonstrations), I really need them to understand the practical use of this information and how to be engaged actors within their learning experience.” Helping students succeed in their academic work so they can complete their assignments, receive their grades, and attain the degree they seek is by no means incompatible with the goals of critical information literacy, but requires attention of its own. In an article on the tension between neoliberal definitions of student success and critical library instruction, Ian Beilin notes the necessity of reconciling student preparation and the demands they must meet with teaching approaches that emphasize broader structural ideas: “Especially for first-generation students, students of colour, and working-class students, librarians have a responsibility to teach skills, so many of which more-privileged students have already acquired” (2016, p. 17). One possibility for meeting this challenge is for library educators to “encourage alternative definitions of success while at the same time ensure success in the existing system,” which can be a necessary but difficult balance to strike (2016, p. 18).
Apart from some experiences with student resistance and pedagogical challenges, the librarians I spoke with also faced difficulties in terms of faculty expectations of what information literacy instruction is or could be. As noted by one interviewee: “I had an activity where the students were evaluating different sources and sharing, like going up to the front of the class and sharing what they found, and demoing the resource themselves…and the instructor just cut them off. Because they wanted just the traditional librarian standing up there, telling them where to click.” Several librarians stated the biggest barrier to critical information literacy was faculty and course instructor expectations. Part of the reason for this lies in the power differentials between librarians and faculty, described by a librarian at a small liberal arts college:
I move carefully with faculty because they are the ones holding the power. If I want to have them bring their classes in and send their students to me, I have to be very respectful of their wishes. I am even more careful with more established faculty. At a small college, especially, a wrong move with the faculty can seriously undermine your ability to do anything with a specific department or can appear to reflect badly on your performance in the eyes of supervisors in the library.
Related to the challenge of faculty expectations is that of course instructors impinging on librarians’ classroom decisions or interjecting themselves into discussions. One interviewee noted the raced and gendered dynamics among librarian teachers and faculty, and in particular the tendency for certain faculty to interrupt librarians when they are in the middle of teaching: “My supervisor who’s an older white woman…she can really command a class. She’s also an excellent teacher. But would an instructor tell her to stop doing what she’s doing? I don’t think so….Someone like me, I look young, and am a woman of color.” This observation draws attention to the challenges that librarians with marginalized identities are likely to face in classroom environments.
Beyond student resistance and faculty expectations, the greater domain that education takes place within was discussed as a challenge. The increasing corporatization of higher education and expectations fueled by universities that students “invest” in an education in order to receive a “return” such as a high-paying job is one major factor at odds with the goals of critical information literacy: “Though we get great support and have the enthusiastic backing of the administration, it’s impossible to ignore that there is an explicitly career-oriented education on offer here. This undermines efforts to make the student the subject of the learning process rather than the object and certainly cuts against any effort to liberate learning from creeping corporatization.” A related issue is the culture of assessment embraced by many universities. The values of assessment and reporting and the tension with critical education approaches is related by one interviewee: “I’ve chaired our campus Academic Assessment Committee before; I fully understand the stakes of assessment in higher ed in a state-funded institution. Assessment culture privileges ways of teaching and learning that are quantifiable. I can’t put ‘changed lives and enacted social change’ on a rubric, but I am pressured to report student learning findings in ways that are rubric-able.”How factors contribute positively to the practice of critical information literacy
While critical librarians face significant challenges, it is important to also understand what factors enable their practice and help it to thrive. The librarians I interviewed identified several elements: seeing critical information literacy work–something that has been a big contributing factor in my own practice–having a community of librarians who are attempting similar things, having colleagues at their own workplace to talk and collaborate with, and online spaces for discussion such as #critlib on Twitter.
“When you know you’ve been successful, that you have had an impact on the student in some way, this reinforces the practice of this kind of teaching in a very positive, affirming way,” one librarian wrote. “Knowing I’ve had an impact, for me, is more through informal observation and conversations, through the questions students ask, or what they write down on the 3-2-1 sheet I use a lot.” Seeing critical information literacy work in their classes, and knowing firsthand the difference that this approach can make, was cited by several interviewees as being a major positive factor. As one person stated, “the biggest thing is my past experiences where it’s worked. Where things have happened in a classroom and people have said things that I never, ever, ever, would have thought or said.” These opportunities for unexpected and authentic conversations inspired several critical librarians. In considering the challenges faced in her critical and feminist pedagogy, Maria Accardi writes, “What gave me hope, what kept me going, what helped me remember that feminist teaching is worth the effort and difficulty, was that even amongst all my failures and flops, there were shining moments of success” (2013, p. 3).
Librarians need to not just see that this approach to teaching works for them and their students, but to be supported in their efforts. For some librarians I spoke with, this meant finding a community of colleagues interested in critical librarianship. “Being connected to a like-minded community who also cares about critical information literacy instruction is hugely important. Without having anyone else to talk to about it, I don’t know if I’d have the fortitude to keep doing what I’m doing,” notes one interviewee. Another librarian came to the same conclusion: “The biggest positive factor is talking to other librarians trying to do the same thing. This is wonderfully beneficial at conferences, but even better was when people at my institution got interested.”
In a similar way that finding a community of librarians helped foster librarians’ critical practices, locating colleagues at one’s own institution, inside the library or out, was important for the same reasons. One interviewee identified this as building allies: “A big thing for me is when I can build allies. Whether that’s with the course instructors, or in the library. So I tend to sort of, try to find people, try to find instructors who I know are engaged with Women’s and Gender Studies, or some other kind of department that thinks about these kinds of things and uses these methods already.” Five other interviewees noted that they made strong connections with faculty colleagues and this contributed positively to their critical information literacy efforts. In describing the experiences of critical librarians she interviewed, Downey found a similar theme: “Even having just one other person at their institution who is cognizant of and uses critical information literacy can make a big difference for librarians’ comfort level with critical approaches and content” (2016, p. 141). At the same time, Downey makes a strong case for the necessity of librarians, solo or with others, working to change the direction of teaching at one’s institution, for “teachers inspire other teachers and trust other teachers” (2016, p. 145).
One tool that interviewees used to connect with others was the “critlib” hashtag on Twitter. Short for “critical librarianship,” this hashtag is used for chats on a variety of topics as well as a way to converse with other librarians following the hashtag. One interviewee discusses the helpfulness of #critlib in establishing an online meeting place for critical librarians: “The #critlib community and artifacts they create (conferences, website, etc.) have been really helpful, not necessarily for things like lesson planning or creating activities, but for giving me content to think about that I can then integrate into the classroom…It’s also really nice to know there are people out there thinking and excited about the same things as me.” For one librarian at a liberal arts college, #critlib provided an umbrella for different theories and approaches that could later be applied: “I sort of fumbled my way through information literacy at first, and that’s why I find #critlib very helpful. The nice thing about #critlib is that it has help [sic] provide a (loose) framework for bringing together many different theories and approaches that could be considered critical.”Conclusion
The librarians I interviewed appreciated the fact that this approach to librarianship has been blooming. This shift was described by one librarian who has been interested in critical librarianship for several years: “The Progressive Librarians Guild and Library Juice Press have been going for some time, but criticality seems to be flowering these days. Just a few years ago I was looking for a conference to go to actually talk to some live folks about critical pedagogy. I looked all over the web in all sorts of disciplines and countries and didn’t hardly find anything, and definitely nothing LIS related.” But now there are many events, from those on critical librarianship specifically, such as the first #critlib unconference, events organized by the Radical Librarians Collective in the UK, the 2016 Critical Librarianship and Pedagogy Symposium, and the Critical Information Literacy Unconference held prior to the European Conference on Information Literacy, to the presence of critical sessions in large American conferences such as ACRL and ALA.
In noting the increasing popularity of critical information literacy, one interviewee urged librarians to continue applying critical thought to other areas as well: “Instruction doesn’t stop at the classroom door, and then this whole process of conscientization doesn’t just stop there as well.” Critical information literacy is not limited to teaching, and thinking broadly about the implications of libraries can encourage positive changes. Relatedly, critical librarianship must be informed by diverse perspectives. The issue of perceived barriers was brought up as one problem: “the biggest concern that I have…is that there are these perceived barriers, and that there is a price of admission. That you cannot be a critical librarian until you’ve read these six books, or have this degree, or this background. That really bugs me. Just because it’s the antithesis of what critical pedagogy is supposed to be, which is valuing the experiences and understandings that everybody brings.” One thing these interviews clearly showed me was that every person’s background contributed immensely to their critical practice, and it was drawing upon their individual passions that made them the librarian they are.
Regarding future directions for critical information literacy, some interviewees responded that they wished for information literacy to become “something that by nature needs to be critical”: “My hope is that someday there will come a time when information literacy and critical information literacy are the same thing. And we don’t have to live with this older model of, you know, dealing with the tools kind of instruction.” In contrast to this hope that information literacy and critical approaches would become one in the same, another person made the point that, “I also don’t think that it has to be for everybody. We are all very different people and we don’t all have the same political views, and…we definitely don’t have the same philosophies when it comes to our approach to being librarians. So, at the same time, if you don’t feel like this is for you, that’s also fine.”
Whether or not one makes efforts to adopt a critical approach to librarianship through action, reflection, and theory, the relationships we develop with our communities and ways to meaningfully work towards creating a better world should be a central consideration. “The way we research, the way we teach, the way we practice our profession are all really building relationships with scholarly communities and with students that are becoming part of scholarly communities,” one interviewee wrote. “So one thing about getting into critical practices is that I’m connecting with a whole network of folks that think deeply and believe a better future is possible. Librarians, scholars, and students.”
As I finished one interview, the librarian I was in contact with shared several reflective questions, stating, “These are the things that I try to reflect on to distance myself from the daily grind and getting caught up in the monotony or frustrations of work. I thought you might enjoy them as well.” I would like to conclude with some of those questions that were offered, with the hope that readers might take them as an invitation to reflect intentionally upon their work and themselves.
- What are some existing forms of oppression our students engage with at the academy?
- How do librarians reinforce those systems of oppression in the classroom or inadvertently within library practices? How do our assumptions work their way into our teaching practices?
- What are some ways in which you design the classroom experience to be a democratic, collaborative, and transformative site?
- How do we balance the lived experiences of our students with “canonical” sets of knowledge and skills that they are required to learn?
- How do you view your role as an academic librarian and its relationship to social justice?
A sincere thank you to the reviewers who contributed their insights and expertise to make this a better piece: Lauren Smith, Sofia Leung, and Ryan Randall. Thank you to Annie Pho for serving as the Publishing Editor for this article and seeing it through the Lead Pipe publication process. The Institute for Research Design in Librarianship gave me the methodological footing I needed to begin this project and the camaraderie I needed to see it through. A special thanks to the 13 librarians I interviewed. It was a true pleasure to talk with them about critical information literacy and their work, and they left me inspired and hopeful that librarians and libraries can help create positive social change.
Accardi, Maria. (2013). Feminist pedagogy for library instruction. Sacramento, CA: Library Juice Press.
Accardi, Maria T., Emily Drabinski, and Alana Kumbier, eds. (2010). Critical library instruction: Theories and methods. Duluth, MN: Library Juice Press.
Beilin, Ian. “Student success and the neoliberal academic library.” Canadian Journal of Academic Librarianship 1, no. 1 (2016): 10-23. Available at: http://www.cjal.ca/index.php/capal/article/view/24303 (retrieved 19 September 2016).
Berman, Sanford. (1971). Prejudices and antipathies: A tract on the LC subject heads concerning people. Metuchen, NJ: Scarecrow Press.
Downey, Annie. (2016). Critical information literacy: Foundations, inspirations, and ideas. Sacramento, CA: Library Juice Press.
Drabinski, Emily. “Teaching the radical catalog.” In Radical cataloging: Essays at the front, edited by K.R. Roberto, 198-205. Jefferson, NC: McFarland and Company, 2008. Available at: http://www.emilydrabinski.com/wp-content/uploads/2012/06/drabinski_radcat.pdf (retrieved 19 September 2016).
Drabinski, Emily. “Critical pedagogy in a time of compliance | Information literacy keynote, Emily Drabinski.” YouTube video, 01:09:36. Posted by Moraine Valley Community College Library, May 3, 2016. https://www.youtube.com/watch?v=41ymjLLteBo
Elmborg, James. (2006). “Critical information literacy: Implications for instructional practice.” Journal of Academic Librarianship 32, no. 2: 192-199. Available at: http://borg.slis.uiowa.edu/pubs/Critical_Information_Literacy_Implications.pdf (retrieved 19 September 2016).
Ettarh, Fobazi. (2014). “Making a new table: Intersectional librarianship.” In the Library with the Lead Pipe. Available at: http://www.inthelibrarywiththeleadpipe.org/2014/making-a-new-table-intersectional-librarianship-3/ (retrieved 19 September 2016).
Fister, Barbara. “Practicing freedom in the digital library.” Library Journal, 26 August 2013. Available at: http://lj.libraryjournal.com/2013/08/future-of-libraries/practicing-freedom-in-the-digital-library-reinventing-libraries/ (retrieved 19 September 2016).
Gregory, Lua, and Shana Higgins, eds. (2013). Information literacy and social justice: Radical professional praxis. Sacramento, CA: Library Juice Press.
bell hooks. (1994). Teaching to transgress: Education as the practice of freedom. New York: Routledge.
Jacobs, Heidi L.M. “Information literacy and reflective pedagogical praxis.” The Journal of Academic Librarianship 34, no. 3 (2008): 256-262. Available at: http://scholar.uwindsor.ca/leddylibrarypub/23/ (retrieved 19 September 2016).
Kopp, Bryan M., and Kim Olson-Kopp. “Depositories of knowledge: Library instruction and the development of critical consciousness.” In Critical library instruction: Theories and methods, edited by Maria T. Accardi, Emily Drabinski, and Alana Kumbier, 55-67. Duluth, MN: Library Juice Press, 2010.
Kumbier, Alana. Interview by Robert Schroeder in Critical journeys: How 14 librarians came to embrace critical practice, 157-173. Sacramento, CA: Library Juice Press, 2014.
Lynn, Marvin, and Adrienne D. Dixson, eds. (2013). Handbook of critical race theory in education. New York: Routledge.
Morrone, Melissa, and Lia Friedman. “Radical reference: Socially responsible librarianship collaborating with community.” The Reference Librarian 50, no. 4 (2009): 371-396. Available at: http://eprints.rclis.org/23443/ (retrieved 19 September 2016).
Schroeder, Robert. (2014). Critical journeys: How 14 librarians came to embrace critical practice. Sacramento, CA: Library Juice Press.
Smith, Lauren. “Towards a model of critical information literacy instruction for the development of political agency.” Journal of Information Literacy 7, no. 2 (2013): 15-32. Available at: http://dx.doi.org/10.11645/7.2.1809 (retrieved 19 September 2016).
Tewell, Eamon. “A decade of critical information literacy: A review of the literature.” Communications in Information Literacy 9, no. 1 (2015): 24-43. Available at: http://www.comminfolit.org/index.php?journal=cil&page=article&op=view&path%5B%5D=v9i1p24 (retrieved 19 September 2016).
- A Case Study on Emulation-based Preservation in the Museum: Flusser Hypertext, Padberg et al.
- Towards a Risk Model for Emulation-based Preservation Strategies: A Case Study from the Software-based Art Domain, Rechert et al.
- Exhibiting Digital Art via Emulation – Boot-to-Emulator with the EMiL Kiosk System, Espenschied et al.
Himmelsbach's keynote showed how artists themselves respond to changes in the technical infrastructure by evolving their own works. For example, Marc Lee's TV Bot has been through three major versions. The work mashes up real-time news feeds available from the Internet in a way analogous to, and commenting on, TV news. The artist has evolved the work from RealVideo to Flash to HTML5. At each stage the appearance has changed somewhat to echo then-current TV aesthetics and, of course, to cope with the evanescent nature of Internet news feeds. Himmelsbach illustrated this evolution with screencast videos of several versions of several art works.
Flusser HyperCard stackPhilipp Tögel presented his work on the Flusser exhibit, a HyperCard stack consisting of an annotated text of a talk with associated audio of the talk itself. Preparing the original disk image for emulation using bwFLA and the Basilisk II emulator was a part-time task over about 18 months. It involved modifying the image obtained from the original disk in three ways:
- Using a "hack" to get the HyperCard stack into edit mode in order to understand why initially the emulation wasn't working.
- Replacing the HyperCard software with a more recent version that had adequate audio support; the original had custom audio add-ons making up for the original version's lack of it.
- Fixing a bug caused by a file that was corrupt on the original disk that was imaged.
Subtitled PublicKlaus Rechert's presentation used the example of emulating a work from the Tate Museum's collection, Rafael Lozano-Hemmer's Subtitled Public, to illustrate the process of analyzing the preservation risks of emulated environments that their paper describes. The work analyzed video from cameras and drove projectors to display words on visitors' chests. It used several Windows XP computers on an IP network, accessing the cameras via the Windows video API, so was less hardware-dependent than many similar works.
As usual, the emulation process started by imaging the original system disks. Running the images in emulation revealed a couple of dependencies:
- The set of device drivers resulting from the original XP installation didn't match the set of devices presented by the emulation. XP was re-configured to use original but compatible drivers.
- The original XP configuration assumed FireWire cameras. These cameras, and their host interfaces, are no longer easily available. XP was re-configured to use USB cameras, and this needed an update to the USB driver in the original version of XP.
My boyfriend came back from the warDragan Espenscheid et al won the Best Paper award. His presentation described the workflow used to prepare an exhibit at HeK using bwFLA running from USB boot drives. It celebrated 20 years of Olia Lialina's My boyfriend came back from the war:
"My Boyfriend Came Back From the War" is a unique case and a fascinating example of a work that has inspired artists for 20 years; they cite it time and again, they focus on it or reinterpret it through remixing. Lialina has collected these works in her private archive under the title Last Real Net Art Museum. So far she has gathered 27 versions, 13 of which she has chosen for the HeK exhibition.The exhibit presented the preserved Web pages of these works using contemporaneous browsers, mocked-up contemporaneous hardware (as props), and emulated networks slowed to contemporaneous speeds. The works themselves were unmodified, and the browsing environment was appropriate and unmodified.
Thus among these four presentations we have the full range of emulation preparation tasks, from no change, through re-configuring the operating system, to modifying the object itself, in one case by the artist and the other by the curators.
How likely are these tasks to arise in emulating mainstream software rather than software art? Note that all the modifications by curators described in the papers addressed issues arising from peripheral support; Flusser working around the lack of adequate sound support in early HyperCard, Lozano-Hemmer with video camera support in Windows XP. More modern operating systems dynamically re-configure themselves instead of requiring static pre-configuration, and network effects mean that I/O interfaces are very likely to be backwards-compatible (e.g. Espencheid et al's comments about USB).
bwFLA's inverted pyramidThus while issues such as these are rampant in software archaeology such as the Flusser exhibit and De Digitale Stadt, they are much less likely in preserving software going forward, especially software that isn't tightly coupled to I/O devices. In particular, the bwFLA approach of composing the emulated system image from layers, that they are now calling the "inverted pyramid", should make these issues much less likely.
Rechert et al describe these modifications as "migration" which is likely to confuse people. Migration is a term that, at least for the two decades since Rothenberg, has been used as a contraction for format migration. Format migration involves completely transforming the digital object, the "migrations" involved in preparing these emulations mostly involve the same bits as the original object, just slight changes to the environment the object needs. Even the cases here involving changes to the object are restricted to what are effectively bug fixes. They deserve a more descriptive term, perhaps "porting"?
Rechert suggests "image migration", stressing the parallelism between the processes involved, identifying the risks, identifying the "significant properties", and verifying that they have been preserved. I disagree. If we consider the O/S and hardware as the renderer for the software, the changes in this case are normally made to the renderer, not to the object as in format migration.
There are two big exceptions to the optimistic view that intensive tasks are unlikely to be needed in preparing emulations:
- Games are problematic in two ways. Some, like some software-driven art, need specific peripherals, such as steering wheels. Some have DRM, and need to be "cracked" to evade it.
- There is a more general problem. Much current software makes the assumption that the user accesses them via a PC. Looking ahead, because sales of PCs and tablets are in free-fall, this won't be the case. Using a phone to access software built for a PC rarely provides an acceptable user experience.