Subject: [NGC4LIB] Designing future "catalogs" - where do the users come in?

From: "B.G. Sloan" <[email-masked]>
Date: Fri, 23 Apr 2010 08:57:42 -0700

Every now and then I'm struck by the notion that library systems, generally, are designed BY librarians FOR librarians. I've had some people argue with this notion in the past, saying some library systems are difficult for even librarians to use. :-)
 
But I think you all get my drift...where in the process of designing a new "catalog" does user (patron) input come in? If the user's perspective doesn't come into play until the usability testing stage, that's probably too late.
 
I'll focus my question on one particular type of library as an example. Has anyone ever focused on the information-seeking behavior of students and faculty when developing the requirements for a "catalog" for an academic library? I'm not talking about holding a few focus groups early in the process. I'm talking about an in-depth look at studies of information-seeking behavior in a higher education environment. And I'm not talking about studies of how library users use library catalogs (although that would be useful information). I'm talking about the whole information-seeking environment...not just the part that the library controls.
 
I've been involved in several projects to develop requirements for OPACs. Generally, when we'd come to the "end-user requirements" part of the process, we'd listen to public services librarians describe catalog functionality that they THOUGHT users needed. Nothing against public services librarians...they're definitely closer to the end-user than catalogers and administrators. But they're still *librarians*, and they still view things through a librarian lens.
 
In short, how do we design systems that help users find what they really need, rather than designing systems where we librarians assume we know what users need?
 
Just some random thoughts on a rainy Friday...
 
Bernie Sloan

---

From: MJ Suhonos <[email-masked]>
Date: Fri, 23 Apr 2010 13:28:56 -0400

Some great thoughts there Bernie, for a lovely sunny Friday here.  :-)

> But I think you all get my drift...where in the process of designing a new "catalog" does user (patron) input come in? If the user's perspective doesn't come into play until the usability testing s…

I'll redirect your question to my library, since I think it serves as a similar example.  We are a large public library, but with two large "research and reference" branches that think and act in many ways as academic libraries.  It is a source of much institutional schizophrenia.  Compounding the issue is that the proportion of "layperson" requests to "serious researcher" requests has not been well-quantified.  Many suspect a 90-10 type of rule, but what generally happens then is that the advocates (usually, but not always, cataloguers) wind up forcing practice to revolve around the exception.

The *first* thing we need to do is stop lumping our users into a one-size-fits-all category of information-seeking behaviour.  Use cases can be extremely varied, and we need to recognize that — for our institution anyway — there are *at least* two major personas we need to address (and quite likely an infinitude of more), and their needs are almost diametrically different.  I use two not because of an affinity for bimodal models, but because one of the main concepts of access is the balance between precision and recall.

So, let's start there.  Design one interface for high-recall, low-precision that will satisfy the (perhaps 90% of) users who want to search independently, without reference help, for that kind of use case.  Design a more "expert" interface for low-recall, high-precision that will address the kind of detailed "serious research" the other users need.  Such an advanced interface will almost certainly require expert training, but this is an opportunity both for professional reference librarians to flex their skills, as well as for education toward users who want to learn how to do it themselves.

Now, do this in the same system.  Many will say that we already do this, but I think historically we've done a poor job of it (or, more specifically, the technology has done a poor job).  Just looking at the recent increase in faceted search implementations points to one example of how the system can adapt to the user in a more organic way than we've had in the past.  And maybe that's the by-and-for-librarians aspect that we mean; boolean search is powerful, but clunky.  We can do better in 2010.

More importantly, we should design our metadata models and our cataloguing practices around this same notion — we already do it to some degree with copy-cataloguing (and eg. "levels of record completeness"); but looking at it as *intentionally* providing less description to meet the user need, not just as a way to avoid duplication of effort.  One of the most frustrating phrases I see and hear on a daily basis is "good enough" — good enough for what?  For whom?  Practice needs to be tied to specific groups of behaviour, and one-size-fits-all doesn't cut it any more.

> I'm talking about an in-depth look at studies of information-seeking behavior in a higher education environment. And I'm not talking about studies of how library users use library catalogs (although that would be useful information). I'm talking about the whole information-seeking environment...not just the part that the library controls.

I *think* (but I don't know, not having done a lit. review) that there is a fair amount of research that could be pulled together to inform this.  Maybe not comprehensive, and probably not as deep as we'd like for non-library information sources, but certainly this can't be an unexplored area.  My curiosity is why isn't this research being distilled into practice?

> I've been involved in several projects to develop requirements for OPACs. Generally, when we'd come to the "end-user requirements" part of the process, we'd listen to public services librarians describe catalog functionality that they THOUGHT users needed.

Absolutely.  As an anecdote, we're about to launch a (long overdue) new website interface for our entire library, including replacing the catalogue with a faceted search UI and some very un-librarian design.  An intended-to-be-derogatory remark which one of my cataloguer colleagues made that I think resonates well is that "they want to make it look like a bookstore".  It's probably one of the first things that we've designed for the 90%, and although public feedback has been overwhelmingly positive, the 10% — mostly librarians — have been extremely vocal about how they feel it erodes the sanctity and authority of the library's reference functions.

They may well have a point, but until we qualify and quantify the "serious researchers" next to our "general public" as a user group, someone is going to be left disappointed.

MJ

---

From: Karen Coyle <[email-masked]>
Date: Fri, 23 Apr 2010 10:38:20 -0700

Quoting "B.G. Sloan" <[email-masked]>:

>  
> Every now and then I'm struck by the notion that library systems,   
> generally, are designed BY librarians FOR librarians. I've had some   
> people argue with this notion in the past, saying some library   
> systems are difficult for even librarians to use. :-)
>  
> But I think you all get my drift...where in the process of designing  
>  a new "catalog" does user (patron) input come in? If the user's   
> perspective doesn't come into play until the usability testing   
> stage, that's probably too late.
>  
> I'll focus my question on one particular type of library as an   
> example. Has anyone ever focused on the information-seeking behavior  
>  of students and faculty when developing the requirements for a   
> "catalog" for an academic library?
>

Not precisely what you are looking for, but I often make use of this
   http://kcoyle.net/temp/behaviors.jpg
as a comparison to the FRBR tasks. It comes from a Mellon project, and  
if I dig I can get you a link to that.

kc
-- 
Karen Coyle
[email-masked] http://kcoyle.net
ph: 1-510-540-7596
m: 1-510-435-8234
skype: kcoylenet

---

From: MJ Suhonos <[email-masked]>
Date: Fri, 23 Apr 2010 13:46:32 -0400

On 2010-04-23, at 1:38 PM, Karen Coyle wrote:

> as a comparison to the FRBR tasks. It comes from a Mellon project, and if I dig I can get you a link to that.

Karen, this is *extremely* helpful — I just had the insight yesterday that modelling metadata around events, rather than ("just") entities might be another way to look at the problem, and figured that someone must have already gone down this road before.  This one's going on my wall.  Thanks.

MJ

---

From: Karen Coyle <[email-masked]>
Date: Fri, 23 Apr 2010 10:51:35 -0700

The other thing I like about it, other than it is activity-driven, is  
that it is about the "use" of information, not just finding resources.  
I feel like the FRBr model, ending with "obtain" ignores a whole swath  
of user activities that only begin once someone has obtained a  
resource. Yet manipulation of bibliographic resources is key to a  
whole host of activities that take place after a person passes by the  
circ desk.

kc

Quoting MJ Suhonos <[email-masked]>:

>> as a comparison to the FRBR tasks. It comes from a Mellon project,   
>> and if I dig I can get you a link to that.
>
> Karen, this is *extremely* helpful — I just had the insight   
> yesterday that modelling metadata around events, rather than   
> ("just") entities might be another way to look at the problem, and   
> figured that someone must have already gone down this road before.    
> This one's going on my wall.  Thanks.
>
> MJ
>

---

From: Ed Jones <[email-masked]>
Date: Fri, 23 Apr 2010 11:15:38 -0700

I believe this is the Mellon project: http://www1.lib.umn.edu/about/mellon/docs.phtml

-----Original Message-----
From: Next generation catalogs for libraries [mailto:[email-masked]] On Behalf Of Karen Coyle
Sent: Friday, April 23, 2010 10:38 AM
To: [email-masked]
Subject: Re: [NGC4LIB] Designing future "catalogs" - where do the users come in?

kc

---

From: "Walker, David" <[email-masked]>
Date: Fri, 23 Apr 2010 12:03:52 -0700

> If the user's perspective doesn't come into play until the 
> usability testing stage, that's probably too late.

This depends on how you do your usability testing.

Usability experts all agree that if you wait to do user testing until the end of the process -- after the system has already been built -- then it's too late.

But if you bring users into the design process from the beginning, starting with focus groups, and then again at different iterations in the applications development lifecycle -- paper prototypes, wireframes, and so on -- you can get user feedback early and often enough to make useful design decisions.

You need user-feedback from the very beginning, before you write a line of code.  And I agree that simply talking to reference and instruction librarians is a poor substitute for actual end-users.

--Dave

==================
David Walker
Library Web Services Manager
California State University
http://xerxes.calstate.edu
________________________________________
From: Next generation catalogs for libraries [[email-masked]] On Behalf Of B.G. Sloan [[email-masked]]
Sent: Friday, April 23, 2010 8:57 AM
To: [email-masked]
Subject: [NGC4LIB] Designing future "catalogs" - where do the users come in?

Bernie Sloan

---

From: Laval Hunsucker <[email-masked]>
Date: Sat, 24 Apr 2010 13:27:21 -0700

> . . . ignores a whole swath of user activities that
> only begin once someone has obtained a resource.

This is indeed, as it has long (always?) been, the one 
great black hole of LIS research, LIS knowledge, and 
LIS practice. And it's devilishly difficult, at best. But 
had LIS really been interested in seriously pursuing 
the question ( beyond perhaps a few naive, over-
simplified, superficial models ), that process would 
surely have begun long before now. Not that it really 
matters all that much. Even if we *had* made the 
necessary effort, we would most probably have 
flopped miserably. 

We're just out of our depth here, and it's not really 
any of our business anyway. Voici the reason that 
we'll never be more than peripheral to the scholarly, 
creative enterprise. And that's imho precisely as it 
should be. Acknowledging that circumstance ( at 
long last ), and learning to live with it elegantly, 
would really not be difficult, and would make life 
a lot more pleasant for all concerned.

- Laval Hunsucker
   Breukelen, Nederland

----- Original Message ----
From: Karen Coyle <[email-masked]>
To: [email-masked]
Sent: Fri, April 23, 2010 7:51:35 PM
Subject: Re: [NGC4LIB] Designing future "catalogs" - where do the users come in?

The other thing I like about it, other than it is activity-driven, is that it is about the "use" of information, not just finding resources. I feel like the FRBr model, ending with "obtain" ignores a whole swath of user activities that only begin once someone has obtained a resource. Yet manipulation of bibliographic resources is key to a whole host of activities that take place after a person passes by the circ desk.

kc

> Karen, this is *extremely* helpful — I just had the insight  yesterday that modelling metadata around events, rather than  ("just") entities might be another way to look at the problem, and  figure…

--Karen Coyle

---

From: Alexander Johannesen <[email-masked]>
Date: Sun, 25 Apr 2010 08:56:11 +1000

B.G. Sloan <[email-masked]> wrote:
> In short, how do we design systems that help users find
> what they really need, rather than designing systems
> where we librarians assume we know what users need?

There's an approach that is rarely used but that I have some
experience in doing :

   Ask them.

Ok, you can also call it User-centered design, where after asking them
you integrate direction into your designs, go back ask for more, test
what you've got, integrate that into your design, go back, ask for
more, test what you've got, rinse, repeat. :)

You'd be amazed at how many decisions are made for others that they
never wanted, no matter the walks of life. This is a human being
thing, though, and got nothing to do with libraries specifically. It's
how the experts stay experts. It's how the commercial world tries to
get that business advantage that will bring in the cash ; *guess* what
they want, and if you get it right, you win. If not, guess some more.

User-centered design has gained some traction in systems design, and
some entities are better than others. Do you remember the first
generation of government online services? Yup, that's how bad it can
get. But there's risk in asking your users, some actual, some not, but
most importantly are the mythical parts to the conundrum ;

1. You're showing weakness (that you don't know what you're doing)
2. You're telling your competition what you're up to
3. Your people aren't flexible enough (or too big ego's) to handle the dynamics
4. You know better because you're the expert
5. We've always done it this way

These are just some of the hurdles that gets in the way, an no wonder
; we're humans, we build up our knowledge over time, we take pride in
what we do, and we want to keep us in a position of advantage.
User-centered design throws a lot of this upside-down.

If you truly asked your patrons (layman as well as specialists) what
they wanted, it may also not fit into policy, domain or budget. This
one turns political quite quickly, but certainly shouldn't be ignored
(which is easy to do; why 'deal' with a problem you can't deal with?)
but should at least be used for future planning.

I know a lot of people have done this at various libraries, but I
doubt anyone has thought about their ILS in that way. OPAC maybe, or
bits of one. It would be an interesting exercise. There's librarian
needs and patron needs, and probably a lot of hidden needs for both
that aren't current. And then there are the needs that are long gone,
needs that are coming. You need a team of people to work on these
things all the time, but I doubt any library do this (it can be a
scary process), but this is certainly something I've been talking
about a lot ; a concentrated and ongoing effort in shaping the future.
Libraries love committees, and this one would be one with an amazing
value.

Kind regards,

Alex
-- 
 Project Wrangler, SOA, Information Alchemist, UX, RESTafarian, Topic Maps
--- http://shelter.nu/blog/ ----------------------------------------------
------------------ http://www.google.com/profiles/alexander.johannesen ---

---

From: Weinheimer Jim <[email-masked]>
Date: Mon, 26 Apr 2010 09:46:01 +0200

Alexander Johannesen wrote:
<snip>

   Ask them.
</snip>

Absolutely. This is the major problem with FRBR and its user tasks that supposedly, everything is based upon: I don't think anybody would ever say that the FRBR user tasks are what people want and/or need in the current information environment. Nobody has ever pointed to any research that shows FRBR builds a tool that users want. If anybody questions themselves about their own searching habits, I doubt if anybody will say that what I really want and need are the FRBR user tasks. This doesn't mean that people no longer need to be able to: find "stuff" by their authors, titles, and subjects, but we need to recognize the startling fact that in era of "Google-one-box" services, the very idea of *being able" to limit a search by "author" or "title" or "subject" is becoming lost. And people on this list know what I think of the use and even artificiality of "Work-Expression-Manifestation-Item," which have no relationship to user tasks (except for those very rare users who need the!
  1917 Harper's edition of Huckleberry Finn; but let's face it--if somebody needs that level of detail, they probably need more information than what a regular library catalog gives them). And anyway, full-text searching has made clear that people want and expect many other powers as well.

But on a broader level, if we define a catalog as an "aid for finding different resources that exist within a local collection," I think this needs to change because now with the web, where people can see a lot more than ever before, the "local collection" is necessarily limiting from the very beginning. And our users see it clearly. Much more useful for our patrons would be for everyone who feeds into the catalog (i.e. those who select materials for cataloging, plus those who do the actual cataloging itself) to think in broader terms than a "local catalog," envisioning something more akin to a "bibliography."

In this way, selection turns into something other than, "what is being paid for and/or housed in my individual collection and other institutions where I have a special relationship" into "what are the useful materials available to the members of my local community no matter where they happen to be and if they are free or not."

Showing people what is really and truly available to them and not just what is in a local collection is what I mean by thinking in terms of "bibliography" which is not limited to any one collection. Only when you know what is really out there can the individual or institution decide if it is worth the costs of getting access to it. One major step further in this direction would be to create an "annotated bibliography," and this could be done easily enough with Web2.0 tools. 

Making such a tool would simply be too much for librarians to do alone, and we would have to enlist the help of scholars, teachers and other experts for the task of selection; for that of description, access, and record maintenance, we would need lots of other help. Of course, we would have to rethink our current standards into what could be done *practically* to ensure some standards of quality in these records that must be built cooperatively with many other communities out there. 

For example, what is the purpose of the "bibliographic description" when the item itself is instantly available with a simple click? While I don't doubt there is a purpose, it must be seen as fundamentally different from the purpose of the ISBD standard for a book that is available only after trudging through the stacks, or after spending two days retrieving it from an annex location off-site, or through a very expensive ILL that involves both staff time and funding from a number of agencies.

Using a tool such as this would probably demand all different kinds of interfaces: one for novices, another for experts, or perhaps even by specialty: for classicists, for architects, for physicists, ... 

To me, viewing the problem in these ways would represent a real "maturing" of the local catalog into something greater than it is now, and would create a tool that would become vital to our users, and I think rather quickly.

James Weinheimer  [email-masked]
Director of Library and Information Services
The American University of Rome
via Pietro Roselli, 4
00153 Rome, Italy
voice- 011 39 06 58330919 ext. 258
fax-011 39 06 58330992