Subject: [CODE4LIB] Fwd: [NGC4LIB] Integrating with your ILS through Web services and APIs

From: Ross Singer <[email-masked]>
Date: Wed, 22 Jul 2009 22:56:58 -0400

This seems a _far_ more appropriate list for these questions.

-Ross.
---------- Forwarded message ----------
From: Breeding, Marshall <[email-masked]>
Date: Wed, Jul 22, 2009 at 9:53 PM
Subject: [NGC4LIB] Integrating with your ILS through Web services and APIs
To: [email-masked]

I am in the process of writing an issue of Library Technology Reports
for ALA TechSource titled "Hype or reality:  Opening up library
systems through Web Services and SOA."  Today almost all ILS products
make claims regarding offering more openness through APIs, Web
services, and through a service-oriented architecture (SOA).  This
report aims to look beyond the marketing claims and identify specific
types of tasks that can be accomplished beyond the delivered
interfaces through programmatic access to the system internals.

As part of the research for this article I am soliciting feedback from
libraries that taken advantage of Web Services or other API's in
conjunction with their core Integrated Library System (ILS) to meet
specific needs.  I'm interested in hearing about how you might have
been able to integrate library content and services into applications,
extracted data, automated processes or other novel applications.
Please tell me about your experiences with your ILS in regard to the
APIs it offers:

 - Do you feel like you can pretty much do anything you want with the
system, or do you feel constrained?
 -Are the APIs offered able to address all the data and functionality
within the ILS?
 -On the flip side, do you feel like your ILS is too closed?
 -Do you find the APIs offered by the developer of the ILS to be well
documented?
 - What programming languages or other tools were you able to use to
take advantage of these APIs?
 - What level of programming proficiency is required:  Systems
librarian with scripting languages, software development engineer, or
something in between?
 - What's on your wish list?  What kind of APIs would you like to see
incorporated into your current or next ILS?
 - I'm interested in responses from those that use open source ILS
products as well.  Are you able to programmatically interact with the
ILS?
 - Do you consider your ILS as embracing a true Service-oriented
architecture?  Systems vendors increasingly promote their ILS as SOA.
Can you provide examples where the ILS does or does not exhibit traits
of SOA in your environment.

While it's important for the ILS to offer support for standard
protocols such as Z39.50, NCIP, and OAI, that's not the core of the
issue here.  What I'm looking for are API's that allow the library to
get at data and functionality not addressed by these protocols.

Thanks in advance for sharing your experiences in ILS API's with me
for this report.

I appreciate your assistance.

-marshall

Summary excerpt:
Libraries increasingly need to extract data, connect with external
systems, and implement functionality not included with the delivered
systems.  Rather than being reliant on the products developers for
enhancements to meet these needs, libraries increasingly demand the
ability to exploit their systems using APIs, Web Services, or other
technologies.  Especially in libraries that exist in complex
environments where many different systems need to interact, the demand
for openness abounds.  As libraries develop their IT infrastructure,
it's imperative to understand the extent to which their automation
products are able to interoperate and thrive in this growing realm of
Web services.  This report aims to assess the current slate of major
library automation systems in regard to providing openness through
API's, Web Services, and the adoption of SOA.

Marshall Breeding
Director for Innovative Technology and Research
Vanderbilt University Library
Editor, Library Technology Guides
http://www.librarytechnology.org
615-343-6094

---

From: Eric Lease Morgan <[email-masked]>
Date: Thu, 23 Jul 2009 09:22:20 -0400

On Jul 22, 2009, at 10:56 PM, Ross Singer wrote:

> ...Today almost all ILS products make claims regarding offering more  
> openness through APIs, Web services, and through a service-oriented  
> architecture (SOA)....

I heard someplace recently that APIs are the newest form of vendor  
lock-in.  What's your take?

-- 
Eric Lease Morgan
Head, Digital Access and Information Architecture Department
Hesburgh Libraries, University of Notre Dame

(574) 631-8604

---

From: Robert Fox <[email-masked]>
Date: Thu, 23 Jul 2009 09:59:59 -0400

> lock-in.  What's your take?

If you use a design pattern like MVC on top of the API (assuming you're building a web app on top of the API), then it doesn't have to be a lock-in scenario. It is amazing, though, how much proprietary stuff vendors sneak into so-called standards based web services. No matter what you're doing with the API, you'd probably want to build some sort of abstraction layer on top of it so that it would be easier to swap out if you changed vendors or ditched the vendor all together. That's assuming you could extract your data from the vendor product... ;-)

Rob Fox
Hesburgh Libraries
University of Notre Dame

---

From: Andrew Hankinson <[email-masked]>
Date: Thu, 23 Jul 2009 10:07:58 -0400

I think that it's supposed to be the exact opposite. APIs, and  
especially web APIs, exist to provide access to your data outside of a  
specific vendor implementation. That way you can have your OPAC from  
$BIG_VENDOR, but use the bibliographic data from it in  
$OPEN_SOURCE_PROJECT without having to access the database directly  
(causing your DBAs to have sleepless nights.) Theoretically anything  
that comes over HTTP is some form of structured text, so there  
shouldn't be unreadable binary blobs.

In theory, that's how it's supposed to work. There's still crappy  
implementations, but that's not the fault of the API - that's the  
fault of the implementer.

-Andrew

On 23-Jul-09, at 9:22 AM, Eric Lease Morgan wrote:

>> ...Today almost all ILS products make claims regarding offering  
>> more openness through APIs, Web services, and through a service- 
>> oriented architecture (SOA)....
>

---

From: Jonathan Rochkind <[email-masked]>
Date: Thu, 23 Jul 2009 10:13:36 -0400

> lock-in.  What's your take?

When they are custom vendor-specific APIs and not standards-based APIs, 
they can definitely function that way. I'm still not sure if even a 
vendor-specific API is more or less lock-in than NOT having an API.  On 
the one hand, you will start to have software written against the 
vendor-specific API, that won't work without changing it up if you 
switch vendors.  But on the other hand, with SFX and Umlaut, for 
instance, Umlaut does so much more than SFX, and the SFX adapter piece 
is such a small part, that in that case, for us at least, having SFX 
with an API and Umlaut on top of it it definitely makes it _easier_ for 
us to switch link resolvers without disrupting our services built on top 
of it. 

But really, what you want is standards-based APIs, not vendor-specific 
APIs. That would give you the best of all worlds. There are a couple 
challenges that keep us from getting there though. One is that the 
library community, historically, is, well, pretty AWFUL at writing 
standards.  We come up with standards that don't actually accomplish 
what they were intended to accomplish, are too complicated for anyone to 
implement right (on either producer or consumer side), and leave so much 
wiggle room that someone can claim they support the standard but not in 
a way that any other software will ever understand.  (NCIP anyone?)

So there are a couple ways to try to get better at this. One is 
definitely looking outside the library world for standards to use. But 
unlike code4libbers, I don't think (from my experience) that's always 
possible or easy.  We have priority problems that, while they are not 
entirely foreign to the larger world, aren't as high a priority for most 
of the non-library world, meaning they don't yet have robust standards 
solutions. However, especially when standards are extensible (like XML 
ones often are), you can sometimes start with a general standard and 
extend it for the library space.

Secondly, instead of creating standards before anyone has actually tried 
solving the problem the standard is meant to solve (as we often seem to 
do), the BEST standards are created by generalizing/abstracting from 
existing best practices. A buncha people try it first, you see what 
works and what doesn't, you see what the actual use cases and needs are, 
you take the best out of what's been done, and you standardize it.   But 
doing it this way means you need to go through a period of 
vendor/product specific (eg) APIs before you can get to the standard.  
The library world is still immature in developing good software 
infrastructures, we're going to need to through some more pain for a 
while, no way around it.

But another problem in all of this is that vendors may not have the 
interest OR the in-house expertise to actually provide standards-based 
APIs.  The APIs we often get now from vendors, frankly, are kind of 
kludgey, and do not fill me with confidence that the vendor actually has 
the proper staff or resources allocated to create good standards-based 
APIs -- which, definitely, takes more time than creating a kludgey 
vendor-specific one-off.   Or maybe the vendor actually is 
dis-interested in this because they want lock-in.  Or maybe it's just 
the case that the quality of your APIs doesn't effect your sales at all, 
so it doesn't make (short term at least) business sense to do it well.  
(Heck, the _presence_ of an API has only just begun to effect sales, but 
libraries aren't good enough at judging how good it is, that even a 
crappy API is probably 'good enough' for sales).

One way out of this is definitely open source. We'll work out the best 
practices and standards ourselves, and then we start insisting that 
vendors follow them.  The DLF-DI API is perhaps one example of an 
attempt at this, created from a generalization of the experience of 
library developers.   But the library developer community is also small, 
and generally fairly in-experienced. Creating APIs is done best by 
experienced developers who understand what's going to make the API 
useable or not.

But, anyway, one step at a time. I firmly believe that even 
vendor-specific kludgey APIs are better than no APIs at all -- we learn 
how to do better by trying.

Jonathan

---

From: Till Kinstler <[email-masked]>
Date: Thu, 23 Jul 2009 17:13:58 +0200

Andrew Hankinson schrieb:

> specific vendor implementation. 

If they are open, well documented, free of non disclosure agreements, 
free to be used/implemented in any kind of software and scenario (I 
don't claim they need to be free of fees if they provide a valuable 
service!)...
I heard, there are APIs that don't work that way... Those may lock you 
in, yes.

> That way you can have your OPAC from 
> that comes over HTTP is some form of structured text, so there shouldn't 
> be unreadable binary blobs.

Oh no, please. We are doing some things that way: Screenscraping HTML. 
Sleepless nights...
That's as good as a publisher sending some hundred thousand metadata 
records in a somehow "structured" Word document (no joke, Word even 
refused to open that file, OpenOffice did, very nice: Journal Titles 
etc. in the headline, article data on the page, must have been a real 
punishment for the one compiling that beast).

> implementations, but that's not the fault of the API

I think you mix up APIs with open standards or documentation...
APIs are everywhere in software (they are a fundamental concept of 
software design), but most are not "open" or intended for "public use" 
(even Microsoft has a long tradition of APIs, but it took them a long 
time to embrace the power of open APIs in their products).

Regards,
Till

-- 
http://twitter.com/tillk

---

From: John Fereira <[email-masked]>
Date: Sat, 25 Jul 2009 06:45:50 -0400

Robert Fox wrote:
> If you use a design pattern like MVC on top of the API (assuming you're building a web app on top of the API),

  then it doesn't have to be a lock-in scenario.

That's exactly what I have done to create a suite of library services 
that are currently being integrated into a new web site.  It should be 
noted, however, that using a MVC pattern doesn't mean that content 
brought in via a API in a data access layer can only be consumed using a 
web browser.  For my services, I've created multiple implementations of 
the View layer.  For example, a staff directory, which I obtain using an 
API I wrote which accesses an external staff directory app brings the 
data into a model which can be viewed as html, xml, json, and even will 
produce a dynamically generated PDF that staff can print off and place 
on their cubicle wall.  Other services have data related information 
(workshops, events, etc) have a view which produces iCal.