Subject: [CODE4LIB] Auto-suggest and the id.loc.gov LCSH web service

From: Ethan Gruber <[email-masked]>
Date: Mon, 7 Dec 2009 10:46:29 -0500

Hi all,

I have a need to integrate the LCSH terms into a web form that uses
auto-suggest to control the vocabulary.  Is this technically possible with
the id.loc.gov service?  I can curl a specific id to view the rdf, but I
would need to know the specifics of the search index on the site to feed the
auto-suggest.  For example, when the user types "va" in the box, the results
should filter all subject headings that begin with "va."

I can certainly accomplish this by indexing the ~400 meg XML file into solr
and use TermsComponent to filter terms dynamically as the user types, but
I'd rather use the LOC's service if possible.

So my question is: has anyone successfully done this before in the way I
described?

Thanks,
Ethan Gruber
University of Virginia Library

---

From: Winona Salesky <[email-masked]>
Date: Mon, 7 Dec 2009 10:59:36 -0500

We are also very interested in doing this at UVM. We have been storing  
the files locally as well, and it pretty cumbersome.
-Winona

---

From: "Houghton,Andrew" <[email-masked]>
Date: Mon, 7 Dec 2009 11:05:02 -0500

> From: Code for Libraries [mailto:[email-masked]] On Behalf Of
> Winona Salesky
> Sent: Monday, December 07, 2009 11:00 AM
> To: [email-masked]
> Subject: Re: [CODE4LIB] Auto-suggest and the id.loc.gov LCSH web
> service
>
> > the id.loc.gov service?

Why can't you just add a "*" to the end of the data in your search form
and send the request to the id.loc.gov search, per:

<http://id.loc.gov/authorities/techcenter/searching.html>

then parse the response?

Andy.

---

From: Ethan Gruber <[email-masked]>
Date: Mon, 7 Dec 2009 11:14:28 -0500

It doesn't seem very efficient.  It is taking me at least 30 seconds to load
a page of 'a*' in http://id.loc.gov/authorities/search/

---

From: Doug Chestnut <[email-masked]>
Date: Mon, 7 Dec 2009 11:21:10 -0500

mod_cache might help, assuming that performance gets better as your
query string grows.

--Doug

---

From: Benjamin Florin <[email-masked]>
Date: Mon, 7 Dec 2009 11:42:30 -0500

Delaying the autosuggest until three or four characters have been
entered should save some time.

Unfortunately, the search results aren't very digestible. The query
term is matched anywhere in the heading, and only 30 results are
returned per page. So, for example, the first page of a search for
"doc*":

  http://id.loc.gov/authorities/search/?q=doc*

doesn't return any results that actually start with "doc" (" Act of
state doctrine", "Adventists--Doctrines",
"Agriculture--Documentation", etc).

Ben

-- 
Benjamin Florin
Technology Assistant for Blended Education
Simmons College GSLIS
617-521-2842
[email-masked]

---

From: "LeVan,Ralph" <[email-masked]>
Date: Mon, 7 Dec 2009 11:43:02 -0500

Here in OCLC Research we've been experimenting with AutoSuggester
services.  The folks in charge of our copy of LCSH are considering
putting up an AutoSuggester for that.  I'll let you know in the next
couple of days how that goes.

To make the service work at keystroke speeds, we've had to precalculate
the responses and load them into a database of their own.  We walk
through the database that we're building the AutoSuggester for, pulling
out 4-tuples of data: the term to be suggested, the recordID associated
with the term (in case multiple terms might be suggested for the same
record), a weight for the term and a string of other arbitrary data to
send along with the recommendation (in the case of our VIAF
AutoSuggester, that's a list of authority control numbers).  Those
tuples are then evaluated to generate a list of the 10 best terms that
match each keystroke combination and that list is turned into a record
and the keystroke combination is the key to that record.  We then load
those records into a simple text database indexing on the keystroke
combination.  We front-end that database with a simple service and we're
done.

The only downside to this scheme is that the AutoSuggester database is
relatively static.

Ralph

> -----Original Message-----
> Sent: Monday, December 07, 2009 11:14 AM
service
> > > > auto-suggest to control the vocabulary.  Is this technically
possible
> > > with

---

From: Ethan Gruber <[email-masked]>
Date: Mon, 7 Dec 2009 11:48:17 -0500

Thanks for the input so far.

Ben, another problem with digestibility of the search results is that it's
not XHTML, and therefore not well-formed XML, making it impossible to
process with XPath.

I think I'll experiment with the Solr solution, but like the AutoSuggester
being developed at OCLC, the index would be fairly static unless there was a
way to pull updates from LOC into it.

Ethan

---

From: Ross Singer <[email-masked]>
Date: Mon, 7 Dec 2009 11:56:28 -0500

It has an OpenSearch interface:

http://id.loc.gov/authorities/opensearch

But I don't think there's a way to explicitly limit to, say, the
beginning of a label.

lcsubjects.org has a sparql interface where you could use a regex
filter on the labels:

http://api.talis.com/stores/lcsh-info/services/sparql

But it's probably not going to be fast enough for what you're talking about.

There are around 647,500 distinct labels in the dump files, it might
be easier and better to just grab the n-triples file, pull the lines
with <http://www.w3.org/2004/02/skos/core#prefLabel> and
<http://www.w3.org/2004/02/skos/core#altLabel> and shove them in a
local data store.

On the other hand, if you don't care where your "autocomplete" string
is coming from in the label, you could try:

http://api.talis.com/stores/lcsh-info/items?query=preflabel%3Av+||+altlabel%3Av&max=10&offset=0&sort=&xsl=&content-type=

http://api.talis.com/stores/lcsh-info/items?query=preflabel%3Avi+||+altlabel%3Avi&max=10&offset=0&sort=&xsl=&content-type=

http://api.talis.com/stores/lcsh-info/items?query=preflabel%3Avir+||+altlabel%3Avir&max=10&offset=0&sort=&xsl=&content-type=

etc.

-Ross.

---

From: Karen Coombs <[email-masked]>
Date: Mon, 7 Dec 2009 11:15:22 -0600

Ethan,

In PHP5 you can do Xpath on non-XHTML HTML. You just need to set it to
ignore the errors. I discovered this when trying to do some screen scraping.
The HH I found helpful is at
http://blogoscoped.com/archive/2004_06_23_index.html#108802750834787821

Now sure it is helpful to you, but I thought I'd mention it.

Karen

-- 
Karen A. Coombs
Head of Libraries' Web Services
University of Houston
114 University Libraries
Houston, TX  77204-2000
Phone: (713) 743-3713
Fax: (713) 743-9811
Email: [email-masked]

---

From: Ed Summers <[email-masked]>
Date: Mon, 7 Dec 2009 14:51:58 -0500

In the interests of all-that-is-agile, and the HolidaySeason™ I took a
quick (imperfect) stab at providing some basic suggest functionality
at id.loc.gov which you can find documented in the OpenSearch
Description:

  http://id.loc.gov/authorities/opensearch/

I used the OpenSearch Suggestions extension defined at:

  http://www.opensearch.org/Specifications/OpenSearch/Extensions/Suggestions/1.1

For example:

  curl http://id.loc.gov/authorities/suggest/?q=folklore

Or with an offset:

  curl "http://id.loc.gov/authorities/suggest/?q=folklore&offset=50"

It's something to throw darts at. Comments, questions, suggestions
(ahem) welcome :-)

//Ed

---

From: Ed Summers <[email-masked]>
Date: Mon, 7 Dec 2009 15:00:15 -0500

> not XHTML, and therefore not well-formed XML, making it impossible to

What page did you find that wasn't valid XHTML? The JSON should be
easier to process typically (no XPATH required), but I'm still
curious.

  http://validator.w3.org/check?uri=http%3A%2F%2Fid.loc.gov%2Fauthorities%2Fsearch%2F%3Fq%3Dhtml&charset=%28detect+automatically%29&doctype=Inline&group=0

//Ed

---

From: "LeVan,Ralph" <[email-masked]>
Date: Mon, 7 Dec 2009 15:18:51 -0500

That looks more like an old-style index browse than what I'd think of as a suggestion.  You've returned nothing until the user has typed enough characters to restrict the number of index terms to be returned to a rational number.

An example of the difference between an Index Browse and AutoSuggest can be seen in our VIAF service where a browse on the name "shak" (http://viaf.org/search?operation=scan&scanClause=local.mainHeadingEl+exact+"shak"&version=1.1&responsePosition=10&maximumTerms=1000) returns just slightly over 1000 names, with "shakespeare, william" buried somewhere in the middle.  Whereas sending "shak" to the AutoSuggester (http://viaf.org/AutoSuggest?query=shak) gets back a ranked list of the top ten names that begin with "shak" ("Shakespeare, William, 1564-1616", "Shākyamuni, fl.ca. v560-ca. v480", "Shakuku", ...).  In addition, the AutoSuggester will return something in response to a single keystroke (http://viaf.org/AutoSuggest?query=s).

Ralph

> Ed Summers
> Sent: Monday, December 07, 2009 2:52 PM
> Subject: Re: [CODE4LIB] Auto-suggest and the id.loc.gov LCSH web service
>

---

From: Ed Summers <[email-masked]>
Date: Mon, 7 Dec 2009 15:38:42 -0500

> That looks more like an old-style index browse than what I'd think of as a suggestion.  You've returned nothing until the user has typed enough characters to restrict the number of index terms to b…

Yes, that's fair. Although I still think an index browse for LCSH is
minimally useful. What are you using for relevancy ranking in the VIAF
AutoSuggester? Are you ranking based on the number of WorldCat records
that reference a particular name?

//Ed

---

From: "LeVan,Ralph" <[email-masked]>
Date: Mon, 7 Dec 2009 15:43:49 -0500

> Sent: Monday, December 07, 2009 3:39 PM
> 
> What are you using for relevancy ranking in the VIAF AutoSuggester?

For VIAF, rankings are calculated based on the number of institutions
that have controlled that name and the amount of attention the
institutions have given to that name (e.g. size of their respective name
authority records).

Ralph

---

From: Ed Summers <[email-masked]>
Date: Mon, 7 Dec 2009 17:56:46 -0500

> institutions have given to that name (e.g. size of their respective name

Neat. It would be great to have some external dataset to use in
ranking LCSH suggestions at id.loc.gov. But at the moment it's a
simple mysql db loaded up with some MARC LCSH data. I guess it could
do something smart with PageRank-like ranking of 'super-concepts'
(concepts that are linked to a lot)...but that would've taken longer
than 20 minutes :-)

//Ed

---

From: "[Your Name]" <[email-masked]>
Date: Mon, 7 Dec 2009 18:49:01 -0500

How about feeding back from web request stats?

Things that get pulled more often are probably more popular.

It's admittedly not very clever, but it would be easy to  implement...

---
A. Soroka
Digital Research and Scholarship R & D
the University of Virginia Library

---

From: Keith Jenkins <[email-masked]>
Date: Tue, 8 Dec 2009 08:40:02 -0500

On Mon, Dec 7, 2009 at 5:56 PM, Ed Summers <[email-masked]> wrote:
> It would be great to have some external dataset to use in

The frequency of an LCSH term within the LC catalog could also be
useful for ranking, although I'm not sure if such data would be
readily available.

Another possibility would be a simple count of broader terms +
narrower terms + related terms or something like that.  Although
PageRank would probably be better, since even some "important" terms
might have a relatively small number of immediately-adjacent links.

Keith

---

From: Karen Coyle <[email-masked]>
Date: Tue, 8 Dec 2009 07:16:50 -0800

> useful for ranking, although I'm not sure if such data would be

Couple of things: first, what we have at id.loc.gov is NOT LCSH, but a  
copy of the LC subject authority file. The entries in this file form  
the basis for subject headings, most of which add "facets" to the  
authority entry when forming the subject heading. One could do a  
left-anchored match against actual headings, and that might provide  
some interesting statistics.

Edward Betts of the Open Library project did some casual data  
gathering for subjects, and posted his "top 1000" subject headings  
(not subject authorities):
http://edwardbetts.com/ol/top_1000_subjects
The OL has decided to break up the subject headings into their  
subfields, and somewhere there are some pages that show some subfields  
with the highest ranking subfields they appear with. (There must be a  
better way to say that! Sorry, too early, too few cups of tea.) One  
example is here:
http://home.us.archive.org/~edward/related/Cheese.html
I think that something like this will be incorporated into the next  
version of OL, which will be heavily navigation-oriented rather than  
search-oriented.

kc
p.s. Anyone who wants to play with a file can grab the OL data export:

http://openlibrary.org/dev/docs/jsondump

Unfortunately it includes both LC and non-LC subjects (mainly BISAC  
from Amazon)

-- 
Karen Coyle
[email-masked] http://kcoyle.net
ph: 1-510-540-7596
m: 1-510-435-8234
skype: kcoylenet

---

From: Ethan Gruber <[email-masked]>
Date: Tue, 8 Dec 2009 10:30:47 -0500

If that isn't LCSH, then is the entirety of LCSH available electronically in
some capacity (at least available in some easily accessible file or files
that can be processed)?

Ethan

On Tue, Dec 8, 2009 at 10:16 AM, Karen Coyle <[email-masked]> wrote:

> copy of the LC subject authority file. The entries in this file form the
> basis for subject headings, most of which add "facets" to the authority
> entry when forming the subject heading. One could do a left-anchored match
> against actual headings, and that might provide some interesting statistics.
>
> Edward Betts of the Open Library project did some casual data gathering for
> subjects, and posted his "top 1000" subject headings (not subject
> authorities):
> The OL has decided to break up the subject headings into their subfields,
> and somewhere there are some pages that show some subfields with the highest
> ranking subfields they appear with. (There must be a better way to say that!
> Sorry, too early, too few cups of tea.) One example is here:
> http://home.us.archive.org/~edward/related/Cheese.html<http://home.us.archive.org/%7Eedward/related/Cheese.html>
> I think that something like this will be incorporated into the next version
> of OL, which will be heavily navigation-oriented rather than
> search-oriented.
>
> kc
> Unfortunately it includes both LC and non-LC subjects (mainly BISAC from
> Amazon)
>
>
>

---

From: Ed Summers <[email-masked]>
Date: Tue, 8 Dec 2009 10:35:33 -0500

On Tue, Dec 8, 2009 at 10:16 AM, Karen Coyle <[email-masked]> wrote:
> Couple of things: first, what we have at id.loc.gov is NOT LCSH, but a copy
> of the LC subject authority file. The entries in this file form the basis
> for subject headings, most of which add "facets" to the authority entry when
> forming the subject heading. One could do a left-anchored match against
> actual headings, and that might provide some interesting statistics.

Yes, using the actual headings extracted from bibliographic data seems
to be a better approach. It's easier to rank them, and as Karen points
out you get the actual post-coordinated headings, not just the
headings LC has decided to establish authority records for.

//Ed

---

From: Ross Singer <[email-masked]>
Date: Tue, 8 Dec 2009 10:42:05 -0500

I suppose it would be helpful to actually know the problem that is
trying to be solved here (I mean, a lot of people, including myself,
are throwing out solutions to a problem that's never been actually
defined).

Ethan, what, exactly, are you trying to do?  Do you want authorized
headings?  Or do you want LCSH that appears in the wild?

-Ross.

---

From: Ethan Gruber <[email-masked]>
Date: Tue, 8 Dec 2009 11:08:51 -0500

I am going to integrate subject headings into an XForms application.  I can
work with the authorized headings.  Before taking the XML file that LC
provides and doing my own thing with Solr, I wanted to see if anyone had
used the headings to do their own type of autosuggest.  I was especially
interested if anyone had done it with data provided dynamically by a service
that may or may not have existed on LC.  A Solr index that I populate with
data I download today becomes static, my interest is how I (or other people
developing their own autosuggest systems based on the subject headings) can
pull updates of the master authority XML file into my index of terms.

Ethan

On Tue, Dec 8, 2009 at 10:42 AM, Ross Singer <[email-masked]> wrote:

> copy

---

From: Ed Summers <[email-masked]>
Date: Tue, 8 Dec 2009 11:23:28 -0500

On Tue, Dec 8, 2009 at 11:08 AM, Ethan Gruber <[email-masked]> wrote:
> my interest is how I (or other people

Take a look at the Atom feed for id.loc.gov [1]. I think there's
enough information there for you to pull in creates, updates and
deletes. If not it would be a fun practical, experiment to try to get
it to the point where it can work for you, and hopefully others.

//Ed

[1] http://id.loc.gov/authorities/feed/

---

From: Ethan Gruber <[email-masked]>
Date: Tue, 8 Dec 2009 11:27:26 -0500

Thanks a lot, Ed!

I think that could work.

---

From: "LeVan,Ralph" <[email-masked]>
Date: Thu, 10 Dec 2009 11:11:44 -0500

We've made some progress on this topic.

I have available a list of our FAST subject headings.  They are derived
from LCSH and may be of some use.  The folks that produced this file are
working on producing a similar file for LCSH.

The file can be found at http://orlabs.oclc.org/FAST/fastOutput.zip.

The file contains tab delimited records.  The first column is the ID
number of the FAST record that the term comes from.  The second column
is the MARC Authorities field that the term came from.  The third column
is the term itself.  The 4th column is the count of records in WorldCat
that are retrieved when that term is used in the Keyword index.  The 5th
column is the count of records in WorldCat that are retrieved when that
term is used in the Subject index.

I expect to have an AutoSuggester built on this today.

Ralph

---

From: Ed Summers <[email-masked]>
Date: Thu, 10 Dec 2009 11:53:30 -0500

> term is used in the Subject index.

Holy wow, this is awesome Ralph.

//Ed

---

From: "LeVan,Ralph" <[email-masked]>
Date: Thu, 10 Dec 2009 16:07:15 -0500

In a rare demonstration of doing as promised, I have a FAST
AutoSuggester running based on that data.  An HTML demonstration of the
service can be found at http://orlabs.oclc.org/FAST/autosuggest.html and
the underlying AutoSuggester is running at
http://orlabs.oclc.org/FAST/AutoSuggest.  Searches from the HTML go to
WorldCat.

An example of a suggestion request would be
http://orlabs.oclc.org/FAST/AutoSuggest?query=0

It returns JSON, which the above HTML link demonstrates the use of.

My AutoSuggester is just a thin interface to an SRU database.  The query
that comes to the AutoSuggester is turned into an SRU query and the
resulting record is dropped into the AutoSuggester response.  The link
to the SRU database for the above query for "0" would be
http://orlabs.oclc.org/identities/search/AutoSuggestFAST?query=fragment+
exact+%220%22

I've put up a file containing all the records I loaded into my database.
The records contain 2 tab delimited fields.  The first field is the key
to the record and corresponds to the user's keystrokes (e.g. '0' above).
The second field contains the JSON response which is an ordered array of
terms.  I load that into the moral equivalent of Lucene and you see the
results in that SRU search.  The file can be found at
http://orlabs.oclc.org/FAST/AutoSuggestFAST.zip

Let me know if you have any problems or questions.

Ralph

> From: LeVan,Ralph
> Sent: Thursday, December 10, 2009 11:12 AM
> Subject: RE: [CODE4LIB] Auto-suggest and the id.loc.gov LCSH web
service
> I have available a list of our FAST subject headings.  They are
derived from
> LCSH and may be of some use.  The folks that produced this file are
working on
> producing a similar file for LCSH.
> 
number of the
> FAST record that the term comes from.  The second column is the MARC
> Authorities field that the term came from.  The third column is the
term itself.  The
> 4th column is the count of records in WorldCat that are retrieved when
that term
> is used in the Keyword index.  The 5th column is the count of records
in
> WorldCat that are retrieved when that term is used in the Subject
index.

---

From: Ethan Gruber <[email-masked]>
Date: Thu, 10 Dec 2009 16:35:27 -0500

Nice work, Ralph.  That's really slick.  I have all the subject terms in the
solr index, but I would like to eventually integrate the Worldcat data to
make the results more relevant (rather than just sorted alphabetically since
each subject occurs only once).

I have yet to adapt my Orbeon forms to handle dynamic querying of the terms
in Solr, but would like to have that done in the next week or two (I hope!).

Here's a sample of the data:

http://beta.scholarslab.org:9080/solr-1.4/terms?terms.fl=subject&terms.limit=25&terms.prefix=Egy

The terms.prefix parameter displays terms that start with the letters
inputted.  It's extremely fast, so I have no doubt I can use it for
autosuggest per keystroke.

Ethan

On Thu, Dec 10, 2009 at 4:07 PM, LeVan,Ralph <[email-masked]> wrote:

> exact+%220%22<http://orlabs.oclc.org/identities/search/AutoSuggestFAST?query=fragment+%0Aexact+%220%22>
>

---

From: "LeVan,Ralph" <[email-masked]>
Date: Thu, 10 Dec 2009 16:41:32 -0500

Thanks!

Ethan, your solr response will look great when you get that ranking
information into it.  I hope we'll have the LCSH equivalent of that FAST
data sometime soon.

Ralph

> Sent: Thursday, December 10, 2009 4:35 PM
service
> http://beta.scholarslab.org:9080/solr-
> 1.4/terms?terms.fl=subject&terms.limit=25&terms.prefix=Egy
> 
exact+%220%22<http://orlabs.oclc.org/identities/search/AutoSuggestFAST?q
ue
> ry=fragment+%0Aexact+%220%22>
> >
> > > The file can be found at
http://orlabs.oclc.org/FAST/fastOutput.zip.

---

From: "Ya'aqov Ziso" <[email-masked]>
Date: Thu, 10 Dec 2009 16:49:19 -0500

O Holy Night, O HOCLC-ly Joy, thanks for the gifts Ralph!

at http://orlabs.oclc.org/FAST/autosuggest.html
I did this:
1. I entered cross reference Œwarming, global¹ and it retrieved 1,247 hits;
I checked the first title (Global warming by Cynthia Bily, 2006), its OCLC
record had ŒGlobal warming¹ as subject. So why did warming, global retrieve
it? 
2. I entered Œglobal warming¹ and got 13,243 hits, all dandy. When I I
clicked into the record, the ŒMore like this¹ had under subjects ŒGlobal
warming¹ listed twice, both retrieving 13,326 hits
3. is there anything in the process that I missed also the indication of the
relationship between main heading ŒGlobal warming¹ and cross reference
ŒWarming, global?

Ya¹aqov

---

From: "LeVan,Ralph" <[email-masked]>
Date: Thu, 10 Dec 2009 17:08:43 -0500

At a guess, it's a flaw in my understanding of how to do a precise WorldCat search.  I threw the phrase "warming, global" at the subject index and it looks to have done an implicit AND and a ranking of the results. The real mystery to me is why "global warming" gets 13 times as many records as "warming, global", if my theory is true.

I'll ask the WorldCat database guys.

As to the "global warming" heading showing up twice, it's because it is actually in the record twice.

Thanks for your kind words and for looking at this!

Ralph

> Ya'aqov Ziso
> Sent: Thursday, December 10, 2009 4:49 PM

---

From: "Ya'aqov Ziso" <[email-masked]>
Date: Fri, 11 Dec 2009 10:12:19 -0500

Ethan, kind thanks, and three questions to you (and I presume UVirginia¹s
Solr refers to Blacklight, thus am CC-ing Vufind-ers):

1. I asked myself Œif WorldCat indexes and upkeeps FAST, why do you need to
index FAST in your local Solr?¹, and I answered: because you want to link
FAST to your local bibs. Is this the case?
2. If it is the case, are you going to load FAST on a weekly basis, cronjob
the procedure? 
3. If you are, what do you need FAST autusoggester for?

Ya¹aqov

---

From: Ethan Gruber <[email-masked]>
Date: Fri, 11 Dec 2009 10:46:50 -0500

Hi Ya-aqov,

Many of the indexes we've built over the last three years or so at the UVa
Library have been Lucene/Solr.  What I am working on is actually in no way
connected to blacklight.  I'm interested in using Solr with the subjects for
autosuggest because I can generate XForms instances dynamically from
TermsComponent.

Ethan

On Fri, Dec 11, 2009 at 10:12 AM, Ya'aqov Ziso <[email-masked]> wrote:

> >> > service can be found at http://orlabs.oclc.org/FAST/autosuggest.htmland
> >> exact+%220%22<
> http://orlabs.oclc.org/identities/search/AutoSuggestFAST?query=
> >> fragment+%0Aexact+%220%22>
> >> >

---

From: "Ya'aqov Ziso" <[email-masked]>
Date: Fri, 11 Dec 2009 11:36:22 -0500

Ethan, so you will be dynamcally generating XForms for each term in the FAST
index? and display what the user selects? how will you make them more
relevant?
Could you point us to any user interface where one can see some sample
snippets? Kindest thanks, Ya¹aqov

On 12/11/09 10:46 AM, "Ethan Gruber" <[email-masked]> wrote:

> Hi Ya¹aqov,

---

From: Ethan Gruber <[email-masked]>
Date: Fri, 11 Dec 2009 12:04:16 -0500

Well, the way it works is that I put all of the LC subject authority
headings into a Solr index and my XForms application reads and parses XML
available through a URL (Solr search results).  I don't have any firm ideas
how to tie relevancy in via Worldcat data, but I have a vague idea how I
will experiment with merging data from fastoutput.txt that Ralph provided
into the Solr index to make terms more relevant rather than simply ordered
alphabetically.  I should be able to make the TermsComponent results come
back by each keystroke in Orbeon, but I'll have to fiddle with the js they
prepackaged in a jar file.

If you go to http://beta.scholarslab.org:9080/orbeon/ead/?guide=t1 and then
click on the tab for archival description, you can scroll down to Controlled
Access Headings.  The Subject input is tied to the terms in Solr, but it's
not dynamic yet.  The autosuggest options are from a static list of 250
terms from the index.  Why those particular terms are shown out of hundreds
of thousands, I don't know.

Ethan

---

From: "Ya'aqov Ziso" <[email-masked]>
Date: Fri, 11 Dec 2009 12:38:56 -0500

Ethan, the dynamic terms, indicating their linking/relevancy also as main or
cross references will be of interest also to the VUfind group.
Kindest thanks, Ya¹aqov