Subject: [CODE4LIB] Notes from the OSS Evaluation Breakout Session, C4L 2009 c4l09

From: Karen Schneider <[email-masked]>
Date: Fri, 6 Mar 2009 11:44:11 -0500

Code4Lib 2009 -- February 25, 2009

Draft notes: Breakout Session: Evaluating Open Source

Input welcome from attendees and anyone else.

----------------

This breakout session started from an informal discussion at a C4L
wine and cheese. The group brainstormed questions about questions to
ask for evaluating open source.

These questions are not absolutes and some of them (perhaps many of
them) will elude clearly objective answers. Also, again and again, the
group pointed out that the presence of a question did not translate to
a requirement or a judgment — these are assessment questions, many of
which will not be relevant to every project and will only translate to
meaningful criteria on a selective basis. Additionally, while many of
these same questions could be applied to any software, the consensus
appeared to be that it was helpful to ask these questions specifically
in the context of OSS.

1.     “Openness” of open source

a.      Describe the license(s) used

b.     Is the code freely and publicly available? Is it easy to find?

2.     Growth and growth management

a.      How widely is the code used?

                                                    i.     How many
organizations are known to use it operationally

                                                  ii.     How many
times has it been downloaded

                                                iii.     Is usage
information tracked and reported?

b.     How long has it been in use?

c.      How many developers are actively involved in the project?

d.     What is the commit activity?

                                                    i.     How are
commits reported?

                                                  ii.     Can commits
be tracked in real-time? How?

e.      Describe the enhancement process.

                                                    i.     Are
enhancement decisions publicly available? Who decides?

f.       Describe bug-tracking: what tools, how bugs are evaluated and
prioritized

                                                    i.     Is the
bug-tracking system publicly available?

g.      Describe QA/testing processes.

h.      How is the software updated?

i.       Is there a migration path to the next version?

j.       Describe the development planning model. Is there upgrade
planning? A commitment to a migration path?

k.      What tools are provided for migrations and upgrades?

l.       Has the project forked, and if so, describe

3.     Community engagement…

a.      Are there user groups? How large are they? How often do they
meet (f2f, virtually, etc.)?

b.     Discussion groups, chat channels, etc.—presence, traffic, availability

c.      Activity of support forum, length of support

d.     Other characteristics of the software community: size, diversity

4.     Governance

a.      Describe the governance model (nonprofit, foundation, etc.)

b.     Is the governance transparent? Describe.

5.     Code and standards

a.      Describe the architecture—languages, structure, etc.

b.     Is the project using version control?

c.      How available is the version control system?

d.     Is there a commercial support option?

e.      Interoperability—describe.

f.       Error logging and reporting—describe

g.      Scalability?

h.      Security? Encryption?

                                                    i.     Does it
provide security auditing tools?

i.       How are permissions set and what are the default permissions?

j.       What platforms does it run on, and how easy is it to
implement on each platform?

k.      For dependencies, does it rely on current versions of those programs?

l.       Does the code hew to de facto or de jure standards? Which ones?

m.    Are key developers active in related standards work?

n.      Does the code include proprietary-source codex, flash players,
etc.—and how is that handled?

6.     Documentation

a.      Is it complete?

b.     Current?

c.      Open?

d.     Written to standards (e.g. Docbook or DITA)?

7.     Innovation and quality

a.      Is it cool at what it does? Is it useful? What’s its karma?
Does it work well? Does it solve a problem? that needs to be solved?

b.     Is it easy to use?

c.      Is it focused on end users (including librarians, if they are
the software’s end users)?

d.     Ease of installation? Consistent results?

e.      Accessibility?

f.       Internationalization?

g.      Business intelligence functions?

h.      Incompatibilities?

i.       Failures and deficiencies?

j.       Awards, reviews, citations?

k.      Certifications?

--
--
| Karen G. Schneider
| Community Librarian
| Equinox Software Inc. "The Evergreen Experts"
| Toll-free: 1.877.Open.ILS (1.877.673.6457) x712
| [email-masked]
| Web: http://www.esilibrary.com
| Be a part of the Evergreen International Conference, May 20-22, 2009!
| http://www.solinet.net/evergreen

---

From: Jodi Schneider <[email-masked]>
Date: Mon, 9 Mar 2009 14:00:19 -0400

Here are my notes from the Plone/Zope breakout session. Pardon my messy 
notetaking! -Jodi
PS-I plan to post a version of these at code4lib.org eventually. If you 
attended and prefer I don't, please let me know.

Vernon Chapman, NAL
Kevin Clarke, Appalachian State University
David Cloutman, Marin County Free Library
Douglas Hahn, Texas A&M University
Tom Ceresini, PALINET
Heather Rolan, Yeshiva University
Jodi Schneider, Appalachian State University
Jon Stroop, Princeton University (Metadata Analyst)
================================

repose.bfg ---can run plone/zope right through apache

================================
http://library.tamu.edu

1.5 - 2 years (1 year in production)

from Vignette ($ license, complicated end-user interface). Didn't have 
SDK's, hard to get content contributed.  Content contributor user 
interface (Drupal easier for programmers, sucks for end-user.) 100-150 
content editors  40,000 users

Subject guides, finding aids, class guides--custom metadata types

Pain points: caching, sending out the content, maintaining speed, 
getting locks on content (hard-coded to try 2 times--changed to 99 times)

Run varnish (varnish-cache.com), pound (no longer needed if you run 
varnish 2.0), 2 zeo servers ZODB -- more complicated architecture than 
they would like
Want load w/i 1 second

6 month ramp-out time, wanted display framework
================================
layers in front may be necessary (e.g. searching, load 
balancing)--except for

Content mirror--dumped to DB, pulled to Django
http://blog.kapilt.com/2008/07/02/introducing-content-mirror/

Solr--for searching

add a writer to have a feed--Atom, RSS

120-150
LOTS of checks--wear and tear on performance--an issue for PUBLIC content
================================

plinket using plone

repoze.org--
wsgi---web services gateway interface

http://grok.zope.org/

================================
Plone is the display layer for Zope

================================

http://www.amazon.com/Professional-Plone-Development-content-centric-applications/dp/1847191983/

================================
Can deal with multiple levels:

IA, web design, programming
ZMI level/CSS level, python base code

================================
portal types/content types
copy and paste--problem is when you upgrade, it goes away.

================================
Comparison to http://www.djangoproject.com/ framework

================================
ZODB compared to RDBS:
easy to backup: text file. just do a diff
takes anything (large stuff--videos ok)

================================
Where in the stack something is?

================================
What you like about Plone:

inline editing
versioning
approve/publishing
WSIWIG editors
"just works" if it fits your content types

"Spent all our time on making it pretty" -- "Didn't spend ANY time on 
content editors"
================================

================================

Ask TAMU: Can they share their subject guide content type?
(took 20 hours for them to do--e.g. 1 20 hr/wk --one )

class guide
FAQ

forms took longer--now has form builder product--complex forms with 
javascript checking

UI work on display

Goes out to solr--associated journals, etc.
================================
Tom went to online training from 7technw
4 hrs/day for 3 days, relatively introductory

================================
Clouseau: while developing, can have live python command line

Firebug
Web Developer Toolbar

================================
sole developer
understand IRC

---

From: Nicole Engard <[email-masked]>
Date: Thu, 12 Mar 2009 08:34:18 -0400

Karen,

Are you going to post this on the web somewhere? I'd love to link to
it in a blog post - if not I'll just copy and paste :)

---

Nicole C. Engard
Open Source Evangelist, LibLime
(888) Koha ILS (564-2457) ext. 714
[email-masked]
AIM/Y!/Skype: nengard

http://liblime.com
http://blogs.liblime.com/open-sesame/

---

From: Karen Schneider <[email-masked]>
Date: Thu, 12 Mar 2009 08:36:27 -0400

Nicole, here it is from one archive:

http://serials.infomotions.com/code4lib/archive/2009/200903/0329.html

I think it might be archived elsewhere too -- Gmane or Markmail or
something?

---

From: Jodi Schneider <[email-masked]>
Date: Thu, 12 Mar 2009 09:24:10 -0400

Ideally, we'll link breakout session notes from
http://code4lib.org/conference/2009/breakouts

Anybody have time to do that?

Maybe a separate page at code4lib.org for each breakout?

-Jodi

PS-Currently these sessions are listed:
Tuesday breakout sessions

    * Plone/Zope
    * OCLC Grid Services - Don Hamparian
    * Jangle

Wednesday breakout sessions

    * Summon

Please add the one you attended.