References
Marcia Bates. Indexing and access for Digital Libraries and the Internet. Journal of the American Society for Information Science 49 (November 1998): 1185 Ð 1205. Available http://www.gseis.ucla.edu/faculty/bates/articles/indexdlib.html
J. Stephen Downie. "Who, What, When,Where and Why: Introduction and Acknowledgements (first Edition). Papers Presented at the Workshop on the Creation of Standardized Test Collections, Tasks, and Metrics for Music Information Retrieval (MIR) and Music Digital Library (MDL) Evaluation, 18 July, 2002. Available: http://music-ir.org/evaluation/wp2/wp1_downie_who.pdf
J. Stephen Downie. Establishing Music Information Retrieval (MIR) and Music Digital Library (MDL) Evaluation Frameworks: Preliminary Foundations and Infrastructures. Papers Presented at the Workshop on the Creation of Standardized Test Collections, Tasks, and Metrics for Music Information Retrieval (MIR) and Music Digital Library (MDL) Evaluation, 18 July, 2002. Available. http://music-ir.org/evaluation/wp2/wp1_downie_establishing.pdf [Nov 3. 2002]
J. Stephen Downie. Interim Report on Establishing MIR/MDL Evaluation Frameworks: Commentary on Consensus Building. Panel on Music Information Retrieval Evaluation Frameworks at ISMIR 2002, 17 October, 2002. Available. http://music-ir.org/evaluation/wp2/wp2_downie_con.pdf [Nov 3. 2002]
J. Stephen Downie. "MIR/MDL Evaluation: Making
Progress" SIGIR 2003: Workshop
on the Evaluation of Music Information Retrieval (MIR) Systems, August
1, 2003, Toronto, Canada. Available:
J/ Stephen Downie. "The TREC Like Evaluation of Music Retrieval Systems" Appendix
C of The MIR/MDL Evaluation Project White Paper Collection, Edition
#3 Available: http://music-ir.org/evaluation/wp3/wp3_appendixC.pdf
J. Stephen Downie (ed.) The MIR/MDL Evaluation Project White Paper Collection, Edition #3 Includes: Part I. Papers Presented at the Workshop on the Creation of Standardized Test Collections, Tasks, and Metrics for Music Information Retrieval (MIR) and Music Digital Library (MDL) Evaluation, 18 July, 2002. Part II. Panel on Music Information Retrieval Evaluation Frameworks at ISMIR 2002, 17 October, 2002. and Part III. Workshop on the Evaluation of Music Information Retrieval (MIR) Systems at SIGIR 2003, 1 August, 2003. Available http://music-ir.org/evaluation/wp.html [December 2004]
Charles Rhyne. "Images as Evidence in Art History and Related Images" Museums and the Web 97: Selected Papers Pittsburgh: Archives & Museum Informatics, 1997, 347-361 Available http://www.reed.edu/~cryhne/papers/images.html
Charles Rhyne. Computer Images for Research, Teachning
and Publication in Art History and Related Disciplines. Visual Resources.
An International Journal of Documentation. Vol
XIL (1995) 19-51. Available: http://www.reed.edu/~crhyne/papers/computer.html
Dagobert Sorgel. Indexing and Retrieval Performance: The Logical Evidence. Journal of the American Society for Inforamation Science. 45:8, 589-599. 1994.
Text REtrieval Conference (TREC). web site. http://trec.nist.gov/overview.html
An initiative developing set of genomics-focussed retrieval tasks to be evaluated at TREC. Chaired by William Hersh, Professor and Head, Division of Medical Informatics & Outcomes Research, Oregon Health & Science University http://medir.ohsu.edu/~genomics/index.html
Available: http://www-nlpir.nist.gov/projects/t2002v/t2002v.html
Ellen Voorhees, Chair, TREC Program Committee, September 2002 Available: http://trec.nist.gov/trec.tracks.html
The Tenth Text Retrieval Conference (TREC-2001) Proceedings.
NIST Special Publication 500-250 http://trec.nist.gov/pubs/trec10/t10_proceedings.html
Donna Harman. Overview of the first Text Retrieval Conference (TREC-1). NIST Available http://trec.nist.gov/pubs/trec1/papers/01.txt
Ellen M. Voorhees. Whither Music IR Evaluation Infrastructure: Lessons to be learned from TREC. Papers Presented at the Workshop on the Creation of Standardized Test Collections, Tasks, and Metrics for Music Information Retrieval (MIR) and Music Digital Library (MDL) Evaluation, 18 July, 2002 Available http://music-ir.org/evaluation/wp2/wp1_vorhees.pdf:
Ellen Voorhees , Chair TREC Program Committee, Procedure for Proposing New TREC Tracks. [2002]. http://trec.nist.gov/trec.tracks.html
Burford, B, Briggs, P, & Eakins, JE "A Taxonomy of the Image: On the
Classification of Content for Image Retrieval" Visual Communication 2(2), 2003. 123-161.
A.E. Cawkell. Selected aspects of image processing and management: review and future prospects. Journal of Information Science 8:3 179-192.
Hsin-lian Chen and Edie M. Rasmussen. "Intellectual Access to Images". Library Trends, Special Issue " Progress in Visual Information Access and Retrieval Edited by Beth Sandore. 48:2, Fall 1999.
Youngok Choi and Edie M. Rasmussen, "Searching for Images: the Analysis of Users' queries for Image Retrieval in American History". Journal of the American Society for Information Science and Technology, 54:6 498-511, 2003
Heting Chu. Research in Image Indexing and Retrieval as Reflected in the Literature. Journal of the American Society for Information Science and Technology. 52(12) 1011-1018, 2001
Lynne R Conniss, A Julie Ashford and Margaret E Graham, Information seeking behaviour in image retrieval: VISOR I final report. Newcastle-upon-Tyne: Institute for Image Data Research, 2000. 147pp. (Library and Information Commission Research Report 95)
J P Eakins,. Towards intelligent image retrieval. Pattern Recognition. 35, 2002, 3-14.
John Eakins and Margaret Graham.Content-based Image Retrieval. JISC Technology Applications Programme Report 39.JTAP Report
Peter Enser. Visual image retrieval: Seeking the alliance of concept-based and content-based Journal of Information Science. 26: 4. 2000, pg. 199-210
A..Goodrum, A. Image Information Retrieval: An Overview of Current Research" Informing Science. Vol. 2, No.2, 2000 33-66.
Abby A Goodrum, Mark E. Rorvig, Ki-Tai Jeong, Chitturi Suresh. "An Open Source Agenda for Research Linking Test and Image Content Features". Journal of the American Society for Information Science and Technology. 52(11) 948-953. September. 2001.
M E. Graham, The cataloguing
and indexing of images: time for a new paradigm? Art Libraries Journal. 26(1), 2001, p22-27.
Jane Greenberg. A quantitative Categorical Analysis
of metadata elements in image applicable metadata schemas. Journal
of the American Society for Information Science and Technology. 52:11
917-924 September 2001.
P. Bryan Heidorn and Beth Sandore, the 33rd Annual Clinic
on Library Applications of Data Processing, University of Illinois, Urbana-Champaign,
March 1996, Urbana-Champaign: University of Illinois at Urbana- Champaign,
1997, 29-41.
Alejandro Jaimes and Shih-Fu Chang. "A conceptual Framework for Idexing Visual Information at Multiple levels. I&ST/SPIE Internet Imaging. Vol 3964. San Jose, CA, Jan 2000. Available http://www.ctr.columbia.edu/~ajaimes/Pubs/spie00_internet.pdf
Corinne šrgensen. "Image Indexing: An analysis of selected classification systems in relation to iamge attributes named by na•ve users. Annual Review of OCLC Research 1999. Available: http://www.oclc.org/research/publications/arr/199/jorgensen/ [August 15, 2002]
Corinne Jšrgensen, "Introduction and Overview" Journal of the American Society for Information Society and Technology. 52(2): 906:910. 2001.
Kevin Kiernan, Charles Rhyne and Ron Spronk, Digital Imagery for Works of Art, A report of a meeting at the Harvard University Art Museums, Cambridge, Massachusetts, November 19-20, 2001 Sponsored by The Andrew W. Mellon Foundation, The National Science Foundation, The Harvard University Art Museums. Available: http://www.dli2.nsf.gov/mellon/index.html
Paul Lewis, David Dupplaw and Kirk Martinez. "Content-Based Multimedia Information Handling: Should we Stick to metadata?" Cultivate Interactive, Issue 6 February 2002.
Erwin Panofsky, Studies in iconology: Humanistic themes in the Art of the Renaissance New York: Harper & Row, 1962, 3-31.
Olle Persson. "Image indexing Ð a first author co-citation map" Available: http://www.umu.se/inforsk/Imageindexing/imageindex.htm [Nov 4, 2002]
Henry Pisciotta, Roger Brisson, Eric Ferrin, Michael Dooris, and Amanda Spink. Penn State Visual Image User Study D-Lib Magazine Volume 7 Number 7/8. July/August 2001 Available: http://www.dlib.org/dlib/july01/pisciotta/07pisciotta.html [Nov. 6, 2002]
Edie Rasmussen "Indexing Images", Annual Review of
Information Science and Technology (ARIST)
32 (1997)169-196.
Helene E. Roberts, "A Picture is Worth a Thousand Words: Art Indexing in Electronic Databases". Journal of the American Society for Information Scienct and Technology. 52(11) 911-916. 2001.
Beth Sandore (ed.), "Progress in Visual Information Access and Retrieval" (Thematic Issue), Library Trends, 48, #2 (Fall, 1999)
Sarah Shatford. Analysing the subject of a picture: A theoretical approach". Cataloguing and Classification Quarterly.6. 39-62.
Sara Shatford Lane. "Some Issues in the Indexing of Images". Journal of the American Society for Information Science. 45:8, 583-588, September 1994.
Smeulders, Arnold W. M., Worring, Marcel, Santini, Simone, Gupta, Amarnath;,Jain, Rameth. Content-based image retrieval at the end of the early
years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22
(12), December (2000), 1349-1379.
Elaine Svenonius. Access to Nonbook materials: The Limits of Subject Indexing for Visual and Aural Languages. Journal of the American Society for Information Science. 45:8, 600-606. 1994.
Kobus Barnard, Pinar Duygulu, David Forsyth, Nando de Freitas,David M. Blei, Michael I. Jordan; Matching Words and Pictures JMLR Special Issue on Machine Learning Methods for Text and Images3(Feb):1107-1135, 2003 Available: http://www.jmlr.org/papers/volume3/barnard03a/barnard03a.pdf
Kobus Barnard, Pinar Duygulu, and David Forsyth, Clustering Art, Computer Vision and Pattern Recognition, 2001, pp. II:434-439. Available http://www.cs.arizona.edu/people/kobus/research/publications/CVPR-01/index.html [August 17, 2002].
Kobus Barnard and David Forsyth, "Exploiting Image Semantics for Picture Libraries' The First ACM/IEE-CS Joint Conference on Digital Libraries, 2001. p 469
Available: http://www.cs.arizona.edu/people/kobus/research/publications/JCDL-01/index.html
Kobus Barnard, Pinar Duygulu, Nando de Freitas, David Forsyth, David Blei, and Michael I. Jordan, "Matching Words and Pictures", submitted for publication to JMLR. Available: http://www.cs.arizona.edu/people/kobus/research/publications/JMLR/index.html [Nov. 3, 2002]
V. Castelli, L. D. Bergman, I. Kontoyiannis, C.-S.
Li, J. T. Robinson and J. J. Turek,
on "Progressive search and retrieval in large image archives" IBM Journal
of Research and Development, Vol. 42, No. 2, Multimedia Systems 1998. Available: http://www.research.ibm.com/journal/rd/422/castelli.html [July
2003].
Pinar Duygulu, Kobus Barnard, Nando de Freitas, and David Forsyth, "Object recognition as machine translation: learning a lexicon for a fixed image vocabulary' Seventh European Conference on Computer Vision, IV:97-112, 2002. available: http://www.cs.arizona.edu/people/kobus/research/publications/ECCV-02-1/ECCV-02-1.pdf
Eakins, John and Margaret Graham, Content-based Image Retrieval, JISC Technology Applications Programme (JTAP). January 1999. Available http://www.jisc.ac.uk/jtap/htm/jtap-039.html. [August 12, 2002] http://www.unn.ac.uk/iidr/report.html [August 14, 2002]
Jing Huang and Ramin Zabih. Combining Color and Spatial Information for Content-based Image Retrieval. European Conference on Digital Libraries. 22-24 September 1999 Available http://www.cs.cornell.edu/rdz/Papers/ecdl2/spatial.htm [Aug 15, 2002]
Jia Li, James Z. Wang, Gio Wiederhold, ``IRM: Integrated region matching for image retrieval,'' Proceedings of the 2000 ACM Multimedia Conference, 147-156, Los Angeles, ACM, October, 2000.
Jia Li and James Z. Wang, "Automatic Linguistic
Indexing of Pictures by a Statistical Modeling Approach". IEEE Transactions
on Pattern Analysis and Machine Intelligence.
Vol 25, no 10. Octover 2003.
Yong Rui, Thomas S. Huang, Shih-Fu Chu. Image Retrieval: Past, Present and Future. International Symposium on Multimedia Information Processing, 1997 Available:
Venters, C. C., and Cooper, M. D. A Review of Content-Based Image Retrieval Systems. JISC Technology Assessment Program Report, March 2000. Available http://www.jtap.ac.uk/reports/htm/jtap-054.html [August 12, 2002]
Annette A. Ward , Margaret Graham, Jonathan Riley, Neil Eliot, and John Eakins, Nic Sheen, and Cathy Pringle. "Collage and Content-based Image Retrieval: Collaboration for Enhanced Services for the London Guildhall Library". Museums and the Web 2001. Archives & Museum Informatics. Available http://www.archimuse.com/mw2001/papers/ward/ward.html [August 15, 2002]
James Z. Wang, Jia Li, Gio Wiederhold, ``SIMPLIcity: Semantics-sensitive Integrated Matching for Picture LIbraries,'' IEEE Trans. PAMI, vol 23, no.9, pp. 947-963, 2001.
Zoran Pecenovic, "Image Retrieval using Latent Semantic Indexing" Final year graduate thesis, Department of Electrical Engineering Audiovisual Communications Laboratory, L'Ecole Polytechnique Fédérale de Lausanne, May, 1997.Available: http://lcavwww.epfl.ch/LSI/documents/final.html
Methods for constructing such systems are discussed at http://www.benchathlon.net/activity/index.html, and in more general terms at http://viper.unige.ch/benchmarking/
Kobus Barnard and Nikhil V. Shirahatti, "A method for comparing content based image retrieval methods", Internet Imaging IX, Electronic Imaging 2003. Available: kobus.ca/research/publications/SPIE-03-bench/SPIE-03-bench.pdf
Kobus Barnard, Lindsay Martin, Brian Fund and Adam Coath, "A Data Set for Color Research," Color Research and Application. 27:3, 148-152, 2002.
Kobusbus Barnard, Brian Funt, Vlad Cardei "A Comparison of Computational Color Constancy Algorithms; Part One: Methodology and Experiments with Synthesized Data," IEEE Transactions in Image Processing, Vol. 11, No. 9, pp. 972-984 Available: http://www.cs.arizona.edu/people/kobus/research/publications/comparison_1/index.html [Nov 3. 2002]
Charly Bauer. Relevace Judgements for Images: Pilot Study. Unpublished research paper, Spring 1997. Available: http://images.grainger.uiuc.edu/research/Relevance/MARS.html
John A. Black, Jr., Gamal Fahmy, and Sethuraman Panchanathan
A Method for Evaluating the Performance of Content-Based Image Retrieval Systems Based on Subjectively Determined Similarity between Images", Image and Video Retrieval, International Conference, CIVR 2002, London, UK, July 18-19, 2002, Proceedings. Lecture Notes in Computer Science 2383 Springer 2002, p. 356 ff. Available: http://link.springer.de/link/service/series/0558/bibs/2383/23830356.htm [August 15, 2002].
Council on Library and Information Resources, Research Libraries Group and the Digital Library Federation. and Research Libraries Group, Guides to Quality in Visual Resource Imaging. July 2000 (http://www.rlg.org/visguides/).
Neil J. Gunther, Giordano Beretta . A Benchmark for Image Retrieval using Distributed Systems over the Internet: BIRDS--I . HP Labs, Palo Alto, Technical Report HPL--2000--162, San Jose, 2001
Donna Harman. The Text Retrieval Conferences (TRECs): Providing a test-bed for information retrieval systems. ASIS Bulletin April/May1998. Available http://www.asis.org/Bulletin/Apr-98/harman.html
Donna Harman, What we have learned, and not learned from the TREC conferences, TREC/ NIST, April 5-7, 2000, Cambridge, England (http://www.itl.nist.gov/iaui/894.02/works/presentations/bcs-irsg/index.htm
Nicholas R. Howe. "Using artificial queries to evaluate image retrieval. IEEE Workshop on Content-based Access of Image and Video Libraries. in Proceedings of CBAIVL '00, 12 June 2000 in Hilton Head, South Carolina. 5-9
Donna Harman , "The Text REtrieval Conferences (TRECs): Providing a Test-Bed for Information Retrieval Systems" in The Bulletin of the American Society for Information Science, April 1998 (http://www.asis.org/Bulletin/Apr-98/harman.html)
Corinne Jšrgensen and Rohini K. Srihari. Creating a Web-Based Image database for benchmarking image retrieval. Proceedings of SPIE. Vol. 3644, 534-541.
Marjo Markkula and Eero
Sormunen, "End-User Searching Challenges Indexing Practices
in the Digital Newspaper Photo Archive", Information
Retrieval, no.4 2000,
259-285.
Henning MŸller, Wolfgang MŸller, David McG. Squire, Stéphane Marchand--Maillet, Thierry Pun . Performance Evaluation in Content--Based Image Retrieval: Overview and Proposals , Pattern Recognition Letters, Vol. 22, No. 5, pp. 593--601, 2001.
Wolfgang MŸller, Stéphane Marchand-Maillet, Henning MŸller, Dsvid McG. Squire, and Theirry Pun. "Evaluating Image Browsers Using Structured Annotation." Journal of the American Society for Information Science and Technology. 52(11):961-968, September 2001.
Henning MŸller, Wolfgang MŸller, Stéphane Marchand-Maillet, Theirry Pun and David McG Squire. "Automated Benchmarking in Content Based Image Retrieval. Tech. Rep. 01.01, University of Geneva, May 2001. Available: http://vision.unige.ch/publications/postscript/2001/MuellerHMuellerWMarchandSquirePun_tr01.pdf
Henning MŸller, Stephane Marchand-Maillet, and Thierry Pun, "The Truth about Corel - Evaluation in Image Retrieval". Image and Video Retrieval, International Conference, CIVR 2002, London, UK, July 18-19, 2002, Proceedings. Lecture Notes in Computer Science 2383 Springer 2002, p. 38 ff. Available http://link.springer.de/link/service/series/0558/bibs/2383/23830038.htm [August 15, 2002].
Edie Rasmussesn. Evaluation in Information Retrieval. Panel on Music Information Retrieval Evaluation Frameworks at ISMIR 2002, 17 October, 2002 Available: Panel on Music Information Retrieval Evaluation Frameworks at ISMIR 2002, 17 October, 2002
K. Rodden, W. Basalaj, D. Sinclair, and K. Wood, Does
Organisation by Similarity Assist Image Browsing?, ACM Conference on
Human Factors in Computing Systems (ACM CHI 2001), Seattle, April 2001.Available:
Kerry Rodden, Wojciech Basalaj, David Sinclair and Kenneth Wood. "Evaluating a Visualization of Image Similarity. Poster. Proceedings of SIGIR'99. ACM August 1999.
* Schmidt, C. & Over, P. Digital Video Test Collection. In proceedings of the Twenty-Second Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, Berkeley, CA, USA. (1999, August). Available: http://www.itl.nist.gov/iad/894.02/works/papers/digital.video.html [Nov 3, 2002]
Slaughter, L., Marchionini, G. and Geisler, G. Open Video: A framework
for a test collection. Journal of Network and Computer Applications, 23(3):
219-245, July 2000.
Alan F. Smeaton, Paul Over and Ramazan Taban. The TREC-2001 Video Track Report. NIST Special Publication 500-250: The Tenth Text REtrieval Conference (TREC 2001) Available: http://trec.nist.gov/pubs/trec10/papers/TREC10Video_Proc_Report.pdf
John.R. Smith. "Quantitative Assessment of Image Retrieval Effectiveness." Journal of the American Society for Information Science and Technology. 52(11): 969-979. September 2001.
Conceptual Modeling of audio-visual content. IEEE International Conference on Multimedia and Expo (CcME), New York, NY.
Sormunen E., Markkula M. and JŠrvelin K. (1999). The Perceived Similarity of Photos - A Test-Collection Based Evaluation Framework for the Content-Based Image Retrieval Algorithms. In: Draper S. et al., eds. Mira 99: Evaluating interactive information retrieval. Electronic Workshops in Computing. Available: http://www.ewic.org.uk/ewic/workshop/fetch.cfm/MIRA-99.pdf
Voorhees, E., & Harman, D. Overview of the Sixth Text REtrieval Conference (TREC-6). In proceedings of the Sixth Text REtrieval Conference (TREC-6), Gaithersburg, Maryland, USA., November 1997. Available: http://trec.nist.gov/pubs/trec6/papers/overview.ps.gz
James Z. Wang, Jia Li, Sui Ching Lin, ``Evaluation strategies for automatic linguistic indexing of pictures,'' Proc. IEEE International Conference on Image Processing (ICIP), Barcelona, Spain, pp. -, IEEE, September 2003. Available: http://www-db.stanford.edu/~wangz/project/imsearch/ALIP/ICIP2003/
Don Williams, "An Overview of Image Quality Metrics," in Moving Theory into Practice: Digital Imaging for Libraries and Archives, Anne R. Kenney and Oya Y. Rieger (editors and principle authors). Mountain View, CA: Research Libraries Group, 2000
Researchers in metadata based retrieval, or hybrid retrieval systems that use metadata, are less organized than their peers in CBIR. In the absence of a complete bibliography, some recent and useful articles include:
Angeles, M. Information Organization and Information Use of Visual Resources Collections. VRA Bulletin, 25 (3), 51-58. available: http://studioid.com/articles/vruse/ [August 15, 2002]
Barnett, P.J. (1988). An art information system: From integration to interpretation. Library Trends, 37 (2), 194-205.
Dan Brickley, Harmany: a mid-term report (Or "Resource Discovery for Multimedia Metadata" revisited...) http://www.ilrt.bris.ac.uk/discovery/2001/03/multimeta/ March 2001
Abby Goodrum, M. Rorvig, K. Jeong and C. Suresh. An Open Source Agenda for Research Linking Text and Image Content Features. Journal of the American Society of Information Science, Available:
http://www.unt.edu/ir/papers/jasisg00.htm [August 15, 2002]
Andrew S. Gordon. "Browsing Image Collections with Representations
of Common-Sense Activities". Journal of the American Society for Information
Science and Technology 52(11): 925-929,
Sept. 2001.
Jane Greenberg. "A Quantitative Categorical Analysis of Metadata Elements in Image-Applicable Metadata Schemas." Journal of the American Society for Information Science and Technology, 52(11): 917-924 September 2001
Samantha Hastings. Query categories in a study of intellectual
access to digitized art images. American Society for Information Science,
annual meeting, Chicago IL, October 1995.
Jane Hunter. Combining the CIDOC CRM and MPEG-7 to Describe
Multimedia in Museums. Museums and the Web 2002. Archives & Museum
Informatics.2002.
2002Available http://www.archimuse.com/mw2002/papers/hunter/hunter.html [August 15, 2002]
Jane Hunter and Zhimin Zhan "An Indexing and Querying System for Online Images, Based on the PNG Format and Embedded Metadata" September, 1999 http://archive.dstc.edu.au/RDU/staff/jane-hunter/PNG/paper.html
Corinne Jšrgensen. "Image Indexing, an Analysis of Selected Classification Systems in relation to Image Attributes names by Na•ve Users". Annual Review of OCLC Research, 1999. Available: http://www.oclc.org/research/publications/arr/1999/jorgensen/ [August 15, 2002].
Corinne Jšrgensen. "A Conceptual Framework and Empirical Research for Classifying Visual Descriptors:. Journal of the American Society for Information Science and Technology. 52(11): 938-947, September 2001.
Guojun Lu and Ben Williams, Gippsland School of Computing and Information Technology Monash University, An Integrated WWW Image Retrieval System AusWeb99, The Fifth Australian World Wide Web Conference, held in Balliina, NSW, Australia from 17-20 April, 1999 http://ausweb.scu.edu.au/aw99/papers/lu/paper.html)
Markey, K. (1988). Access to iconographical research collections. Library Trends, 37 (2), 154-174.
Kathleen Perez-Lopez, Brian Krasner, Gregory Baraghimian Mark Berlin, Smart Metadata for Content-based Retrieval from Large Image Databases, http://www.computer.org/conferences/meta96/krasner/perez_lopez.html
Helene E. Roberts: A picture is worth a thousand words: Art indexing in electronic databases. JASIST 52(11): 911-916 (2001)
Shatford, S. (1986, Spring). Analyzing the subject of a picture: A theoretical approach. Cataloging & Classification Quarterly, 6 (3), 39-62.
A.M.Tam and C.H.C. Leung. "Structured Natural-Language Descriptions for Semantic Content Retrieval of Visual Materials". Journal of the American Society for Information Science and Technology. 52(11):930:937. September 2001.
CNI/OCLC Image Metadata Workshop, September 24 - 25, 1996 (3rd Dublin Core Workshop),
http://www.dlib.org/dlib/january97/oclc/01weibel.html
http://www.library.cornell.edu/cts/elicensestudy/home.html
http://www.amico.org/AMICOlibrary/dataspec.html [Aug. 15, 2002]
Categories for the Description of Works of Art. The Report of the Art Information Task Force. http://www.getty.edu/research/institute/standards/cdwa/
Proposed ISO 21127. Available http://cidoc.ics.forth.gr/index.html [Aug. 15, 2002].
A User Model for CIMI Z39.50 Application Profile, Kody Janney & Jane Sledge, September 1995 Available http://www.cimi.org/public_docs/Z3950_app_profile_0995.html [Aug 15, 2002]
CIMI
XML Schema for SPECTRUM
http://www.cimi.org/wg/xml_spectrum/index.html
A standard for describing social science datasets, represented as an XML DTD.
http://dublincore.org/groups/admin/
A set of 15 elements for simple resource description, used heavily in digital library applications, especially for visual materials and items in collections where full cataloging may not be warranted. http://www.dublincore.org [Aug. 12, 2002]
Machine-readable representation of archival finding aids, represented as an SGML DTD.
The FGDC maintains this portal to geospatial metadata resources, including the Content Standard for Digital Geospatial Metadata (CSDGM).
The LC Network Development and MARC Standards Office maintains this site relating to MARC Standards, which includes an HTML version of the MARC21 Concise Format for Bibliographic Data and other documentation.
An emerging schema for encoding descriptive, administrative, and structural metadata regarding objects within a digital library.
International Standards Organisation / Organisation internationale de normalization, ISO/IEC JTCL/SC29/WG11 Coding of moving pictures and audio. Klangenfurt, July 2002 Available at:
http://mpeg.telecomitalialab.com/standards/mpeg-7/mpeg-7.htm
NISO Z39.87-2002 AIIM 20-2002 Data Dictionary - Technical Metadata for Digital Still Images (pdf format) Available: http://www.niso.org/standards/resources/Z39_87_trial_use.pdf [Aug. 12, 2002]
SPECTRUM: The UK Museum
Documentation Standard, Second
edition Revised and edited by Jeff Cowton
Visual Resources Association Data Standards Committee. VRA Core Categories, Version 3.0
A metadata standard for describing visual resources, related to but much smaller than the Getty Categories for the Description of Art. Available http://www.vraweb.org/vracore3.htm [Aug. 12, 2002]
http://www-2.cs.cmu.edu/~cil/txtv-groups.html.
University of Geneva list of Groups involved in Image Retrieval
http://viper.unige.ch/other_systems/
http://www-2.cs.cmu.edu/~cil/vision.html
A collaboration headed by the University of Geneva to develop image retrieval benchmarks. http://www.benchathlon.net/.
http://www.benchathlon.net/resources/publiByYear.html
http://whitechapel.media.mit.edu/vismod/
previously at Stanford; http://wang.ist.psu.edu/IMAGE; recipient of NSF STIMULATE grant
http://www.elib.cs.berkeley.edu
http://elib.cs.berkeley.edu/vision.html
http://elib.cs.berkeley.edu/kobus/famsf/model_2/text_and_blobs/bbox.html
Visual Information Processing for Enhanced Retrieval; http://viper.unige.ch/ http://viper.unige.ch/~marchand/research/
http://www.cs.washington.edu/research/imagedatabase
Recent awards include
NSF Org IIS
Latest Amendment Date June 21, 2002
Award Number 0097329
Award Instrument Continuing grant
Program Manager Bhavani Thuraisingham
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date September 15, 2001
Expires August 31, 2004 (Estimated)
Expected Total Amount $270000 (Estimated)
Investigator Linda G. Shapiro shapiro@cs.washington.edu (Principal Investigator current)
Sponsor U of Washington
3935 University Way NE
Seattle, WA 981056613 206/543-4043
NSF Program 6855 INFORMATION & DATA MANAGEMENT
Field Application 0104000 Information Systems
Program Reference Code 9218,HPCC,
Abstract
The goal of this research is to develop the necessary methodology for automated recognition of generic object and concept classes (such as buildings, cars, boats, and trees) in digital images in order to substantially improve the process of content-based image retrieval, which has relied mainly on low-level color and texture features for matching queries to database images. The approach has three major aspects: (1) to design new high-level image features including cluster features that group together lower-level features and relationship features that capture spatial relationships among them; (2) to develop a unified representation that can express a large variety of both low- and high-level features in a form that can be used by learning systems; and (3) to automate the development of recognizers for object and concept classes through the use of a hierarchical, multiple classifier methodology. The resulting techniques are being evaluated on several different large image databases, including commercial databases whose images are grouped into broad classes and a ground-truth database that provides a list of the objects in each image. The results of this work will be a new generic object and concept recognition paradigm that can immediately be applied to automated or semi-automated indexing of large image databases. The methodology will help to bridge the gap between the high-level needs of users of image retrieval systems and the low-level features typically extracted from an image. The generic object class recognition algorithms we develop will begin a new era of object recognition research, leaving the geometric domain and entering the conceptual domain.
https://www.fastlane.nsf.gov/servlet/showaward?award=0097329
NSF Org IIS
Latest Amendment Date July 14, 1999
Award Number 9711771
Award Instrument Continuing grant
Program Manager Maria Zemankova
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date September 15, 1997
Expires August 31, 2001 (Estimated)
Expected Total Amount $247000 (Estimated)
Investigator Linda G. Shapiro shapiro@cs.washington.edu (Principal Investigator current)
Sponsor U of Washington
3935 University Way NE
Seattle, WA 981056613 206/543-4043
NSF Program 6855 INFORMATION & DATA MANAGEMENT
Field Application 0104000 Information Systems
Program Reference Code 9139,HPCC,
Abstract
The goal of this research project is to design and implement a system for content-based image retrieval that can (1) provide a large variety of image-distance measures that can be used singly or in combination to satisfy a wide range of user needs and (2) provide rapid access to images, even in an extremely large database. The focus of the work is the development of a general, scalable architecture to support fast querying of very large image databases with user-specified distance measures. This includes the development of distance-measure-independent algorithms and data structures for efficient image retrieval from large databases. Methods for merging the general, distance-measure-independent algorithms with other useful techniques that may be distance measure specific, such as keyword retrieval and relational indexing, are being pursued. The problem of providing users with multiple distance measures of many different varieties is being studied. New methods for combining distance measures and a language in which users can specify their queries without detailed knowledge of the underlying metrics are being designed. A prototype system is being implemented to test the developed methods, and evaluation is being performed on both a large general image database and a smaller controlled database. The results of this research will be: (1) techniques that facilitate rapid retrieval of images by eliminating huge portions of the image database from the search, making content-based retrieval feasible on very large and growing databases; (2) new, high-level methods by which users can combine distance measures to form meaningful queries, so that content-based queries can become a standard way to query image databases; and (3) a general framework for content-based retrieval that can accommodate new distance measures as they are developed by other research efforts. The work has application to medicine, art, photography, entertainment, and advertising/marketing. https://www.fastlane.nsf.gov/servlet/showaward?award=9711771
NSF Org DUE
Latest Amendment Date September 10, 2001
Award Number 0121596
Award Instrument Standard Grant
Program Manager Lee L. Zia
DUE DIVISION OF UNDERGRADUATE EDUCATION
EHR DIRECT FOR EDUCATION AND HUMAN RESOURCES
Start Date January 1, 2002
Expires August 31, 2003 (Estimated)
Expected Total Amount $494424 (Estimated)
Investigator Scott T. Acton acton@virginia.edu (Principal Investigator current)
Sponsor University of Virginia
Post Office Box 9003
Charlottesville, VA 229069003 804/924-0311
NSF Program 7444 NATIONAL SMETE DIGITAL LIBRARY
Field Application 0000099 Other Applications NEC
Program Reference Code 7444,9178,SMET,
Abstract
The Decentralized Image Retrieval for Education (DIRECT) project is developing a peer-to-peer content based image retrieval (CBIR) service for the National SMETE Digital Library program. CBIR allows the user to designate a query image so that the service can search the library for images of similar content. DIRECT matches images not by text metadata but by the color, texture, and shape of the image objects. With such a system the users of the NSDL do not need to know specialized languages to initiate a search. Furthermore, the matching process does not depend on a match between the cataloguer description and the user description. The system is available to all collections in the NSDL without imposing new standards or protocols. This offers the promise for the NSDL to support images that have not yet been cataloged or have incomplete metadata, without the image collection provider or aggregator having to incur additional cataloging overhead.
https://www.fastlane.nsf.gov/servlet/showaward?award=0121596
NSF Org CCR
Latest Amendment Date June 21, 2002
Award Number 0209006
Award Instrument Continuing grant
Program Manager John Cozzens
CCR DIV OF COMPUTER-COMMUNICATIONS RESEARCH
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date June 1, 2002
Expires May 31, 2005 (Estimated)
Expected Total Amount $300000 (Estimated)
Investigator Thrasyvoulos N. Pappas pappas@ece.nwu.edu (Principal Investigator current)
Sponsor Northwestern University
633 Clark Street
Evanston, IL 602081110 847/491-3003
NSF Program 4720 SIGNAL PROCESSING SYS PROGRAM
Field Application
Program Reference Code 9216,HPCC,
Abstract
The rapid accumulation of large collections of digital images has created the need for efficient and intelligent schemes for image retrieval. Since humans are the ultimate users of most retrieval systems, it is important to organize the contents semantically, according to meaningful categories. This requires an understandingof the important semantic categories that humans use for image classification, and the extraction of meaningful image features that can discriminate between these categories. Recent research efforts have addressed the first problem, but the second remains quite elusive. This research effort is aimed at addressing this second problem, that is, the extraction of low-level image features that can be correlated with high-level semantics and used to capture the semantic meaning of an image.
The key to this research is the development of a new methodology for segmenting images, based on perceptual models and principles about the processing of texture and color information. This involves the identification of semantically important, spatially adaptive, low-level color and texture features that can be combined algorithmically to obtain image segmentations that convey semantic information. The same perceptual models and principles can be used to relate the features of the segmented regions (color and texture features, as well as segment location, size, and boundary shape) to semantic concepts that can be used for content-based image retrieval. An integral part of this research is the design and execution of subjective experiments in order to obtain some key parameters for the color and texture features, as well as for linking low-level image features to image semantics.
https://www.fastlane.nsf.gov/servlet/showaward?award=0209006
NSF Org IIS
Latest Amendment Date August 20, 1998
Award Number 9813584
Award Instrument Standard Grant
Program Manager Maria Zemankova
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date August 15, 1998
Expires January 31, 2000 (Estimated)
Expected Total Amount $99737 (Estimated)
Investigator Padhraic Smyth smyth@ics.uci.edu (Principal Investigator current)
Dennis Kibler (Co-Principal Investigator current)
Michael J. Pazzani (Co-Principal Investigator current)
Sponsor U of Cal Irvine
160 Administration Building
Irvine, CA 926971875 949/824-7106
NSF Program 6855 INFORMATION & DATA MANAGEMENT
Field Application 0104000 Information Systems
Program Reference Code 9139,9237,HPCC,
Abstract
Over the last two
decades there has been an explosive growth in online data storage of
various forms. These large datasets have motivated the rapid development
of data mining methods. However, until now, there has been a lack of
an online repository of large data sets for researchers to evaluate
and compare their methods. In this project, an online repository of
large and difficult data sets are being gathered that are representative
of the diverse character of many important scientific and business
domains. This repository includes high-dimensional data sets as well
as data sets of different data types (time series, spatial data, transaction
data, and so forth). The primary role of the repository is that of
a benchmark testbed to enable researchers in data mining (including
computer scientists, statisticians, engineers, and mathematicians)
to scale existing and future data analysis algorithms to very large
data sets. Each data set in the respository contains online documentation,
metadata, and links to relevant background domain information such
as prior published work. Availability of a standard set of large benchmark
data sets will directly stimulate and foster systematic progress in
data mining related research, similar to the affect that the UCI Machine
Learning Data Repository has had on machine learning research. This
repository will play a substantial role in brokering the gap between
research-oriented algorithm development in the laboratory and the real-world
practicalities and challenges of very large data sets. http://www.ics.uci.edu/~mlearn/MLRepository.html
https://www.fastlane.nsf.gov/servlet/showaward?award=9813584
NSF Org IIS
Latest Amendment Date July 23, 2002
Award Number 9905603
Award Instrument Continuing grant
Program Manager Stephen Griffin
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date August 15, 2000
Expires July 31, 2003 (Estimated)
Expected Total Amount $400000 (Estimated)
Investigator Aidong Zhang azhang@cse.buffalo.edu (Principal Investigator current)
David M. Mark (Co-Principal Investigator current)
Raj Acharya (Co-Principal Investigator current)
Sponsor SUNY Buffalo
501 Capen Hall
Buffalo, NY 14260 716/645-2977
NSF Program 6857 SPECIAL PROJECTS (IIS)
Field Application 0104000 Information Systems
Program Reference Code 1364,9216,HPCC,
Abstract
The objective of the proposed research is to investigate novel approaches to supporting effective and efficient access to various geographic image databases over the Internet leading to design of distributed geographic image retrieval systems. The research involves three research teams from: SUNY Buffalo, National Center for Science Information Systems (NACSIS) in Japan, and the University of Nantes in France. (IRESTE) The technical challenges are the creation of a new meta-level system for geographic image databases. To achieve this, research issues concerned with representation models for geographic data, relationships between metadata and resource discovery and efficient query processing in a distributed environment must be addressed. The problems addressed are critical to retrieval on large volume, multidimensional, distributed data over the Internet. Results can be broadly applied to numerous domain and interdisciplinary research areas.
https://www.fastlane.nsf.gov/servlet/showaward?award=9905603
NSF Award Abstract - #9619117 AWSFL008-DS3
NSF Org IIS
Latest Amendment Date February 23, 2001
Award Number 9619117
Award Instrument Continuing grant
Program Manager Michael E. Lesk
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date March 1, 1997
Expires February 28, 2002 (Estimated)
Expected Total Amount $769111 (Estimated)
Investigator James M. Allan allan@cs.umass.edu (Principal Investigator current)
Raghavan Manmatha (Co-Principal Investigator current)
Allen R. Hanson (Co-Principal Investigator former)
Sponsor U of Massachusetts Amherst
408 Goodell Building
Amherst, MA 010033285 413/545-0698
NSF Program 6845 HUMAN COMPUTER INTER PROGRAM
Field Application 0104000 Information Systems
0116000 Human Subjects
Program Reference Code 9139,HPCC,
Abstract
*** In the rapidly emerging area of multimedia information systems, effective indexing and retrieval techniques are critically important. In this project, the Center for Intelligent Information Retrieval (CIIR) will study algorithms and mechanisms that facilitate the.process of indexing and retrieving images, video, and associated text. Specifically, the CIIR will: 1) Develop techniques for browsing and retrieving images and videos by content. Image indexing is an emerging technology which is poorly understood. New and powerful techniques will be developed for indexing images and videos using visual attributes such as color, texture and 'appearance'; 2) Develop techniques for indexing images and videos using text present in the images. Methods of assisting with manual annotation of images will also be studied. The text indexing methods will be combined using the INQUERY probabilistic retrieval engine; 3) Develop and evaluate interactive retrieval and browsing interfaces for image, video, and text retrieval. These interfaces will be based on novel visualization and user interface work done at CIIR. The CIIR will perform user studies to evaluate the appeal and effectiveness of the interfaces. In summary, the project will integrate innovative image matching, sophisticated text retrieval and new query formulation and visualization methods to create a powerful platform for indexing and retrieving images, video, and text.***
https://www.fastlane.nsf.gov/servlet/showaward?award=9619117
NSF Award Abstract - #9908441 AWSFL008-DS3
NSF Org IIS
Latest Amendment Date August 21, 2001
Award Number 9908441
Award Instrument Continuing grant
Program Manager Bhavani Thuraisingham
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date October 1, 2000
Expires October 31, 2003 (Estimated)
Expected Total Amount $211000 (Estimated)
Investigator Yuan-Fang Wang yfwang@cs.ucsb.edu (Principal Investigator current)
Sponsor U of Cal Santa Barbara
c/o Office of Research
Santa Barbara, CA 93106 805/893-4188
NSF Program 6855 INFORMATION & DATA MANAGEMENT
Field Application 0104000 Information Systems
Program Reference Code 9216,HPCC,
Abstract
Many image databases employ image features that describe the aggregate shape and color properties of a class of objects, and hence, are only capable of "inter-classes" discrimination (e.g., airplanes vs. butterflies). These features are sensitive to incidental changes in camera viewpoint and scene illumination, which severely constrain the appearance of query objects. The goal of this project is to design a set of new image-derived features which utilize detailed local image analysis to enable discriminating objects of very similar appearance within the same class (e.g., "intra-class" discrimination of different species of butterflies in a database of butterflies). These new features capture the essential traits of imaged objects in a way that is insensitive to incidental changes in both global environmental factors, such as viewpoint and illumination, and local configuration, such as shape deformation and articulated motion. The project addresses the efficiency and utility issues in deploying these invariant features in large, real-world image databases. In particular, flexible class templates are constructed for automating image segmentation and cataloging of database objects. Invariant image features are organized in a hierarchical manner, using both image-derived and domain-specific information, for efficient pruning of unlikely matches. A representation strategy, which combines global structure models with local invariant features, is used to achieve recognition insensitive to incidental articulated motion and deformation. As a result, the new image features are highly discriminative yet insensitive to incidental changes in shape, viewpoint, and illumination. In particular, they enable invariant, intra-class image retrieval, i.e., highly precise recognition and retrieval of images in large databases of similar objects (e.g., retrieving images of Old World Swallowtails from a database of butterflies) with few constraints on the appearance of query images. The results of the project will be disseminated by scientific papers, software that can be downloaded, and prototype image databases that can be queried on the Web. http://www.cs.ucsb.edu/~yfwang
https://www.fastlane.nsf.gov/servlet/showaward?award=9908441
[NSF Logo] NSF Award Abstract - #0136348 AWSFL008-DS3
NSF Org IIS
Latest Amendment Date November 19, 2001
Award Number 0136348
Award Instrument Standard Grant
Program Manager Bhavani Thuraisingham
IIS DIV OF INFORMATION & INTELLIGENT SYSTEMS
CSE DIRECT FOR COMPUTER & INFO SCIE & ENGINR
Start Date November 1, 2001
Expires September 30, 2002 (Estimated)
Expected Total Amount $50000 (Estimated)
Investigator Jing Peng jp@eecs.tulane.edu (Principal Investigator current)
Douglas R. Heisterkamp (Co-Principal Investigator current)
Sponsor Tulane University
6823 St. Charles Avenue
New Orleans, LA 701185665 504/865-4000
NSF Program 6855 INFORMATION & DATA MANAGEMENT
Field Application 0104000 Information Systems
Program Reference Code 9218,9237,HPCC,
Abstract
The objective of this project is to design efficient indexing strategies that support flexible retrieval metric through relevance feedback learning. However, trying to satisfy both goals (efficiency and flexibility) at the same time leads to a conflict. A novel approach is explored to capture the inherent interplay between flexible metrics and indexing that has the potential to resolve the conflict. It is hypothesized that the interplay can be exploited to create effective content-based retrieval systems that meet performance and computational challenges encountered in practical image database applications. This exploratory project seeks to establish the proof of concept of an approach that trades off accuracy for efficiency, and that can avoid exhaustive search in large-scale image databases. The methods to be explored are based on bump-hunting in high-dimensional data for inducing a set of (possibly overlapping) boxes that capture the local data distributions. The induced boxes effectively cover the feature space, thereby providing an index to the image database. The flexibility and efficiency of the novel indexing technique will be tested in heterogeneous image databases that support a variety of query types, ranging from query-by-image to query-by-region. If successful, the results of this project will enable the use of flexible metric learning in large scale image databases, which will have a significant impact in content-based image retrieval in broad areas such as health-care, scientific images, education, or art.
https://www.fastlane.nsf.gov/servlet/showaward?award=0136348
http://www.mellon.org
"One of the most serious bottlenecks in digitizing collections is in making them easy to search. The strategy proposed has the potential to provide rich, subject-oriented indexing for large image collections that would otherwise be prohibitively expensive to describe and index using manual techniques. A further advantage of the approach is that the descriptive metadata generated may be derived from authoritative scholarship in a way not normally feasible in standard cataloging practice. The project goal of CLiMB is to develop and test automatic approaches to the creation of descriptive metadata for improving access to digital library special collections. http://www.columbia.edu/cu/lweb/news/files/2002-04-26.climb.html [Aug 15, 2002.]
The Andrew W. Mellon Foundation has awarded $755,000 to the Penn State University Libraries to support an extensive study of digital image delivery.
Leading the study, the Libraries will partner with other Penn State units, including the Center for Education Technology Services, the Center for Quality and Planning, Library Computing Services, and the School of Information Sciences and Technology. The Visual Image User Study (VIUS, pronounced views) will examine the use of digital pictures at Penn State in the disciplines of the arts, environmental studies, and the humanities. The project includes the development and testing of a prototype system for image delivery.
Phase one of the project will employ a variety of needs assessment methods and information retrieval studies to analyze current and future needs of teachers, learners, and archival managers. The second phase, based on the results of phase one, will create the design and content of the prototype system.
Reviewers of the plan praise its client-centered approach, interdisciplinary scope, institutional teamwork, and potential to contribute useful data to an important aspect of digital library development. The project began in May 2001, and activities will continue for twenty-six months.
http://www.libraries.psu.edu/crsweb/vius/section1.html#1-1
Grant number: RE/103
Grantee: University of Northumbria at Newcastle
Duration of grant: 1 January 2000 - 31 December 2001
Amount of award: £69,045
Contact: Mrs Margaret Graham
Address: Institute for Image Data Research, University of Northumbria at Newcastle, Newcastle upon Tyne NE1 8ST
Tel: 0191-227 4646
Fax: 0191-227 4637
Email: margaret.graham@unn.ac.uk
WWW: http://www.unn.ac.uk/iidr/staff/margaret.html
In recent years there has been enormous growth in interest in the potential of digital images, especially as technological advances make it now possible to store and access large quantities of data relatively cheaply. Coupled to this has been the rapid growth if imaging on the World Wide Web (WWW). Many organisations are taking advantage of various funding opportunities to digitise parts or all of their collections. But the process of digitisation does not in itself make image collections easier to manage or to use. There are several computerised image data management systems on the market which help to organise and view the digital images. Some forms of cataloguing and indexing are still necessary, since browsing is not an option except with small collections. These problems have stimulated research into content-based image retrieval (CBIR), the selection of images from a collection via features automatically extracted from the images themselves. Current CBIR systems typically provide image retrieval by low-level attributes such as colour, texture or shape, and few attempt higher levels of retrieval, such as by semantic content (e.g. the presence in an image of specific types of object, or the depiction of a particular type of event). There has been little systematic evaluation of CBIR system effectiveness on a large scale. Key questions, such as whether CBIR techniques can bring about worthwhile improvements in performance with real-life image retrieval systems, or where and how such techniques can most profitably be used, thus remain unanswered.
The aim of this research is to evaluate CBIR systems in an operational setting. The project will install appropriate commercially-available CBIR software as additional functionality to the image data management systems currently in use in three pictorial libraries in the public and private sectors. An initial user study will be conducted to obtain the first impressions of the CBIR functionality by both staff and other end-users. Following a six months "gestation" period, detailed user evaluations will then be conducted. The outcomes will be three case studies, demonstrating CBIR in practice, and a body of evidence regarding the usefulness and effectiveness of CBIR as a searching tool in the context of the individual organisations. Specific research questions which will be investigated include:
* How successful are CBIR systems in meeting user needs?
* What are the effects of CBIR provision on user search behaviour?
á To what extent can the use of CBIR systems be justified in different contexts?
Grant number: RE/104
Grantee: University of Northumbria at Newcastle
Duration of grant: 1 June 2000 - 31 May 2001
Amount of award: £43,136
Contact: Mrs Margaret Graham
Address: Institute for Image Data Research, University of Northumbria at Newcastle, Newcastle upon Tyne NE1 8ST
Tel: 0191-227 4646
Fax: 0191-227 4637
Email: margaret.graham@unn.ac.uk
WWW: http://www.unn.ac.uk/iidr/staff/margaret.html
The chief aim of the project is to develop a user-oriented evaluation framework for electronic image retrieval in the workplace. This framework would span the overall process of image retrieval system design, development and implementation - something which includes an ongoing process of evaluation throughout all these stages. It is primarily intended as a practical tool to assist and guide those responsible for conducting evaluations of image retrieval systems from the user's perspective, though there are other potential benefits for researchers, professional groups and organisations as well as users themselves.
This endeavour will utilise and build upon the results of and groundwork laid by the first phase of the VISOR research programme (Information seeking behaviour in image retrieval, LIC project LIC/RE/031). Thus the models of user search behaviour focus on users of images in context and this will lie at the heart of the development of an evaluation tool.
A central focus of the evaluation framework will be the consideration of how well the system supports the user in their work, particularly in terms of functionality, interface/access and decision support. It must be emphasised that a search system and the search process together form the entire information searching and retrieval process and it is not acceptable to evaluate the system alone. The image retrieval systems will therefore be evaluated in context and from a user-oriented perspective.
The evaluation tool will be developed by performing user-centred system evaluations with existing image retrieval systems currently in use in various organisations. The initial evaluation procedure will be informed by the models of image seeking behaviour developed during VISOR I, by a comprehensive review of the relevant literature and by input from experts in the field of evaluation. The results of the practical work conducted during a pilot phase (i.e. initial user-centred system evaluations) will be used to reformulate the evaluation procedure accordingly. The resulting procedure will then be used in a second organisation to verify the approach and make further modifications as necessary. Finally, an evaluation framework to guide the development of electronic image retrieval systems will be formulated, incorporating the experience gained throughout VISOR II. Throughout the project, feedback sessions will be held with both participant organisations and experts in the field.
Grant number: RE/031
Grantee: University of Northumbria at Newcastle
Duration of grant: 5 May 1998 to 4 May 2000
Amount of award: £60,560
Contact: Margaret E Graham
Address: Institute for Image Data Research, University of Northumbria at Newcastle, Ellison Building, Newcastle upon Tyne NE1 8ST
Tel: 0191 227 4646
Fax: 0191 227 4637
Email: margaret.graham@unn.ac.uk
WWW: http://www.unn.ac.uk/iidr/staff/margaret.html
The aim of this project is to investigate the information-seeking behaviour of image users in specific disciplines or domains, in order to increase our understanding of how and why people seek for and use visual information. It is intended that the results should influence the design of future image retrieval systems so that digital collections of images can be exploited to their full potential.
There is a wealth of research into users' information-seeking behaviour, but almost exclusively in the area of textual information. There is little understanding of how people seek for or use images. Recently, there has been research and development activity in so-called content-based image retrieval (CBIR) where images are selected from a collection via features automatically extracted from the images themselves. Current CBIR systems typically provide image retrieval by low-level attributes such as colour, texture or shape, and few attempt higher levels of retrieval, such as by semantic content (e.g. the presence in an image of specific types of object, or the depiction of a particular type of event). How important is colour, texture or shape to the retrieval of visual information? When, and by whom, are higher levels of retrieval needed? What searching strategies are appropriate for image retrieval? How important is browsing to the retrieval of visual information?
The research will be essentially qualitative in nature. The project will study the behaviour of real users of visual information in real situations. It will build up a series of case studies based on representative organisations from within certain subject domains, such as Architecture, Medicine, Journalism and Art History, and will study subjects in these organisations who use images/visual information as part of their normal work. Through the analysis of relevant documentation and current systems and the recordings of interviews and observations, the project will categorise the uses of images in different domains and thus develop models of the information seeking behaviour of image users.
Grant number: RE/029
Grantee: University of Brighton
Duration of grant: 20th March 1998 to 30th June 2000
Amount of award: £54,429
Contact: Dr P G B Enser
Address: School of Information Management, University of Brighton, Watts Building, Moulsecoomb, Brighton BN2 4GJ
Tel: 01273 643505
Fax: 01273 642405
Email: P.G.B.Enser@brighton.ac.uk
WWW: http://www.it.bton.ac.uk/im/
This project aims to contribute to the development of effective retrieval mechanisms for visual image material. It will seek to develop deeper, evaluative insights into the emerging and future practice of image retrieval, looking beyond the technical characteristics and embracing the economic, political and social factors which are shaping the design and implementation of electronic visual information retrieval systems among the library/information service, gallery and museum communities.
Central to this endeavour is a recognition of the need to find effective indexing and retrieval strategies amid the proliferation of techniques and tools for the storage and manipulation of digitised images. The imperative to incorporate, to a far greater extent, the real needs of end-users and to move to a standardised platform, or approach, for visual information retrieval developments are clearly indicated and will also figure prominently in the project.
The objective is to produce a state-of-the-art review of the planning, organisation and implementation of electronic visual information retrieval systems, based on a study of a representative selection of organisations and their client interaction. The study will involve the identification of key participants and decision processes, the examination of individual system solutions and the local and sectoral contexts within which these solutions emerge.
By providing a richer picture of the diverse elements which form the inputs and outcomes of current initiatives in visual information retrieval systems for existing customer bases, the study seeks to engage with the economic, political and social factors which will condition the further development of such systems. From this examination, a guide to good practice will be generated, of relevance to those many organisations which are planning to update to electronic visual information retrieval mechanisms in the next few years.
Grant number: RE/013
Grantee: University of Northumbria at Newcastle
Duration of grant: 25th July 1997 to 24th July 2000
Amount of award: £99,250
Contact: Dr J P Eakins
Address: Department of Computing, University of Northumbria at Newcastle, Ellison Place, Newcastle upon Tyne NE1 8ST.
Tel: 0191 227 4539
Fax: 0191-227 3662
Email: john.eakins@unn.ac.uk
WWW: http://computing.unn.ac.uk/~johne/IIDR.html
Effective shape retrieval is still proving an elusive goal; no current system appears able to mirror human judgements of shape similarity with sufficient reliability. One possible reason for this is that virtually all current shape retrieval systems derive indexing features purely from explicit representations of image elements such as region boundaries. The ARTISAN system, developed at the University of Northumbria at Newcastle under a previoue British Library Research and Innovation Centre grant, is an exception to this, in that it derives indexing features from virtual boundaries generated by grouping image regions into families using principles derived from Gestalt psychology. Cognitive psychologists have accumulated a considerable body of evidence about the way the human eye and brain perceive and interpret an image. ARTISAN appears to yield promising retrieval results, even though it makes only limited use of such findings.
The aim of the present investigation is therefore to test the extent to which a systematic application of findings from human visual cognition studies can improve retrieval performance. The investigation will review the current state of the art in human visual perception in some depth, identifying hypotheses and experimental findings of potential relevance to shape retrieval. A model of the shape retrieval process will be constructed, and used to guide the development of an improved shape retrieval prototype, capable of representing and matching shapes more effectively than any current system. Specific aspects on which the model is expected to concentrate include:
* replacing ill-defined image elements with "idealized" equivalents
* building up multi-view representations of "ambiguous" shapes, allowing alternative interpretations of their content
* investigating the utility of fractal shape representation techniques
* measuring the salience of different types of image feature to a given query
The effectiveness of this model of the shape retrieval process will be tested by measuring its effect on retrieval performance over the same set of test queries and images as that supplied by the Patent Office for the evaluation of the earlier ARTISAN system. It is expected that this study will lead to a significant improvement in the understanding of the shape retrieval process, which should enable the development of more powerful image retrieval systems in the future.
http://www.europa.eu.int/comm/dgs/information_society/index_en.htm
Association of Computing Machinery (ACM) SIG Multimedia
http://www.acm.org/sigmm/
http://www.nii.ac.jp/mir2000/;
http://www.info.uqam.ca/~misrm/);
Association of Computing Machinery (ACM) Special interest group on Information Retrieval
http://www.acm.org/sigir/
Association of Computer Machinery / Institute for Electronic and Electrical Engineering Joint Conference on Digital Libraries
http://www.jcdl.org/
American Society for Information Science and Technology
http://www.asis.org
http://www.civr.org
Challenge of Image Retrieval, 1998
Challenge of Image Retrieval, 1999
Challenge of Image Retrieval, 2000
Challenge of Image and Video Retrieval, 2002 http://www.civr2002.org/ Proceedings: Springer Lecture Notes in Computer Science series. Vol 2383
Challenge of Image and Video Retrieval, 2003
IEEE Computer Society Conference on Computer Vision and Pattern Recognition
http://www.cs.toronto.edu/cvpr2003/
http://www.ercim.org/publication/ws-proceedings/DELOS4/index.html
DELOS Network of Excellence for Digital Libraries
European Conference on Digital Libraries
http://www.ecdl2003.org/
The Society for Imaging Science and Technology
http://www.imaging.org
International Conference on Information Visualisation, IV 2000, 19-21 July 2000, London, England, UK. IEEE Computer Society, online publications: http://computer.org/proceedings/iv/0743/0743toc.htm
ACM/IEEE Joint Conference on Digital Libraries
http://www.acm.org/jcdl/
SPIE / IS&T / Internet Imaging
http://electronicimaging.org/Program/03/
The International Society for Optical Engineering.
All over in information science, digital libraries, museum studies/informatics, computer science; the field is not coherent.
Archives & Museum Informatics: annual, since 1997
http://www.archimuse.com/mw.html
Archives & Museum Informatics http://www.archimuse.com/ichim.html
http:://www.drh.org.uk/
regularly throughout Europe since 1995
Electronic Imaging in the Visual Arts Conference Series
http://www.vasari.co.uk/eva/index.htm
Library and Information Technology Association, a Division of the American Library Association
http://www.lita.org/
http://www.specbench.org/ "MISSION: To establish, maintain, and endorse a standardized set of relevant benchmarks and metrics for performance evaluation of modern computer systems
Frequently Asked Questions at http://www.specbench.org/spec/faq/
http://www.isobenchmarking.com
http://www.ismbc.org
http://www.isobenchmarking.com/
http://www.kmba.org
http:://www.eembc.org/about.asp
Edward Jones, The Importance of Benchmarking (Connecticut Department of Social Services. Biometric ID Project.) First published Biometrics In Human Services User Group, Newsletter, May 1997 . Available: http://www.dss.state.ct.us/digital/bench.htm {August 16, 2002]
The following is only a selection of the great many image databases available on the Web (in whole or in part). Descriptions are abstracted from the sites referenced and minimally edited for consistency.
http://search.amico.org
contains over 100,000 works of art from the collections of 30+ museums in the United States, Canada and the UK which are members of the non-profit Art Museum image Consortium. It is available under subscription for use by educators worldwide and datasets from it could be made available for retrieval researcher.
ARTStor propose 250,000 images in an initial release..
However public information about the initiative is slight. No public
URL, no release date
http://www.lib.colum.edu/commwais.html
contains information on over 2,300 examples of advertising and related art. This index to selected images used in the study of advertising and commercial art at Columbia College Library can be searched in several ways.
http://www.lib.colum.edu/costwais.html
contains information on over 9,000 examples of costume and fashion design artwork and imagery. This index to selected images used in the study of fashion and costume at Columbia College Library can be searched in several ways.
http://www.nga.gov/collection/srchart.html
15,000+ including many details of works
http://www.nationalgallery.org.uk/collection/default_online.htm
http://ngda.cooper.edu/ngdasite/info.html
An electronic archive that preserves and disseminates material related to twentieth century graphic arts and design. The system enables interactive and interdisciplinary analysis among faculty, students and the design community worldwide. Educators with password registration and cataloging level permissions may post notes, analysis and images from off-site locations. The Cooper Union's 20,000 piece Graphic Design Slide Library is the database's original source. It includes donations from design organizations, private collectors, and educators, and features designs by noted American graphic artists. In the Fall 1998 the project acquired the 6,000 digital image Danziger Collection from the Art Center College of Design.
http://www.npg.org.uk/live/collect.asp
over 12,500 illustrations of portraits
http://www.tate.org.uk/collections/default.jsp
Permanent collection 8000+ images including several thousand works by J.M.W. Turner
http://www.thinker.org
Permanent collection, over 82,000 images of works of 12,000+ artists
http://vads.ahds.ac.uk
American Memory is a gateway to rich primary source materials relating to the history and culture of the United States from the collections of the Library of Congress. The site offers more than 7 million digital items from over 100 historical collections.
http://www.chin.gc.ca/English/Artefacts_Canada/index.html
200,000 images form museums across Canada, divided into Arts and Humanities, Natural Sciences, and Archaeological Sites
http://eikon.divinity.yale.edu/
is a faculty-library initiative at Yale Divinity School that provides digital resources for teaching and research in the field of Biblical studies. Images in the EIKON database are a subset of the Yale Divinity School Digital Library. Some images in the EIKON database are restricted to Yale use, due to copyright agreements. The EIKON project was initiated during the 1998 -1999 academic year with a grant from the Wabash Center for Teaching and Learning in Theology and Religion.
http://www.washington.edu/ark2/
A collection of digitized images of buildings and cities drawn from across time and throughout the world, available to students, researchers and educators on the web. Begun in 1995, the Database was conceived as a multi-disciplinary resource for students, faculty, and others in the academic community. It has grown steadily since then, with contributions from a wide range of scholars, and now contains over 5000 images ranging from New York to Central Asia, from African villages, to the Parc de la Villette, and conceptual sketches and models of Frank Gehry's Experience Music Project. These have all been scanned from original slides or drawn from documents in the public domain. They are freely available to anyone with access to the Web for use in the classroom, student study, or for individual research purposes.
http://collage.nhil.com/
An image database containing 20,000 works from the Guildhall Library and Guildhall Art Gallery London. By the Corporation of London and Ibase Image Systems Its categories include:
Abstract Ideas; Archaeology & Architecture; History; Leisure; Military & War; Natural World; Politics; Religion & Belief; Society; Trade & Industry. One may search by People, Places, Artists, Engravers, and Publishers.
A growing collection of 140,000+ images from the Research Libraries Group which contains collections from research libraries documenting the entire range of human activity.
The Visual Information Access (VIA) system is a union catalog of visual resources at Harvard. It includes information about slides, photographs, objects and artifacts in the university's libraries, museums and archives. This system represents the first phase of an ongoing effort and additional information will be added on a regular basis. Check the repository's web sites for more information about access policies and coverage for their visual collections.
http://via.harvard.edu:748/html/VIA.html
http://ind156b.wi.leidenuniv.nl:2000/
is a database of 19389 Dutch Carte de Visite Studio Portraits taken from 1860-1914.
Growing digital library of ancient Greek and Rome,
expanding to include other areas
http://www.perseus.tufts.edu
http://www.scran.ac.uk/homepage/
is a history and culture site with more than one million records, include 160,000+ images, from museums, galleries, archives, the media and contemporary and performing arts in Scotland. SCRAN gives worldwide access to multimedia educational resources on international culture and world heritage.
"thousands" of images from collections across Canada.
includes more than 9000 press release photos spans the manned space program, from Mercury to the STS-79 Shuttle mission. The collection includes a full text search. A list of frequently asked questions is now available. NASA generally does not assert copyright for these photos. Some conditions.
hosted by JSC's Office of Earth Sciences and Image Analysis, provides a standard way of describing the contents of the database that would be comparable between different applications and over the coming years. Statistics about specific subsets (e.g., number of photos of Namibia) can be compiled from the Search Page (http://eol.jsc.nasa.gov/sseop/sql.htm) [i]
http://imagelib.ncsa.uiuc.edu/imagelib.html
collects astronomical, research-quality images and makes them available to the astronomical community and the general public. Patrons access the Library through the World Wide Web to search for and browse images. Once images are located in the Library, users may download them to their local machines in FITS format for further analysis.The Library is being developed and maintained by the Radio Astronomy Imaging Group at the National Center for Supercomputing Application (NCSA) on the campus of the University of Illinois at Urbana-Champaign (UIUC).
http://www.seds.org/messier/
During the years from 1758 to 1782 Charles Messier, a French astronomer (1730 - 1817), compiled a list of approximately 100 diffuse objects that were difficult to distinguish from comets through the telescopes of the day. Discovering comets was the way to make a name for yourself in astronomy in the 18th century -- Messier's aim was to catalog the objects that were often mistaken for comets. Fortunately for us, the Messier Catalog became well known for a much higher purpose, as a collection of the most beautiful objects in the sky including nebulae, star clusters, and galaxies. It was one of the first major milestones in the history of the discovery of Deep Sky objects, as it was the first more comprehensive and more reliable list: Only four objects were initially missing because of data reduction errors, which could be figured out later though. Today's versions of the catalog usually include also later additions of objects observed by Charles Messier and his collegial friend, Pierre Méchain, but not included in his original list. The study of these objects by astronomers has led, and continues to lead, to important, incredible discoveries.For each object, an image is presented together with a short description. In addition to the images, we have also included some data on these objects such as celestial position (right ascension in hours and minutes [h:m], declination in degrees and minutes [deg:m], both for the epoch J2000.0), apparent visual brightness in magnitudes [mag], apparent (angular) diameter in arc minutes [arc min], and approximate distance in thousands of light-years (kilo-light years [kly] please note that the decimal point in the distance does not represent true accuracy; see explanation of the terms used here). Also, we have constellation images which show Messier and NGC (New General Catalog) objects down to 12th magnitude.
http://edcwww.cr.usgs.gov/bin/html_web_store.cgi
contains some of the millions of digital images in the EROS Center including aerial photographs, mainly for mapping, and various kinds of satellite images for scientific study. Among all these pictures, naturally, are some that are beautiful, some that record events of historic significance, and some that stir the imagination for other reasons. Categories for searching include: states, cities of the us, napp major metro, cities of the world, space, countries and continents, weather, natural hazards, and natural features.
http://www.uq.edu.au/nanoworld/images_1.html
at The University of Queensland ,is a database of images from electron microscopes. The copying of images, and their subsequent FREE use, is permitted with the following conditions:
* that the source of the image(s) is clearly indicated;
* and that their use is recorded by completing our Registration Form.
* if used on a web page then a link to the Nanoworld must be provided.
http://elib.cs.berkeley.edu/photos/
CalPhotos is a collection of 42,680 images of plants, animals, fossils, people, and landscapes managed by the Digital Library Project
University of California, Berkeley. A variety of organizations and individuals have contributed photographs to CalPhotos.
http://pbio07.uni-muenster.de/html/About.html
Currently consists of 3700+ images on Drosophila development and genetics, especially on expression patterns of genes (enhancer trap lines, cloned genes).The concept of FlyView includes compatibility to FlyBase, the main Drosophila database. Its aim is to establish the possibilty to compare images on the computer screen and to search for special patterns at different developmental stages. Therefore, all images are accompanied by text descriptions that can be used for searching. The success of this database exclusively depends on the activity of the Drosophila community. All Drosophila workers are asked to contribute to this database by submitting images and accompanying text.
is a web-driven database for images. The project was initiated by Hans van der Voort from SVI, a company producing the Huygens software package, used in deconvolution of microscopy images. (SIDB uses a 'light' version of huygens to interpret 3-D image data). Started as a project by and for undergraduate computer sciences students from the University of Utrecht, it matured at the Institute for Molecular Plant Sciences from the University of Leiden, the Netherlands, where SIDB is now used by many to archive images and other file types. SIDB archives 2-D, 3-D images. Image files are stored unchanged in a central directory (archive). Users of the system are subdivided in groups, and whoever owns an image (by uploading it) can determine who else on the system is allowed to view and use the image. Files can be uploaded through HTML, or using a mounted (smb, NFS, etc..) drive. Entering meta-data is facilitated by user-definable templates. The meta-data fields currently in use have been designed for images derived by (confocal) microscopy. When combined with cheap, large hard drives and a fail-save backup mechanism, SIDB provides a perfect means to archive images within the setting of small to medium-sized research groups. However, it might be of use wherever people collaborate on images.
http://www.imb-jena.de/IMAGE.html
is aimed at a better dissemination of information on three-dimensional biopolymer structures with an emphasis on visualization and analysis. It provides access to all structure entries deposited at the Protein Data Bank (PDB) or at the Nucleic Acid Database (NDB). In addition, basic information on the architecture of biopolymer structures is available. The IMB Jena Image Library intends to fulfill both scientific and educational needs.
http://www.nhm.ukans.edu/ashe/aleo/splash2.html
includes photographs, scanning electron micrographs, half-tone drawings and line drawings in this database are being accumulated as potential illustrations for a guide to the aleocharine genera of North America and Mexico, with additional images of taxa from other geographical regions for comparison. Seevers (1978) noted that it is probably impossible to identify most aleocharine genera without a substantial reference collection. It is our ultimate goal to provide habitus illustrations, photographs and line drawings of structural features of all genera. Hopefully, these will serve as the "reference collection" needed to accurately identify aleocharine genera (when used with appropriate keys). Development of this image database is a on-going project, and new images will be added, and images of poor quality replaced, on a regular basis. All images in this database are made available thanks to NSF PEET grant DEB-9521755 to James S. Ashe.
http://www.unibas.ch/botimage/
contasins 4,341 specimens that can be searched by Species, Families, Genera, and Orders including
• Basel (2'077)
• Samos (1'107)
• Tenerife (693)
• Pollinators (287)
• Vegetation (315)
• Woody plants (1'274)
http://www.csdl.tamu.edu/FLORA/gallery.htm
contains thousands of images and metadata regarding vascular plants organized by genus.
The Texas A&M Bioinformatics Working Group includes campus faculty, staff, and students with research and educational interests in the expression of biodiversity data using new information technologies. These pages provide an overview of the Working Group and its activities
(http://www.hcs.ohio-state.edu/plants.html)
is an indexed system of teaching resources for the discipline of Horticulture and Crop Science. It was developed to complement the other electronic resources developed at Ohio State University, collectively called HORTICULTURE and CROP SCIENCE in Virtual Perspective.
http://www.mis.atr.co.jp/~mlyons/jaffe.html
contains 213 images of 7 facial expressions (6 basic facial expressions + 1 neutral) posed by 10 Japanese female models. Each image has been rated on 6 emotion adjectives by 60 Japanese subjects. The database was planned and assembled by Miyuki Kamachi, Michael Lyons, and Jiro Gyoba. We thank Reiko Kubota for her help as a research assistant. The photos were taken at the Psychology Department in Kyushu University. [ii]
http://rvl1.ecn.purdue.edu/~aleix/aleix_face_DB.html
was created by Aleix Martinez and Robert Benavente in the Computer Vision Center (CVC) at the U.A.B. It contains over 4,000 color images corresponding to 126 people's faces (70 men and 56 women). Images feature frontal view faces with different facial expressions, illumination conditions, and occlusions (sun glasses and scarf). The pictures were taken at the CVC under strictly controlled conditions. No restrictions on wear (clothes, glasses, etc.), make-up, hair style, etc. were imposed to participants. Each person participated in two sessions, separated by two weeks (14 days) time. The same pictures were taken in both sessions. This face database is publicly available and can be obtained from this web-site. It is totally free for academics whishing to test their systems. Commercial distribution or any act related to commercial use of this database is strictly prohibited.
http://marathon.csee.usf.edu/Mammography/Database.html
is a resource for use by the mammographic image analysis research community. Primary support for this project was a grant from the Breast Cancer Research Program of the U.S. Army Medical Research and Materiel Command. The DDSM project is a collaborative effort involving co-p.i.s at the Massachusetts General Hospital (D. Kopans, R. Moore), the University of South Florida (K. Bowyer), and Sandia National Laboratories (P. Kegelmeyer). Additional cases from Washington University School of Medicine were provided by Peter E. Shile, MD, Assistant Professor of Radiology and Internal Medicine. Additional collaborating institutions include Wake Forest University School of Medicine (Departments of Medical Engineering and Radiology), Sacred Heart Hospital and ISMD, Incorporated. The primary purpose of the database is to facilitate sound research in the development of computer algorithms to aid in screening. Secondary purposes of the database may include the development of algorithms to aid in the diagnosis and the development of teaching or training aids. The database contains approximately 2,500 studies. Each study includes two images of each breast, along with some associated patient information (age at time of study, ACR breast density rating, subtletly rating for abnormalities, ACR keyword description of abnormalities) and image information (scanner, spatial resolution, ...). Images containing suspicious areas have associated pixel-level "ground truth" information about the locations and types of suspicious regions.
The Digital Database for Screening Mammography is organized into "cases" and "volumes." A "case" is a collection of images and information corresponding to one mammography exam of one patient. A "volume" is simply a collection of cases collected together for purposes of ease of distribution. A case consists of between 6 and 10 files. These are an "ics" file, an overview "16-bit PGM" file, four image files that are compressed with lossless JPEG encoding and zero to four overlay files. Normal cases will not have any overlay files. We have 2620 cases available in 43 volumes.
http://braid.rad.jhu.edu/interface.html
is a large-scale archive of normalized digital spatial and functional data with an analytical query mechanism. One of its many applications is the elucidation of brain structure-function relationships. BRAID stores spatially defined data from digital brain images which have been mapped into normalized Cartesian coordinates, allowing image data from large populations of patients to be combined and compared. The database also contains neurological data from each patient and a query mechanism that can perform statistical structure-function correlations
http://www.cytopathnet.org/imagedb/
are offered by Cytopathnet, a 501(c)3 non-profit organization which aims to provide great and innovative solutions for online education and professional services in the field of cytopathology and pathology. We are dedicated to providing cytopathology information to healthcare professionals and improving the standards and quality of cytopathology through online education and collaboration.
http://rad.usuhs.mil/synapse/
contains 11351 Images, 3002 Factoids, and 2949 Patients.
In the aggregate, the largest number of freely available images are those of objects being sold or traded in the marketplace. These include vast imagebases from numerous sources of automobiles, boats, cars, clothing, furnishings, real estate, and every consumer product available today. They are accompanied by metadata regarded by their owners as important to the sale of the item. Harvesting huge databases of such images from the public web would not be difficult and could support a range of metadata based retrieval research. No effort is made here to list sources where commercial objects are listed for sale
A Listing Based on data compiled by the CMU Calibrated Imaging Laboratory, see http://www-2.cs.cmu.edu/~cil/v-images.html and other web sites
http://www.iv.optica.csic.es/projects/database.html
Images & sequences for vision research
http://www.bioid-dev.de/facedb/facedatabase.html
includes 1521 images with human faces, recorded under natural conditions, i.e. varying illumination and complex background. The eye positions have been set manually (and are included in the set) for calculating the accuracy of a face detector. A formula is presented to normalize the decision of a match or mismatch.
The first attempt to finally create a real test scenario with precise rules on how to calculate the accuracy of a face detector - open for all to compare their results in a scientific way.
http://www.cog.brown.edu/~tarr/stimuli.html
A variety of datasets including geons, objects, and "greebles". Good for testing recognition algorithms. (Formats: pict)
http://www.vision.caltech.edu/html-files/archive.html
- about 20 images - mostly top-down views of small objects and toys. (Formats: GIF)
http://www.cs.waikato.ac.nz/~singlis/ccitt.html
- 8 images) (Formats: gif)
http://www-2.cs.cmu.edu/~cil/cil-ster.html
- 3 sets of 11 images, including color tiff images with spectroradiometry (Formats: gif, tiff)
http://www.ri.cmu.edu/projects/project_418.html
- A database of 41,368 face images of 68 people captured under 13 poses, 43 illuminations conditions, and with 4 different expressions.
http://www.ius.cs.cmu.edu/idb/
- Images, sequences, stereo pairs (thousands of images) (Formats: Sun Rasterimage)
contains over 5000 images split up over nearly 200 different data sets including:
* Motion Data - A large set of motion series. In most cases, sampling rate is not available.
* Road Sequences - Many road image sequences, taken from our Navlab series of vehicles.
* Stereo Data - A large set of stereo (left/right) images. Baseline information is generally not available.
* CIL's Stereo Data with Ground Truth - 3 sets of 11 images, including color tiff images with spectroradiometry
* JISCT Data - Datasets provided by research groups at JPL, INRIA, SRI, CMU, and Teleos.
* Faces and Facial expressions - Testing images for the face detection task, and the facial expression database.
* Car Data - Testing images for the car detection task.
http://www.cs.columbia.edu/CAVE/curet/
- Texture and reflectance measurements for over 60 samples of 3D texture, observed with over 200 different combinations of viewing and illumination directions. (Formats: bmp)
http://www.cs.sfu.ca/~colour/data/index.html
- A dataset oriented towards computational color constancy, but useful for computer vision in general. It includes synthetic data, camera sensor data, and over 700 images. (Formats: tiff) <i> (<a href="http://www.cs.sfu.ca/~colour/ Computational Vision Lab</a> / <a href="http://www.sfu.ca/ Simon Fraser University</a>) </i>
http://www.cs.washington.edu/research/imagedatabase/groundtruth/
11 sets of color images for testing algorithms for content-based retrieval. Most sets have a description file with names of objects in each image. (Formats: jpg) <i> (<a href="http://www.cs.washington.edu/research/imagedatabase/ Efficient Content-based Retrieval Group</a> / <a href="http://www.washington.edu University of Washington</a>) </i>
Commercially available Corel CD-Roms with ~200,000
images used in whole, or part by a number of groups.
These datasets were gathered as part of Kobus Barnard's Ph.D. research at SFU. Appropriate archival references for the data accompany each dataset.
á Camera Calibration Data for Sony DXC-930. http://www.cs.sfu.ca/~colour/data/camera_calibration/index.html
á
Colour Constancy Synthetic Test Data.
http://www.cs.sfu.ca/~colour/data/colour_constancy_synthetic_test_data/index.html
á Colour Constancy Test Images Captured Using Sony DXC-930.
http://www.cs.sfu.ca/~colour/data/colour_constancy_test_images/index.html
* Images of Objects Under Different Illuminants.
* Spectra of Fluorescent Surfaces.
* Object Recognition Image Database (Old Version - 1998)
http://vision.psych.umn.edu/www/kersten-lab/demos/digitalembryo.html
- Digital embryos are novel objects which may be used to develop and test object recognition systems. They have an organic appearance. (Formats: various formats are available on request) <i> (<a href="http://vision.psych.umn.edu/www/kersten-lab/kersten-lab.html Univerity of Minnesota Vision Lab</a> / University of Minnesota) </i>
http://www.dlib.org/test-suite/testbeds.html
A set of datasets that include many still images as well as moving images, texts and sound.
- images of Earth from the space shuttle - good search engine
The Earth and Space Science Browser http://focus.eecs.umich.edu/
1,400 images of the earth, planets, star systems, space craft and other earth and space science subjects.
http://www.si.umich.edu/Space/overview.html for earth and space science data;.
http://www.gastrointestinalatlas.com
- Images and Videos of his-res of studies taken from Gastrointestinal Video endoscopy. (Formats: jpg, mpg, gif)
http://bias.csr.unibo.it/fvc2000/
- FVC2000 is the First International Competition for Fingerprint Verification Algorithms. Four fingerprint databases constitute the FVC2000 benchmark (3520 fingerprints in all). <i> (<a href="http://bias.csr.unibo.it/research/biolab Biometric Systems Lab</a> / University of Bologna) </i>
ftp://ftp.eedsp.gatech.edu/database/images
- many images (Formats: unknown)
ftp://ftp.tu-graz.ac.at/pub/images
- stereo pairs (2 image pairs) (Formats: TIFF)
http://hlab.phys.rug.nl/archive.html
- 4000+ 1536x1024 (16 bit) calibrated outdoor images (Formats: homebrew)
Hyperspectral dataset of natural scenes http://www.crs4.it/~gjb/ftpJOSA.html
- Hyperspectral images of 29 natural scenes with 31 bands each collected by Bristol University for DRA UK. (Formats: radiometric pixels & gif preview)
http://www.ien.it/iengf/is/vislib.html
- 1000+ images, mostly outdoor sequences (Formats: raw, ppm)
ftp://krakatoa.inria.fr/pub/IMAGES_ROBOTVIS
- many images (Formats: PGM, homebrew) <i> (INRIA) </i>
http://www-syntim.inria.fr/syntim/analyse/images-eng.html
- 15 color image of simple objects (Formats: gif) <i> (<a href="http://www-syntim.inria.fr/syntim/ Syntim</a> / <a href="http://www.inria.fr/ INRIA</a>) </i>
http://www-syntim.inria.fr/syntim/analyse/paires-eng.html
- 34 calibrated color stereo pairs: (Formats: gif) <i> (<a href="http://www-syntim.inria.fr/syntim/ Syntim</a> / <a href="http://www.inria.fr/ INRIA</a>) </i>
http://www.mic.atr.co.jp/~mlyons/jaffe.html
- The JAFFE database consists of 213 images of Japanese female subjects posing 6 basic facial expressions as well as a neutral pose. Ratings on emotion adjectives are also available, free of charge, for research purposes. (Formats: TIFF Grayscale images.) <i> (<a href="http://www.mic.atr.co.jp/ ATR Research, Kyoto, Japan</a>) </i>
ftp://ftp.vislist.com/IMAGERY/JISCT/
- 44 image pairs. These data have been used in an evaluation of stereo analysis, as described in the April 1993 ARPA Image Understanding Workshop paper ``The JISCT Stereo Evaluation'' by R.C.Bolles, H.H.Baker, and M.J.Hannah, 263--274. (Formats: SSI) <i> (SRI) </i>
ftp://ftp.cs.jhu.edu/pub/natimages/
- several sets of four images (Formats: compressed Sun Rasterimage) <i> (Johns Hopkins) </i>
Lena
Standard image of 'Lena' used in JPG tests
http://www.ks.informatik.uni-kiel.de/~jpa/images.html
- dozens of images (Formats: PNG)
- image database (including a <a href="ftp://isy.liu.se/images/calib.iccalibrated outdoor stereo scene</a>; see <a href="ftp://isy.liu.se/images/READMEREADME</a> and <a href="ftp://isy.liu.se/images/Stereo.txtStereo.txt</a>) (Formats: homebrew)
ftp://ftp.c3.lanl.gov/pub/WSQ/print_data
- many images (Formats: raw 8-bit)
Other datasets available for research and browsing include (http://www.nrl.navy.mil/clementine/clib/ for lunar data;
ftp://ftp.cse.psu.edu/pub/vision/MACHINE_VISION/images/
- Images from the textbook by Jain, Kasturi, Schunck (20+ images) (Formats: GIF TIFF)
http://marathon.csee.usf.edu/Mammography/Database.html
- 100 or more images of mammograms with ground truth. Additional images available by request, and links to several other mammography databases are provided. (Formats: homebrew)
ftp://ftp.cps.msu.edu/pub/prip
- many images (Formats: unknown)
ftp://whitechapel.media.mit.edu/pub/images
- hundreds of images (Formats: homebrew) <i> (MIT) </i>
http://www-white.media.mit.edu/vismod/imagery/VisionTexture/vistex.html
- Image archive (100+ images) (Formats: ppm)
http://ltpwww.gsfc.nasa.gov/MODIS/MAS/
- High Altitude Imagery from around the world for environmental modeling in support of NASA EOS program (Formats: JPG and HDF)
http://www.cselt.it/mgeg/standards.htm, 7,000 still images,
http://www.designrepository.org
- Over 55,000 3D CAD and solid models of (mostly) mechanical/machined engineerign designs. (Formats: gif,vrml,wrl,stp,sat) <i> (<a href="http://gicl.mcs.drexel.edu Geometric & Intelligent Computing Laboratory</a> / <a href="http://www.drexel.edu Drexel University</a>) </i>
ftp://sequoyah.ncsl.nist.gov/pub/databases/data
- datasets - thousands of images (Formats: unknown)
ftp://ftp.cs.columbia.edu/jpeg/other/uuencoded
- compressed multipart uuencoded tar file <i> (Columbia) </i>
http://www.nist.gov/srd/nistsd18.htm
is being distributed for use in development and testing of automated mugshot identification systems. [iii]
The database consists of three CD-ROMs, containing a total of 3248 images of variable size using lossless compression. Each CD-ROM requires approximately 530 megabytes of storage compressed and 1.2 gigabytes uncompressed (2.2:1 average compression ratio). There are images of 1573 individuals (cases) 1495 male and 78 female. The database contains both front and side (profile) views when available. Separating front views and profiles, there are 131 cases with two or more front views and 1418 with only one front view.
This NIST Special Database has the following features:
* 3248 segmented 8-bit gray scale mugshot images (varying sizes) of 1573 individuals
* 1333 cases with both front and profile views (see statistics above)
* 131 cases with two or more front views and 89 cases with two or more profiles
* images scanned at 19.7 pixels per mm
* image format documentation and example software.
http://www.nlm.nih.gov/research/visible/visible_human.html
- Color, CAT and MRI image samples - over 30 images (Formats: jpeg)
http://www.cam-orl.co.uk/facedatabase.html
- 400 images (Formats: pgm)
ftp://csd.uwo.ca/pub/vision/TESTDATA/
- see <em>Performance of Optical Flow Techniques</em> under <a href="v-source.htmlRESEARCH CODE</a> (6 synthetic and 4 real image sequences) (Formats: Sun Rasterimage)
http://eewww.eng.ohio-state.edu/~flynn/3DDB/Models/
- several sets of 3D object models collected over several years to use in object recognition research (Formats: homebrew, vrml)
http://eewww.eng.ohio-state.edu/~flynn/3DDB/RID/
- Hundreds of real and synthetic images (Formats: gif, homebrew)
http://sampl.eng.ohio-state.edu/~sampl/database.htm
- Over 1000 range images, 3D object models, still images and motion sequences (Formats: gif, ppm, vrml, homebrew) <i> (<a href="http://sampl.eng.ohio-state.edu Signal Analysis and Machine Perception Laboratory</a> / <a href="http://www.osu.edu The Ohio State University</a>) </i>
http://www.cs.otago.ac.nz/research/vision/Research/OpticalFlow/opticalflow.html
- Synthetic and real sequences with machine-readable ground truth optical flow fields, plus tools to generate ground truth for new sequences. (Formats: ppm,tif,homebrew) <i> (<a href="http://www.cs.otago.ac.nz/research/vision/index.html Vision Research Group</a> / <a href="http://www.otago.ac.nz/ University of Otago</a>) </i>
ftp://ftp.limsi.fr/pub/quenot/opflow/testdata/piv/
- Real and synthetic image sequences used for testing a Particle Image Velocimetry application. These images may be used for the test of optical flow and image matching algorithms. (Formats: pgm (raw)) http://www.limsi.fr/Recherche/IMM/PageIMM.html
ftp://ftp.balltown.cma.com/pub/images
many formats
http://bion.nada.kth.se/~zucch/CAMERA/PUB/seq.html
- synthetic sequence for testing structure from motion algorithms (Formats: pgm) <i> (<a href="http://bion.nada.kth.se CVAP</a>)
http://www.cee.hw.ac.uk/~mtc/sofa
- 9 synthetic sequences designed for testing motion analysis applications, including full ground truth of motion and camera parameters. (Formats: gif (<a href="http://www.cee.hw.ac.uk/~mtc/research.html Computer Vision Group</a> / <a href="http://www.cee.hw.ac.uk/ Dept of Computing and Electrical Engineering, Heriot-Watt University</a>)
560,000 images of wihc ~17,000 were used in the CHABOT and CYPRESS projects at the University of California, Berkeley
Ogle & Stonebraker,1995
http://elib.cs.berkeley.edu/photos/dwr/about.html
http://www-dbv.cs.uni-bonn.de/stereo_data/
- a small set of synthetic images of a hallway with varying amounts of noise added. Use these images to benchmark your stereo algorithm. (Formats: raw, viff (khoros), or tiff)
ftp://ftp.ifp.uni-stuttgart.de/pub/wg3
- stereo and/or infrared images with approximate ground truth (4 compressed datasets) (Formats: PGM)
http://range.informatik.uni-stuttgart.de
- A collection of synthetic range images taken from high-resolution polygonal models available on the web (Formats: homebrew) <i> (<a href="http://www.informatik.uni-stuttgart.de/ipvr/bv/bv_home_engl.html Department Image Understanding</a> / <a href="http://www.uni-stuttgart.de University of Stuttgart</a>) </i>
http://rvl1.ecn.purdue.edu/~aleix/aleix_face_DB.html
- Contains over 4,000 color images corresponding to 126 people's faces (70 men and 56 women). Frontal views with variations in facial expressions, illumination, and occlusions. (Formats: RAW (RGB 24-bit)) <i> (<a href="http://rvl.www.ecn.purdue.edu/RVL/ Purdue Robot Vision Lab</a>) </i>
http://sipi.usc.edu/services/database/Database.html
is a collection of digitized images. It is maintained primarily to support research in image processing, image analysis, and machine vision. The first edition of the USC-SIPI image database was distributed in 1977 and many new images have been added since then. The database is divided into volumes based on the basic character of the pictures. Images in each volume are of various sizes such as 256x256 pixels, 512x512 pixels, or 1024x1024 pixels. All images are 8 bits/pixel for black and white images, 24 bits/pixel for color images. The following volumes are currently available:
Textures Brodatz textures, texture mosaics, etc.
Aerials High altitude aerial images
Miscellaneous Lenna, the baboon, and other favorites
Sequences Moving head, fly-overs, moving vehicles
http://www-white.media.mit.edu/vismod/imagery/VisionTexture/vistex.html
is a collection of texture images. The database was created with the intention of providing a large set of high quality textures for computer vision applications. Unlike other texture collections, the images in VisTex do not conform to rigid frontal plane perspectives and studio lighting conditions. The goal of VisTex is to provide texture images that are representative of real world conditions. While VisTex can serve as a replacement for traditional texture collections, it includes examples of many non-traditional textures. The database has 4 main components:
* Reference Textures: 100+ homogeneous textures in frontal and oblique perspectives.
* Texture Scenes: Images with multiple textures. ("real-world") scenes.
* Video Textures: Sequences of temporal textures. (UNDER CONSTRUCTION)
* Video Orbits: Images within a common projective group.(UNDER CONSTRUCTION)
http://i21www.ira.uka.de/image_sequences
- thousands of frames of digitized traffic image sequences as well as the 'Marbled Block' sequence (grayscale images) (Formats: GIF) <i> (<a href="http://i21www.ira.uka.de IAKS/KOGS</a> / <a href="http://www.uni-karlsruhe.de Universitaet Karlsruhe (TH)</a>) </i>
ftp://ftp.iam.unibe.ch/pub/Images/FaceImages
- hundreds of images (Formats: Sun rasterfile)
ftp://freebie.engin.umich.edu/pub/misc/textures
(Formats: compressed raw)
ftp://ftp.ee.oulu.fi/pub/tklab/
- Includes classifications - 1000+ color images (Formats: ppm)
http://www.cis.plym.ac.uk/cis/3Darchive.html
- 1800 views of 5 synthetic 3D objects, the views being collected from the whole upper viewing hemisphere
http://periscope.cs.umass.edu/~vislib/
- Large image database with aerial, space, stereo, medical images and more. (Formats: homebrew)
ftp://sunsite.unc.edu/pub/academic/computer-science/virtual-reality/3d
- many images (Formats: GIF)
http://www.ee.oulu.fi/research/imag/color/pbfd.html
contains color images of faces under different illuminants and camera calibration conditions as well as skin spectral reflectance measurements of each person. Machine Vision and Media Processing Unit. University of Oulu http://www.oulu.fi
ftp://ftp.uu.net/published/usenix/faces
hundreds of images in several formats
was first made available
in 1977 (http://sipi.usc.edu/services/database/Database.html)
.
http://marathon.csee.usf.edu/range/seg-comp/SegComp.html
- 80 image sets (Formats: Sun rasterimage)
ftp://cs.utah.edu/pub/range-database/
- 33 images (Formats: homebrew formats)
http://www.prip.tuwien.ac.at/prip/image.html
- An image database including some textures
http://www-prima.inrialpes.fr/Prima/hall/view_sphere.html
- Images of 8 objects seen from many different view points. The view sphere is sampled using a geodesic with 172 images/sphere. Two sets for training and testing are available. (Formats: ppm) http://www-prima.inrialpes.fr/Prima/
ftp://ftp.vislist.com/IMAGERY/
- Many images, many formats
http://cvc.yale.edu/projects/yalefaces/yalefaces.html
- 165 images (15 individuals) with different lighting, expression, and occlusion configurations.
Center for Computational Vision and Control, Yale University http://cvc.yale.edu/
- 5760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses x 64 illumination conditions). (Formats: PGM) http://cvc.yale.edu/projects/yalefacesB/yalefacesB.html
James Wang's group, for example, maintains a demonstration on-line against 70,000+ images and several downloadable databases for research comparison
- 10,000 test images (misc. database used in WBIIS);
- 60,000 test images (corel1m database used in SIMPLIcity, 1.9GB);
- 1,000 test images (test database used in SIMPLIcity paper). http://wang.ist.psu.edu/docs/related/
Anne R. Kenney. "Test Database for Digital Visual Resources in Art History. CLIR Issues., No. 18. November/December 2000. Available http://www.clir.org/pubs/issues/issues18.html#test [July 31, 2002].
Anne Kenney. "Creating a Test Database for Digital Visual Resources". Coalition for Networked Information Fall 2000 Task Force Meeting. San Antonio, TX, December 7-9, 2000. Unpublished Presentation slides
"Planning Meeting for Test Database for Digital Visual Resources". Association for Research Libraries, Washington, DC. May 31, 2001, Unpublished Report.
Clifford Lynch, "Scenarios for the Development of an Imaging Benchmark Database". Unpublished draft. February 7, 2002.
Image Retrieval Benchmark Database. http://www.cni.org/projects
Proposal brought to planning meeting, May 2001
[i]
The website provides examples of metadata that might
surprise humanists using images:
Number of missions in
catalog: 143
Number of frames in
catalog: 433435
Special film types
Number
of frames that are color visible light: 347896
Number
of frames that are color infrared: 42143
Number
of frames that are black and white: 23793
Number of frames taken
with ESC: 127
Camera / Film formats
Number
of frames taken by Skylab Multispectral Photographic Camera: 33285
Number
of frames taken by Skylab Earth Terrain Camera: 5478
Number
of frames taken by Linhof: 29557
Number
of frames taken by Hasselblad: 328432
Number of frames taken
by 35 mm: 15939
Number of frames with
an entry for tilt angle (suggesting they are earth-looking): 367518
Of
these, number of remaining frames with an entry for focal length:
349195
Of
these, number of remaining frames with normal exposure: 227040
Of
these, number of frames designated with HO tilt angle: 41268
Of
these, subset taken with 40-89 mm lens: 13287
Of
these, subset taken with 90-249 mm lens: 18293
Of
these, subset taken with 250 + mm lens: 8824 Of these, number of
frames designated with LO or NV tilt angle: 80596
Of
these, subset taken with 40-89 mm lens: 6033
Of
these, number with center and nadir point data: 657 Of these, subset
taken with 90-249 mm lens: 32841
Of
these, number with center and nadir point data: 7247 Of these, subset
taken with 250 + mm lens: 41477
Of these, number with
center and nadir point data: 12495
FRAMES <= 25% cloud
cover
Number of frames <=
25% cloud cover in catalog: 218230
Number of these frames
with an entry for tilt angle (suggesting they are earth-looking):
204489
Of
these, number of remaining frames with an entry for focal length:
196922
Of
these, number of remaining frames with normal exposure: 133566
Of
these, number of frames designated with HO tilt angle: 14615
Of
these, subset taken with 40-89 mm lens: 4552
Of
these, subset taken with 90-249 mm lens: 6826
Of
these, subset taken with 250 + mm lens: 3125 Of these, number of
frames designated with LO or NV tilt angle: 47497
Of
these, subset taken with 40-89 mm lens: 2524
Of
these, number with center and nadir point data: 432 Of these, subset
taken with 90-249 mm lens: 19056
Of
these, number with center and nadir point data: 4544 Of these, subset
taken with 250 + mm lens: 25828
Of these, number with
center and nadir point data: 8424
FRAMES >= 70% cloud
cover
Number of frames >=
70% cloud cover in catalog: 65426
SUNGLINT
Number of frames taken
with a sun elevation between 42 and 57 degrees. 75342
[ii]
The Facial Expression Resources Page distinguishes
this field of cognitive studies from facial recognition and provides
links to research groups and other resources concerning facial expression
perception, recognition and synthesis. It lists researchers in Cognitive
Modeling; Experimental Psychology/Cognitive Science; Automatic Facial
Expression Recognition & Cognitive Engineering; anf Facial Expression
Synthesis.
[iii] The Face Recognition Home Page, http://www.cs.rug.nl/users/peterkr/FACE/face.html, lists 75 Research Groups with their urls and half a dozen university face databases being used for facial recognition research.