Image Processing and the (Art) Historical Dicscipline

Image Processing and the (Art) Historical Discipline 5 Image Processing and the (Art} Historical Discipline
Jörgen den Berg, Hans Brandhorst and Peter van Hui tede
Preface
In the cond chapter of this article, „computers and images“, we will introduce some b ic concepts of image proce ing. One can nd in depth coverage on l topics included there in many speci ized publications. We hope that our perspective: image proc sing in relation to the {art) historical discipline, answers for our somewhat cursory treatment of these technical issues.
We are convinced that tbe m n issues of image processing in the (art) historical discipüne are not technical, but intellectual {What do we study and how do we study it?) and conceptual {How do we present the results of our studi ?). With the variation of soft-and hardware av lable today it is not di cult to get a pictorial Information system up and running.1 Anyone who h recently visited a Computer show will know examples of pictorial Information systems. estate rms have applications which allow the customer to view images of houses currently for sale. These applications are real pictorial Information systems in the sense that they contain images (of houses) and Information about these images; they allow users to select the images, and they pr ent the selection on the computer screen. The purpo of these applications is straightforward: they o er the r of the system the available information on the topic of bis choice, for example „houses with gardens“.
a very general nse the situation sketched above can be compared with that of pietonal Information systems in the ( ) historical discipline: any pieton Information system must be designed around the idea that such a system, no matter how (un}advanced its technical featur , should be informative. That is: it should provide others with the p sibility of studying the visual materi presented in the system.
However, pietonal Information systems in the realm of ( ) historical research are set apart from the example cited above, because some very complex issues are involved. Questions we have to de with include:
• What exactly is the subject of historic research and what are its sources?
• What Information can be distilled from images from the p t?
• What are the different, often impücitly held, epistemological and methodological po­
sitions in the realm of (art) history?
These and other theoretic questions have seldom been studied systematically in historical research. We shall have to Iook at them in some detail before we start discussing the subject of image processing proper. But even before that, we must take a closer Iook at two important and related concepts we shall use: Information and the image as a carrier of meaning.
The rst concept is treated in a very interesting way in the book „Silicon Dreams.
Information, man and machine“ by Robert W. Lucky.
1 In this article we will refer to computer system containing imag in a digitized format we textual information about the images and the programs to sel t images using certain criteria pictorial information systems.

6 Jö en van den Be , Hans Brandhorst and Pettr van Huisstede
Information
Lucky lo ly distinguishes four Ievels in the „information hierarchy“. From bottom to top: data, information, knowledge and wisdom. Placem nt in the hierarchy is determined by the extent to which information is organized, distilled and integrated.
Using this concept of the hierarchical arrangement of information, we shall speak about the visu materi om the p t, e.g. in the shape of digitized images, „data“. We shall u the „b ic information“ to denote a rst Ievel of organization of the vi­ su material, in particular information about their size, maker, title, location, provenance, keywords about contents, etc. In doing so we comply with common documentation prac­ tice. The term „extend information“ denotes an even more organized and integrated way of pr enting information about visual materi . It refers in particular to pictorial information systems cont ning detailed, systematic, and consistent descriptions of the images.
This hierarchical distinction has the ditional bene t of bringing two important methodological i u out in the eld:
i) The electronic publication of di tized images- data- without additional informa­ tion will be of little or no use to anyone in the eld of (art) historical research. At rst sight this may m to be a trivial remark, but it bears great weight when we take the issue of proj t management of pictorial information systems into consideration.
ii We go from one Ievel of the information hier chy to another by adding information through interpret ion. It is a sound scienti c principle to present ienti c conclusions in such a way that they may be f si ed. Therefore, if we agree that pictorial informa­ tion systems provide extend information, we should supply u rs of these systems with instruments to ch k the highly integrated information cont ned in them ag nst the data . We ve an ex ple of this when the use of controlled vocabularies to d cribe the contents of im es is explained (section 3.3).
The Im e as a Carrier of M og
With this concept we enter the proper dom n of history and art history. From an epistemolo c point of view we have to k the question whether the historical mean­ ing(s) of an image or its contemporary meaning(s) is (are) our subject. This question someüm is referred to „the art historical dilemma“. The way in which this qu tion is answered h an important e ect upon the methods of art historical research and the p ible communication of the results of that research (Cf. section 1.3).
From a meth ologic point of view it is important to know which elements of an image (colour, form, content or a mixture of form d content) make it possible to state that it, or part of it, carries a particular meaning. Moreover, it important to know whether formal conventions or conventions conceming the contents of images play a part; and if so, whetber there are relations between the two conventions? By far the most important i ue is how to d cribe the topics historic phenomena?

Image Processing and the (Art} Historical Discipline 7
1. General Introduction: Images and Views of Images
1.1. Images
When we speak of images- data- in the context of the (art) historic discipline,
what do we mean by this term? Typically, we refer to reproductions of objects. These objects may be p ntings, engravings, sculptures, drawings, etc. The reproductions can be anything from drawn copies of objects to digital images.
It is often s d that the reproduction of an object is not the same as the object itself; a profound distinction, with which we fully agree. Another matter, also often discus d, is whether the reproduction of an object is always less informative than the object it lf. Or, maybe better: „When we are de ing with reproductions of objects, are we always d nec arily handicapped because of the reduction of information?“
We would argue that this is not always and necessarily the c e. lt depends on the purpose for which the reproductions are used. For many types of research, having reproductions at one’s disposal is conditional to the study of the objects. For example, reproductions of objects have the great advantage that they may be arranged and re­ arranged according to certain hypotheses. This would hardly be a procedure one could follow using the original objects! Especially not if the hypotheses involve a !arge number of objects. Of course, there may be cases where one needs to go back to the original object. For example, if the ex t setting of a painting in its historical surroundings is relevant to the researcb. Reproductions are in many cases M ctly suited to study the objects tbey represent, historical practice shows us daily. This discipline relies almost exclusively of handling photographs, slides and drawings, in short reproductions of the objects it studies.
1.2. Collections of Images
A eper of medieval manuscripts can weil see the bene ts of having the manuscript collection of his library digitized and stored in the computer. The originals can remain in their climatized vault, while the users of the library are glued to the screens of the computers, studying the images and texts, comparing di erent manuscripts and perhaps even use certain images to retrieve other but similar images.
Not only would this Situation contribute to the preservation of these valuable objects, but it would make it easier to study these manuscripts at other places. And what to think of the support the computer could o er to the publication of a manuscript once it is digitized? So, when do we start? Weil, there are still a few problems to be solved, before we can set o to digitize the visual heritage of the past.
The main problems are intellectual (How is it to be done, who will do it and for whom?) and n cial (Sta , equipment). Paradoxically, Iack of funds does not appear to be a major obst le in the initial ph es of a project involving the digitization of images. What is often underestimated, however, is that in order to provide access to the stored images to various researchers, one needs good, that means detailed, systematic, and consistent descriptions of the contents of these images. preferably supplemented by software to manipulate the digitized images.
8 Jö en van den Be . Hans Brandhorst and Peter van Huisstede
th demands might s m to be a bit farfetched for our keeper of manuscripts. The t that he is a realistic k per of a collection will probably make him concentrate on providing tbe u rs of bis collection with b ic information about the objects.
Making preci d systematic de riptions of tl contents of the images in his coll tion, or providing r earchers with ftwar to explore the digitiz images are simply not bis rst priorities. They never were, and it is not likely that the curators of collections of visu materi will on a bro e succ ed in getting these activities funded.
One can e ily see that !arge funds will be needed, becau , ev n with the newest techniques and the most up to date hard- and software, th real investment will hav to be in (p ple) either describing the images in ord r to acquir useful knowledge or d igning and implementing the ftware ne ed for image manipulation and for the medium of pr entation: pictori information systems. lt is h re, ntering the elds of historical and historic r arch, that we are confronted with the m n epistemological d methodologic problems.
1.3. cbers d Images
1.3.1. Historians
Images are used more and more in historical res arch. Historians hav discovered
that images om the p t can cont n v uable historic information. Therefore they are inclined to incorporate interpretations of visual mat rial in their historical re oning. This pr tice can be traced back to the growing inter st of historians for re arch of the everyday life in the p t and with more exotic themes like the history of sm lls or witchcraft. Studying them like this, l sorts of sourc , previously not often used by historians – im , literature, plays – are consulted.
Th ing pr tice is not without its di culti and dang rs. In order to use images a historic urce, the histori n eds cess to !arge quantities of systematically and con stently d ribed visual material. One simply can not build an argument on a few im . Moreover, the historian n ds information on how to „read“ images. In other words: how d he protect bim lf against incorr t interpretations (we can think for e ple of a phenomenon called „eroded“ symbolism)? Only !arge amounts of im es, their forms and contents carefully d cribed and an yzed elements belonging to cert n cultur contexts of the p t, will provide a safe platform to start from.
If imag from the p t cont n historic information, how do we study the imag ?
And more importantly, who is going to provide the information – both b ic and extended – about the images? Historians, who are presently discovering this n w area of research, or art histori s who tr itionally (to be t en quite Jiterally, we are afraid) study visual materi om the p t? ls the study of the visual material from the p t the eld where historians fruitfully can draw on the work of art historians? Not yet. In order to expl n this I of cooperation we have to Iook at some features of historical re arch.
Image ocessing and lhe (Art)Hi8torical Discipline 9
1.3.2. Art Historians
In the preface of tbis article we ideuti ed three issues responsible for the complexity of the topic of image proce ing in the eld of (art) historical research. We now explore them a bit further.
• What ex tly is the subject of historical r arch d what are its sources?
Art historians study �works of art“ from the p t. Tbey do not study all the visu material from the p t av lable to them in any systematic way. Most of their r arch is b d on a prede n sub t of the available material. This su t may be Iabelied „great „. lts lection is b d upon aesthetic preferences from the 19th century in combination with evolutionary view of the autonomaus development of art, an idea that can so be tr ed b k to the 19th century and beyond. Admittedly this subset is by no means static, but l modi cations neverthele branch om it. lts existence results in the neglect of what we may call „sm l art“: images of no or little tistic value, that o en contain very v uable historic information. The re arch questions in which historians implicate images, m t force them to be interested in the results of the study of ALL the visual
material av lable from the p t.
• What information can be distilled from im es of the p t?
lf one intends to study visual material from the p t in a historical way, there are two
promising are of re arch, from a methodologic point of view:
1. The area where a source consists of both image and text. In this c e the student h two contemporary and related sourc at bis disp al to give bis d criptions d interpretations a rm historic b is. There are many historical sources that belong to this category. To name but a : f tiviti , sbop signs, banners, wedding­ poems, emblemata, printer’s devic , illustrat books. Most of the belong to the category of „sm ler “ d e not m e av lable in a systematic way.
2. The �ea where forms are studi carrie of iconographic meaning. ition ly art historians c be divided into two groups.
• Th e who study works of visual phenomena, focu ing.on how thin are depicted (keywords: stylistic research, artistic quality, connoisseur, morpholo ). • Th e who study the contents of works of art, focussing on what is depicted. (keywords: iconography, iconolo , typolo ). Only a few art historians m to be able to bridge this tradition gap and formulat methods that enable them to study forms carriers of meaning within certain pr isely de ribed cultur contexts. An example is Aby M. Warburg who studied „Pathosformel“, forms from Antiquity that were re-u d in l sorts of cultur contexts. What is especi ly interesting in bis c , is that, while studying this phenomenon, Warburg conc ntrated on images that could be cl si ed „small art“. Th images enabled bim to show how cert n forms and motives were used and re­ used. Because these image were closely linked to everyday life, Warburg could sh that he w de ribing a living tradition in the p t. Warburg w ways eager to sbow that bis method allowed him to study „works of art“ (the „great art“ of the historians) with much more insight than bad he started with the „works of t“ right away. This brin us again to th f inating i ue of the art
historical dilemma.
10 J van den Be , Hans B ndhorst and Peter van Huisstede
• What are the di erent, often implicitly h ld, epistemological and methodological po­ sitions in the eld of art historical re arch?
A cruci epistemolo cal position, commonly held by art historians, researchers of
style and iconographers ike, can be formulated : „Works of contain historical information weil the keys necessary to interpret this information“. Or, in other words, the direct c ontation between art historian and work of art renders information about the work of . Whether or not this information can be c l information about what h happen in the p t, is a question often not addressed. It is this much daimed sensitivity of histori s that Ieads them directly into the boldest hermeneulies and, by de nition, away om carefully explaining what h happened in the p t.2
It is this diver nce on the epistemological level between historians, tr ned to produce Statements about what h happened in the p t by carefully studying many elements of that p t p i ble, and art historians, often engag d in a contemporary di u ion with elements om the p t, that accounts for thc di cult relation between the two groups of r archcrs.
1.3.3. Historians d A Historians: Towards a History of lmagery
g the present Situation, we may identify a promising eld of re arch: the systematic study of im es of the p t with the help of the computer. The researchers tr ition ly occupi with the study of works of art of the p t, art histori s, do not (yet) show the inclination to cover this eld. Moreover, their training rarely provides them with tbe right epistemolo c and methodological background or approach. The rese chers inter ted in the r ults of the study of imag ry – historians of various di iplines but r chers om fields like musicolo or botany – do not have the tradition to actu ly cover the eld them lves. At the same time we see that institutions, instigated by the av lability of new techniqu , begin to think of opening up their coll tions of imag and provide acce to their iconography. If historians and historians fail now to set up intellectu guidelines for such projects, the rst e orts in this eld might start o on the wrong foot.
Sto ng di ti imag with the help of the computer is one thing. Creating facilities to retrieve them, not knowing what the special interest will be of the persons that want to nd them, is quite other. It is theoretically impossible to predict l the questions that r archers ll be king. lt is already hard enough to nd out what type of information they want at this moment; with few collections described iconographic ly, r earchers often do not ther to k detailed iconographic questions at all. So, to guarantee a minimum of reliability d e ectiveness, the dcscriptions of the images in a pictori infor­ mation system must at le t be systematic and consistent in their Ievel of det l. It goes without saying that the more detailed they are, the better it will be.
Therefore a rst guideline reads:
• Create systematic, consistent and preferably detailed d criptions of the contents – de ned both formal and iconographic – of images.
2 For a more el rate di i on of tbese problems see R. Klein, Fo and Meaning. E ays on the Renai c e and Modern A , New York, 1979.

Image Processing and the (Art) Hislorical Dis ipline 11
Images, considered to be of limited artistic val e, may weil be of great historical value.
They may even turn out to be of great value for art history it lf, once the discipline h realized their potential. This holds m t strongly if images are documented to have been used for certain occasions by certain groups of people. They also often show the same motives and themes the so-c led great works of art.
So, we may formulate a second guideline:
• The aesthetic qu ity of individual objects should not be taken into consideration when a particular collection of images is selected to be made available a urce for research. .
A good starting point might weil be the study of sources that consist of both image
and text, like illustrated books. A source in which texts and irnages are mutually related­ e.g. contemporary texts describing the images – provides rmer ground for interpreting the images Using this type of source, we are better equipped to get round a common historical problem. Art historical arguments are often ba d on two distinct interpretations, one of an image and one of a text, without a solid historical explanation of the relation between these two sources. Because both images and texts can by de nition be interpreted in various ways, it is the task of art history to explain historically what exactly gives it the right to interpret two objects in that particular relation.
Thus, a third guideline reads:
·
• Goncentrate on sources that consist of both images and texts.
1.4. Im es d Computers
So far we have not paid attention to two important parties playing a role in the pro­ duction of pictori information systems: the computer industry and computer scientists. The reason for this postponement is simple and h been expl ned in the foregoing c­ tions: when we, keepers of visual collections and researchers working within the eld of the humaniti , want to study images with the help of computers, the problems we face are rst of l intellectual and conceptual. But all this not to say we are not interest in the products of these two groups. Of course we are, or better: we have to be. If it were not for the f t acceptance of the personal comp ter within the eld of the humanities, this article would not be written.
Computer industry
The computer industry works with markets and products. At the end of a year they balance earnings and spendings and there needs to be a pro t in order to survive. We, reserachers working with computers within the eld of the humanities, have to rely, to a certain extent, on the products of the computer industry. The simple truth is that history or history is not considered a market by the computer industry. Therefore we can not inßuence technical developments. These developments are said to be technolo and market driven.
This sounds far more serious than it really is. It just means that since we cannot inßuence technical developments, we have to rely on the standard products produced. And now for the good news: standard products are low priced. Where hardware is concerned, we can safely state that we, working in an application driven environment (we have to make applications) have bene tted from these market and t chnology driven developments.
12 Jö en van den Be Hans Brondhorst and P ler t•an Huisstt’de
Personal Computers, storage devices etc. all became bigger, f ter at roughly the same or even mewhat lower prices.
On the other hand, where software (the programs) is concerned things get a bit more complicated. Here the gap between the two nvironm nts (technolo /market driven and application driven) is bigger. If we want to use th standard products, like DBMS’s and edito , developed by the comput r industry we have to work with low priced products with static functionalities. If we do not want these static functionalities, and chances are good that we do not want them because the programs were developed with a completely di erent market in mind, we have two options:
• Develop with the computer industry custom design d products. They will have a user de ned functionality, but will also be very exp nsiv .
• Develop our own ftware and only rely on the computer industry for Standard prod­ ucts we can . It is in this are , an area we can rightly call humanities and com­ puting, that we interesting developments and products.
To name just a few:
• computer languag : SNOBOL, SPITBOL and ICON.
• DBMS’s: e , HIDA/MIDAS.
• Controlled v abularies/thesauri: ICONCLASS Browser, Art and Architecture The­
saurus, Union List of Artists‘ Names.
mputer scientists.
It is rare for projects within the humanities to have computer scientists members of sta . Especially where the user de ned functionality of software is concerned, cooperation could be very fruitful. The theme of this cooperation should be the design and production of so w e for research within the eld of humanities: informatics for the humanities. At the moment we see at universities the development of research groups around this theme.
in l we may distinguish four parties with di erent objectives: • Collections of imag .
Collections of prints and drawings, museums and libraries. Collections of photograpbs. Slide libraries. Main t ks: archiving, presentation, basic registration. In their hold­ ings a complete range from (historical) objects to reproductions of these objects. The collections themselves usually have neither the manpower nor the intention to describe their holdin systematically and in great depth. Their task lies more in what we call b ic registration of their holdings, with or without the computer.
• Researchers: historians and art historians.
While historians e getting more and more interested in imag from the p t a historical urce, many art historians safely circle around in the limited domain of M(great) works of art“, not botbered whether their research can be called historic or even ienti c research. Th e art historians who want to change this situation and take a serious intt>rest in the study of the main art historical source – the imagery of the p t- with the help of the computer, will have to Iook at collections of visual materi , historians weil computer scientists for support.
Image Processing and the (Art) Historical Discipline 13
• Computer scientists.
Since problems within the eld of the humanities difef r fundament ly from the prob­ lems addr ed by the computer industry when they develop software, computer i­ entists are needed to develop user de ned software for the humaniti .
• Computer industry.
Products and markets. Revenues. Standard products.
The articles in this book deal with the use or study of images and m u ript urc
from the p t in an automated environment. It is a very promising area of r earch, provided we take the right steps and are willing to treat the data in such a that others can bene t from it. One can e ily that the four corners of the square, collections, r archers, computer scientists and the computer industry, are complementary elements.
1.5. Conclusions.
To sum up. The ingredients for succe fully generating pictori information systems
· in the art historical discipline seem to be:
• Cooperation between the parties involved: computer scientists, the computer industry,
historians and historians and keepers of collections.
• New ways of studying imagery of the p t by art historians: systematic and det led
study of images historical phenomena.
• Sound project management. If the main problems can be identi ed nanci – it is time consuming and expensive to have educated p ple study imag rical phenomena in a det led and systematic way- and intellectual- new ( art} historical methods have to be developed – it is usele to begin with spending the I ger part of a budget on hard-and software.
2. Pictorial Information Systems: Hardware and nctionality.
2.1. What a Pictorial Information System?
We el it is important to describe at this stage as exactly p sible what de tion
used for the term Pictorial Information System (PIS} within the context of th k. A Pictori Information System is the information system that controls and manag the devices for input, processing, storage, communication and output to provide a collection of pictori data for e y access by its users. (S.I<. Chang, 1989) We re ize that t bro de nition incorporates many if not all activities in the eld. It is so to say the m ri e of image processing and information system management. One day they b to be reconciled within one de nition. Many activities in the eld of image processing and information system management cover only a part of the eld and even have never the intention to come close to a PIS.
If we Iook at the most relevant topics related to image processing we should include
the following List:
• Digitization.
• Coding and data compression.
• Enhancement and restoration. • Segmentation.
• Imag� analysis and description.
• Image understanding.
• Pietorl information management.
14 Jö en van den Berg, ans Brandhorst and Prter van Huisstcde
Traditionally the rst four topics r fer to image processing (s.s.). Computer vision includ the rst six topics and PIS includes all ven topics. There is a duality of pic­ ture repre ntations (data structures) and processing (algorithms, processes). Th duality can be considered the duality of the physical picture r presentations and the logical picture repr entations. Physical picture representations are dir etly related to the pic­ ture obtained from the picture input devices. They includ the image as represented by a bitmap and/or v tors. Image proce ing usually deals with the physical picture represen­ tation. Logic picture representations are high-Jevel abstract d representations, denoting the relation structur i.e. the logical and semantic relationships a ong picture objects. Computer vi on and pictorial information management usually emph ize Jogical picture repr entations.
2.1-1- The Use of Pictori Information Systems.
It only in the l t ve years that the management of non-alphanumeric information, such digital images, h received the attention it deserves in the sub eld of designing and implementing information systems. Information systems that allow handling of digital imag require ! ge storage capacity even for images of average complexity. Due to the development of new Storage techniqu substantial amounts of digital images can now be handled in information systems at a ordable prices. In recent years video-boards (ded­ icated video proce rs with their own memory) have been developed with a re onable qu ity like VGA and Super VGA for the huge MSDOS-market. The size of this market lows for low prices, which brings pictorial information systems within the reach and on the desk of the aver e small workstation u r (the more powerful MSDOS-PC, Macint h etc).
The next generation of low c t video boards shows extreme improvement of the colour qu ity from 256 di erent colours (8 bits per pixel) to more than 32.000 di erent colours (15 bits per pixel). The booming market for graphic u r interfaces (the called GUI’s ke XWindows, Windows 3.0 and others) h been faciliated by the fact that the graphi colour c d h b ome standard equipment, replacing the monochrome boards­ glowing green scr ns! – of several years ago. These vances are absolutely ‚technology driven‘ and the humanities eld is going to benefit from it in the very same way from the introduction of person computers. In other elds this trend h been recognized and ted on much earlier. Pictori information handling shows a growing Iist of applications:
• Computer Aided Design.
• Remote sensing o f earth resources.
• Carto aphic and mapping.
• Geographie data proce ing.
• Robotics.
• Medic Picture Archiving and Communication System (PACS).
• O ce/Document automation.
So, maybe late, but certainly not le t: pictures an historical source: a use of PIS in the humanities.

Image Processing and the (Art) Histo,·ica/ Discipline
Picture Input Device I Picture Output Device �
Picture Processor
Picture Compression Processor
Picture Communication Interface I I Picture Storage Device Figure 1: Hardware uirements for Pictorial Information Systems
2.2. Hardware Requirements for Pictorial Information Systems. 2.2.1. Picture Input
High resolution scanners
Scanners come in di erent shapes and quality. Quality depends on the dynamic range (the maximal colour variation that still can be detected) and the re lution. Very cheap hand-held scanners exist that are perfectly suitable for personal desktop publishing but do not o er the quality one needs for a serious PIS. The professional scanners can be devided in: drum scanners, at-bed scanners and slide-scanners. For the drum-scanner the at material to be scanned should be placed around a cylinder which rolls p t a scanning head. Resolution is highest (typically 600-700 dpi) compared with the other scanners. The at-bed scanner resembles a simple copier: the at material is to be placed on top of the scanner and it is scanned by a moving light beam or by tbe movement of the top part of the scanner that contains the material, depending on the type and manufacturer (typical resolutions range from 300 – 400 dpi). The slide-scanner is a small box that h a
slot for 35 mm slides. The slides are scanned at typical resolutions of 500-600 dpi. All the scanners are available for monochrome and full colour images (with di erent price tags). For scanning coloured images the scanners process the image three times and lter the re ected or transmitted light for the red, green and blue component respectively.
15
16 Jö en van den Berg, Hans Brandhorst rmd Peter van Huisstede
Di t camer
A copystand camera and video camera resemble the classical camera but deliver a di tal image instead of a fotograph. Resolution and dynamic range depend highly on the type of camera and Jen s. Technolo m ing dramatic advances in developing capturiog devices at chip Ievel that digitize the caught light in real time and at resolutions that will be clo to photo realistic quality in the mid nineties. Since the companies involved m at a consumer market, prices will be relatively low. A professional digital video camera delivers a re lution of 300 dpi with 24 bit colour information per pixel (16 million colours). Norm video c er and the att hed frame-grabbers (video signal di tizing boards) have r olutions that are around the 100 dpi and quite weil coincide with VGA and Super VGA resolutions. Typical colour qu ity is 256 (8 bit) to 32.000 di erent colours (15 bit) per pixel. When images are digitized from transparent originals ooe o special featur like a backlit box with di usion lter(s).
The nu ber of colours (or dynamic range) and the resolution that the input device can o er are the technic features most relevant when selecting the most appropriate type of capturing device.
2.2.2. Picture Processor Device
The graphics board in a (micro-)computer represents the picture proce or device.
e graphics boards or adapters appreciably in functionality, performance and price. Th bo ds are responsible for the resolution and colour palette of the image that is rep ed on the monitor. The market for graphics cards is rapidly evolving, due to the growing inter t of u rs in graphic applications. M y companies are tive in this m ket developing proprietary video proce ors and boards.
Standalone proce rs for capturing and processing images These days the graphics board in any (micro-)computer is equiped with a dedicated on·bo d (mostly proprietary) vid pr o rthatusu lyh itsownmemorybanks(1Mbormore)tostorethedi erent colour repr ntatious of the di t image. The output sign is commonly complying RGB standards which enabl a variety of monitors to be att hed. The primary function is to display an image that is stored digitally in the video memory of the board. The more extensive bo ds have dition hardware for functions panning, zooming and pixel proc n g. Boards can be equiped with capturing features: they enable the sampling of a video signal coming from a video camera having NTSC or PAL output. These boards di tize a video image in real time, capture and freeze the image on user command in the vid memory. Typic r olutions range from VGA 800 x 600 with 256 colours/pixel to 1024 x 1024 with 16 million colours/pixel for the higher ranked boards. The more phisticated graphics boards allow high refresh rates (60 · 72 Hz) of the outputted image d produce a cker free image.
Array proc r s for pixel by pixel proce ing The top rank video boards allow con· nection to dedicated array processors (e entially number crunching proce ors) for high s pixel proce og. They o -Jo the CPU for cxample with standard Jogic Opera­ tions on the video memory and boost performance on heavy image processing t ks. These boards are relatively expensive because they are much le � a consumer market article then graphics boards are.
Image Procusing and the (Art) Historical Discipline 17
Optimized ftware tools are peeded t o exploit the ultimate performance of a graphics board. Special device driver software comes with l boards to interface the board to the Operating system and graphics u r interface that is cho en.
2.2.3. Picture Output Device
n an image is digitized and usually alre y durlog the digitizing pr ess itself we feel the urgent need of looking at the digital copy of the ·origin ‚. Reproducing the digit image, altered or not by image proce ing techniques, is crucial. Many di erent techniqu have been developed to visually reproduce a digital image.
Colour graphics r ter display
All COmputers have at le t a (graphics) monitor attached and many u rs have a colour monitor that can function the (colour) graphics r ter display. The re lution and dyn c range of the display have to be in accordance with the attached video board. The dynamic range refers to the number of di erent colours (or shades of gray) that can be repr nted by any one pixel. We have to realize that colours are not consistent across display devices, so one monitor may give a morc realistic image then the other. The colour presentation highly depends on the phosphors used in the display screen, surrounding colours, ambient light, backlight etc. The refresh rate of the image is critical for a stable displayed image.
L r- and photoprinters
In general we are not satis ed with only a digitally visu ized image when we cannot capture that image in a non-volatile way. Devices have been developed to reproduce a hardcopy of a di t ly proce ed im e. First of l images can be printed using bitmap printers or plotters in the c of vectorized images. L er print rs reach a re lution of 300 · 400 dpi and therefore very weil capable to reproduce the image at the needed re lution. L er printers that can print colour im es have become av lable, though the colour r olution is not optim yet. The tcchnolo is coming om advanced colour copiers, which b e very quickly popular with counterfeiters and in due time we may sume that this technolo will be brought to per ction. The best results can be reached with photoprinters that combine photographic techniques and proce s with a high reso­ lution digitally displayed image.
2.2.4. Picture Compression Processor
An average image of a VGA reen (800 x 600) will take up 384 Kbyt of storage with 8 bit colour information per pixel and a 24 bit image will take about 1 Mbyte of storage. Any means to reduce this amount with inimal loss of information is welcomed. Image ze not only h consequences for storage but also for telecommunications. In the humanities eld image compression is a particularly important issue because of the need of high quality pictures.
Dedicated microproce or for compression and decompression of bit-images Compres­ sion is n essary to remedy the storage and tel communication problem. Many di erent algorithms and techniques have been devt’loped: in software and even with d dicated hard­ ware. The multimedia market h inspired chip-manufacturers to design speci chip sets that allow compre ion and decompression at extreme high speed. Lossless compression will typically give reductions of no more than 3:1 times, however reductions of 1 :50 with

18 Jö en van den Be Hans Brandhorst and Pcter t•an Huisstede
ptable l of information are fe ible due to special hardware. Compression boards e a lable at resonable prices which low the user to choose the reduction factor desired ven the lowed information lo . Within a couple of years special chips will be part of the main computer board to reduce storagP nPeds.
2.2.5. Picture Communication Interface
Transferring a single 1 Mbyte image across a 2400 bps communication line will cost appro mately 70 minutes. Even on a 64 KHz line the transfer time of 2 minutes is not a very attractive perspective if you want to browse a remote pictori datab e.
B band dat ommunication h dware
B band networks such token ring (twisted pair) and ethernet (coax cable) b ed networks are m t popular and weil spread in the academic domain covering wide arca networks. The throughput of these networks is heavily challenged when 1 Mbyte images transferred. These networks have typic transfer speed of 8 – 10 Mbps and low there re transfer at re onable speeds when all other network tra c is low. The transfer sp is lowered appreciably due to communication protocols and depends highly on the bu ering in the loc network card. Network cards are available at low prices and come speci drivers for the operating system. Installation and tuning is, however, work for tel ommunication specialists.
B band Multimedia LAN-connection
Bro band networks operate at higher speeds up to 300 Mhz and allow various signal urc such hi audio, video and real time speech signals to be transmitted simulta­ neously. They are therefore much more appropriate for transmitting high quality digital . B eband networks, however, are f more widespre . The construction of optic b networks (Fiber Distributed Data Interf e) will encourage the u of bro band networks in the near Cuture and will solve speed bottlenecks due to bulk vol­ ume datatr smission of high quality digital images.
2.2.6. Picture Storage Device
Storing digital im es demands high capacity storage devices. As we have shown be re a high qu ity digital image will e ily take up 1 Mbyte of storage, whether it be memoryordisksp e. lfoneh tostorea1000 images,notalargecollection bytheway, one should have acce to a device of at le t 1 Gigabyte.
M netic disk
It obvious that in an environment of desktop workstations (PC’s or something big­ ger)magnetic(hard-)diskswithgigabyte-cap ityarenostandardequipmentyet. However during the di tizing proc one needs ce to magnetic disks with high volume capacity. Av e harddisk Capacity doubles yParly and costs decre e yearly about 20reached its phy c limits, this will go on for an other 5 years and we will e our (new) computers be uiped with more and more h ddisk space. Gigabyte cap ity disks h ve become a ordable and cess times have b n improved to an avPrage value of 16 ms. With a high cap ity harddisk one needs also high capacity backup devices. Due to the combination ofm netic tape and digit interf ing technolo backup devices using video t pes (EX­ ABYTE) and audio tapes (DAT-r orders) have become available for background storage at high volumes (typic ly between 1 and 2.5 Gigabyte)

Image Processing and the (Art) istorical Discipline 19 Picture Input I Picture Editing
Pictiure Processing i�
IPicture Output Picture Encoding / Decoding
Picture Communication
Picture Storage
Network Access Pictorial DBMS Figure 2: nction uirements for Pictorial Information Systems
Optical media
The most promising development is with the optical media. Although the video­ disk did not become a consumer market article yet the CD-ROM for audio did very weil. Optical media come in many di erent disguises: analog or digital video-disk, CD-ROM and WORM (Write Once Right Mostly). A CD-ROM has a storage capacity of 600 – 800 Mbytes and is an attractive storage medium when the data have to be distributed in some quantity. The production costs of CD-ROMs have re hed a Ievel that is acceptable for many projects. The (present) major disadvantage of the CD-ROM is the high access times and low data transfer rates compared to magnetic disks. However, the ‚home entertain‘ market pushes technolo in the right direction with new Standards and techniques (D , CD-I, CDROM-XA etc) that combine data compression features (with dedicated build in hardware) and the optic medium. If there is one bene cial pect about the present ‚multimedia hype‘, it will bring us quick access to volume digital data at low cost.

20 Jö en van den Be , I/ans Brandhorst and Peter 1•an 1/uisstede 2.3. nctional R uirements for Pictorial Information Systems
2.3.1. Picture Input
Capturing or digitizing of images is the rst elementary stage in the imaging process that treats the relevant material in such a way that the computcr can store and process the digit imag afterwards. With all objects there is the particular concern about not harming the object during this process. Sources of harm include light and heat during sub­ stanti time coming from the digitizer or capturing uuit during the actual input process. The vulnerability of the object and the time needed during the inputting f e have to be b anced carefully. Depending on the input device the object might be exposed to intense light and heat for at le t several inutes. Altbough the suppliers market offers a growing whe th of solutions for very di erent budgets, to make the right choice is not a matter of money only. Choosing the appropriate input device is a cruci one since it is the quality bottleneck, afterall it is not to be expected very likely that a whole collection of material is to be digitized again soon. Digitizing should be done in a resolution and format that will anticipate needs weil in the future.
Di tizing im es from paper or lm
Many f tors come into play in selecting a ßatbed- or drum- anner such the type of materi , the ultimate display resolution and colour range of the materi (grayscale or full colour). Economic considerations like equipment costs and scanning time play a major role when thousands of images have to be inputted.
Capturing ofimages from objects
When considering p i ble Sources of harm the digitizing process is usually le harm­ than the proc of tu moving the object from its storage location to the capturing ea: a copystand or video camera. Three-dimensional or very !arge two-dimensional obj an extra proble since the digital image of the whole object h to be recon­ siruct om di erent sm ler images that have only covered parts of the original.
Not only the scanning or capturing speed itself is important, the throughput of the whole anning process is more likely to represent the cruci time f tor. Handling of the imag , linking the physical images to the di tal images, qu ity tests, storing on disk of the digitized image in different resolutions for di erent purposes, etc highly in uence the tot throughput time. When thousands of images have to be digitized this can become a critic factor that h to be dealed with in the project management.
2.3.2. Picture Editing
Once the image is di tally available it usually needs some further preprocessing before it can be ded to the pictorial datab e or pr es d in later research stages.
iting existing image
This proce is called picture editing: colours are adjusted, corners or parts that are not relevant can be cut o , textual information can be added to the image to ensure unique Iabels or to mark are of special interest etc. Special graphics editing software to enable th activities is available with all different computer platforms. All graphics u r interfaces have these tools available as a standard option and they o er in general a Iot more functionality then only editing the image such all kinds of paint tools. A known low Ievel tool is for example ‚Paintbrush‘ from Microsoft, which is standard in

Image Processing and the (Art) Historical iscipline 21
Microso Windows environment . More sophisticated tools like ‚Photo styler‘ (Aidus) and ‚P nt shop‘ (Adobe) allow professional editing and have a seamless link to the digitizing or capturing activities.
Creating new images
The need for sophisticated graphics editing and paint tools stimulated the develop­ ment of profession packages known ‚Paintboxes‘ that combine the m t sopisticated video-chips and software to support the creative design process. A Paintbox o ers photo­ re istic quality and many software tools to create, merge or edit high resolution new or existing images. The main use is for profe ional photo-labs a retouching tool and with advertising companies. Artists have discovered this medium and generate new graphic art, that h found its way to art-ga!eries and museums.
2.S.S. Picture Processing
Picture processing is the collective term for many di erent Operations that a digital image can undergo. The choice of Operation depends entirely on the e ect or result that is aimed for. We will not exhaustively enumerate all di erent operations that have been developed, but instead shortly refer to a limited number to show the potentials.
P ture transformation
Due to the fact that the image is stored digitally this digital information can be m i ed by mathematical operations. Transformations allow us to uniformly alter the entire picture. Typic operations are geometric transformations of scale, translation and rotation. These operations low us to zoom in and out on parts of the. image, to copy or translate parts of the image to an other locality within the image itself and to ch ge the orientation of the area of interest.
Enhancement of a picture
Many operations can be performed to enhance a digital picture. Operations on groups of pixels can sharpen or smooth the image depending the result one is after. For example the subtr tion of a weighted average picture om the origin usually will achieve a deblurring e ect. Only local averaging an image can reduce noise and h a smoothing e t. Operations with extreme e ects can show compositional features of the image.
Segmentalion techniques
Picture segmentation is the technique of decomposing a picture into meaningful parts to separate objects from the background and to distinguish among objects. Segmentation techniques are a rst step in pattern recognition because the result is logically linked groups of pixels that discern object from the background.
Edge or boundary detection
Many goritms have been developed that support edge-detection within an image. For example by using a mathematic a!gorithm called Laplacian operator lines can be detected and an image can be deblurred. The nice property of this operator is that it yields the same results regardless of the orientation of the picture. The e ect is that it rst gets an averaged image that is later subtracted from the original. The combined e ect is to sh pen the boundary lines of an image.
22 Jö en van den Be , Hans Brandhorst and Peter van Huisstede
Patte r ognition
With the above techniques depict objects can be isolated, features extr ted and with a pattern cl ier ident i ed. This works weil with optic character recognition (OCR). Although pattem r nition h much attention in the cld of image processing and the r ults e encouraging, pattem recognition will not play a major role when envisage the complex visual art historical re urces. lt will take a loug time before pattern recognition c de with the complex images and what is more with the complex ques­ tions r hers come up with. There is quite a !arge di erence in complexity between nding squar and circl in a thousand images or retrieving repre ntations of particular typ of the Virgin and Child.
2.3.4. PictureOutput
One should re ize that the displayed image or the hardcopy of it is always a surrogate image that serv retriev or image proces ng purposes but hardly ever repl es de ni­ tively the original. The potential use of the digitized images dictat the resolution and colour at which the image is to be displayed. With every pictorial information system a mix is needed of di erent quality (re lution and dynamic colour range) output devices to anticipate the di erent u of the information system.
Displaying/P nting a picture
Although im may be digitized at very high resolutions, they do not have to be dis­ played or reproduced at the ve e resolution or with the very same colour information they were digitiz . In many c es a simple surrogate image {of low resolution) will do perfectly in early stage of inspection. When browsing through a pictori datab e surrogatepictur enoughdetailto lowrecognitionoffeaturesofinter t. After selection of a particul t of im es in this way, the limited set of images of interest can be display at higher r lutions for further inspection. Time can be saved by speeding the retriev p without potenti lo of information. Hard-copies on I rprinters rve the very me way in m t c the purpe of documenting the images of interest d do ne in the later publication to show the issue.
Zooming d Panning
Many vid bo ds zooming and panning of the image at high speed. If these featur e not ble at hardware {chip} Ievel, they can oevertheless be simulated with ftw e routines, but not at high speed because of the extra computiog overhe . Th features low to inspect images at di erent Ievels of detail and allow high reso­ lution di t imag to be displayed at a appreciable lower resolution without loosing the overview.
2.3.5. Pi ore EncodingjDecoding
Although storage is a major issue with pictorial information systems, the use of com­ pr i on and decompr i on techniques should be transparent possible to the ultimate user.
mpr o n and d ompr i on of image.,
During the d ign stage of the PIS, decisions have to be made about compression methods in relation to the information loss that accompanies compre ion of data. De­ compr sion is time consuming and h at le t to equal out the g n in overall storage
.
Image P cessing and the (. rt} Historical Discip/ine 23
capacity at the cost of (some) information loss. Taken into account the earlier remarks about surrogate images the information loss will usually not be the major issue. The speed reduction during the image display f e a result of the necessary decompression of the image has to be t en into consideration becau the user is confronted with it every time a (part of an) image is displayed and that can be very annoying. On the other hand when the images are stored on a CD-ROM with low transfer rates the decre e in le size of the compressed image speeds up the transfer rate of the image. With a hardware decompres­ sion bo d the time gained during transfer of the compres d image is higher then the time consumed with decompression! When images are available on a b eband network server compre ion usually is an attractive solution for gaining transfer speed when hardware decompression is availble on the clients workstation.
2.3.6. Picture Communication
Distribution of the PIS on CD-ROM is a serious alternative to picture communication although speci c network functions can never be replaced by a small silver platter it might very weil be a cost effective alternative when CD-ROM players become cheap as oppy disk drives.
Transmi ion of images to workstations or computers
The issue of image size has been raised earlier and becomes critic when images have to be transmitted over datacommunicatioh lines. The concept of a client-server architecture is attractive when a !arge amount of images and expensive output devices (such plotters, I er-and photo-printers) have to be accessed by many users. Instead of incre ing the clients workstation power the PIS can be reached a network function and data and images are transferred to the dient and displayed on the clients workstation. Resources (disks and output devices) can be shared better this way and (dynamically) connected to the clients workstations depending on the needs. The type of network and its topolo are critical with respect to transfer speed and reliability.
2.3.7. Network Access
When connected to a network one expects some b ic functions to be present such le transfer, remote login, Iist servers, e-mail etc. As for the user these functions have to be transparent possible, whether the user accesses his local disk or the network disk he should not notice the di erence.
Network functions
One of the advantages of networks is the sharing of resourc . Therefore one of the m t appropriate functions that serves PISs is the ce to a CD-ROM jukebox. A CD­ ROM jukebox is a multidisk CD-ROM player that logically can combine physically di erent CD-ROM disks to one !arge disk. It can be loaded with many disks typically 8 to 16. Since pictorial datab es demand !arge storage capacity access to a rather expensive device as a CD-ROM jukebox seems a logical network function. Facilities for transferring image les to Special output devices such as Iaser- and photo-printers are considered standard functions.
One can only speculate on the impact of the major advance that will be made when keepers of digit visual resources are connected to the academic networks and the remote access of visual materi will become av lable on international scale.
24 Jö en van den Be Hans Brandhorst and Peter uan Huisstede
2.3.8. Picture Storage
The ! ge storage volume that PISs require does not have to be repeated here, it has been emph ized in the above paragraphs.
A on gi n stor e medium
Acc to the storage device that holds the PIS’s data should not be different from c i ng a loc disk. Device drivers for CD-ROM players are available that allow this tr p ent c . H ever, the d ign of the PIS heavily depends on the use of a partic­ ul stor e device such a CD-ROM pl er which impo s speci requirements. Since CD-ROM is a read only and rather slow device the implementation of the datamodel of the P should count for that. Deliberate redundancy, extra indices, partitioning and dyoamic bu ering on loc magnetic disk or in memory of heavily used data l help to get ound the CD-ROM constraints.
2.3.9. Pictorial DBMS: Logical storage and retrieval
When d igning pictori datab s two problems have to be solved: manipulation of a !arge nu ber of im es and manipulation of images of great complexity. Tradition ly, r chers wor ng in the eld of image proce ing have concentrated on working with a few im . However the kind of applications we are confronted with today require that a system is capable of handling a !arge number of complex images. From this it follows that r ch must concentrate on new techniques suited for e cient and ßexible retrieval of information, texts and images, om !arge pictori datab es.
The d ign of a pictori datab e must integrate tabular data (text), graphical data (v tors) d image data (bitmaps). One should realize that there is a di erence between the image and the physic image. The logical image can be regarded a model of the re im e. It is de ned a (hierarchic ly) structured collection of picture objects aod t tu information ( m tically) de ribing it. The logical image cao be stored re! ion tabl in a relation datab management system and maoipulated u ng a query language. Inquiri concerning the attributes of picture objects can so be haodled by this datab maoagement system. Once a Jogic picture h been identi ed useful, that is a er retrieval using the av lable information, the corresponding physical image cao be generated on the output device by retrieving it from memory. In fact the presentation the physic image is the e y part. lt is b ed on a relation between the stored image d the textu information about this number, or, to make things a bit more tangible, a simple number refers from stored image to textu information aod vice versa. Software lutions may di er cording to the actu implementation of the above mentioned prin­ ciple. Sometimes ftware (for example PielurePower by PictureW e Inc.) separates the storage of bitmap im es, using an image store, !rom the database management system. for quick and e y access by a speci interface for PC-b ed datab e man­ ement systems like dB . Other datab e management systems use attributes called BLOBs (Binary Large OBjects). These are particularly u ful for storing the bitmaps. other information will be st red in the conventional datab records and fields. As s d before, storing images in a datab management·system is only the simple part of the story, retrieving them in an e cient way is mething completely di erent. The next chapter will de with that proble extensive!y.
Image Processing and the (Art) Historical Discip/ine 25
2.4. Conclusions
The eld of image processing is strongly technology driven and those who not familiar with it are easily impressed by the latest technological achievements. The tech­ nolo improving so fast that any project that starts with a certain Ievel of equipment outdated within a couple of years. This means that every project that h the pretention to last for many years h to develop a special policy to ensure investments in equipment and per nnel, since a dependency on (specialized) technology can be fatal. Only projects that are organized in such a way that switching from one technology to an other are p sible, can keep up with Cuture technological developments.
3. Images, Texts and Computers: Describing Images
3.1. Retrieval of Images: The Basic Need to Provide Textual Keys to Retrieve Images
In the preface of this article we stated that it is hard to see how the electronic pub­ lication of digitized images without additional information, can serve any (art) historlcal research interest. In section 1.3 we concluded that art history should leave its anthological methodology and abandon aesthetic criteria when selecting material for research.
So, on the one hand we would like history – or whatever we would c l this discipline – to put ‚all‘ the visu material of the past at our disposal. On the other hand, we want it to preprocess – describe – it; before we begin to study it. Th e two statements add up to the following paradoxical question: „What is left to be studied in material that h already been described in det l?“ The answer is simple: everything!
To explain this, we must return to what was said before about the di erent stag of the organization of information: data, b ic information, and extended information.
Simply digitizing a large amount of images without any additional information no solution. But even distributing digitized images together with b ic information about them – the second Ievel – is no real Solution. Suppose we de ne the b ic, ctual information about objects consisting of:
• name of collection or institution.
• inventory number.
• indication of object type.
• keywords related to content.
• title.
• description of object.
• maker.
• date.
• material.
• size.
• frame.
• proven ance.3
3 This enumeration is b d upon: Jeanne Hogenboom, Basisregistratie voor co ecties voo ­ erpen en bee1dmateriaa1, Rotterdam, 1988.

26 Jö en van den Be , Hans Brandhorst and Peler 11an uisstede
Such descriptions, without which one can not manage a collection of visual material at all, allow us to group images according to tradition art historic categones.4 This approach, however, will not produce a pietonal information system that may adequately serve r earch within the realm of cultural history. It is highly likely tbat the questions that will be p ed in such acontext, will tran end the information affered in the aforementioned documentation categon 5
Let us Iook at the following example: A maritime historian, interested in 17th cen­ tury scenes of harbour activities, wants to whether Dutch Iand apes of that period cont n information on tbis topic. He wants to search on „harbour tivities“ in 17th cen­ tury Dutch land apes, a Straightforward question by any standard. Consulting the b ic information – the second Ievel – stored in the image databank of an imaginary documen­ tation institute, chanc are that our histonan would have to transform bis question into metbing vaguer. He would have to make a detour, for instance, by king for Dutch 17th century I d ape painters, whom he knows to have depicted river and co tal views. Or he may try to nd all depictions ofcities and villages he knows to be or to have been at the co t or on a river. In other words: he would probably b e bis retrieval on properties of imag – names of artists or topographical names – which are not bis primary interest!
lt goes without saying that „harbour tivities“ could have been used a keyword to denote the content of some of the images in our imaginary databank. The question here, however, is not whether it is likely or not that the term „harbour tivities“ is used a descnptive keyword on the second Ievel. We could just e ily have cited the example of an agronomist interested in the di erent types ofvegetables depicted in still life paintings… The qu tion is wbether the second Ievel ofdescribing images, we have de ned it, will be of when de ing with by de nition unpredictable questions of researchers from various di iplines.
The answer is of cou e: and no. Yes, because the factual information can be u d to delimit questions concemni g the contents of images: landscape, Dutch, 17th cen­ tury. No, this type of information does not aim at providing the pr ise, detailed and consistent descnptions of the contents of the images needed in the kind of pietonal information systems we are discussing here.
From the section above we can draw me conclusions:
• It is imp sible to predict the questions re archers will k from pictorial infor­ mation systems. It is only by supplying extended information that we can begin to swer them.
• F tual information can be supplied for !arge and heterogeneaus collections of images, although it must be stre ed that this is a complex t k in its own nght. Of far greater
4 Cf. section 2 where w that the computer can to group images accor ng to formal qu ities. Here the issue is not much whether these qu tions are tr tional, but rather whether these questions Iead to hlstorical research (another c e of preprocessing).
5 Cf. R. Stenvert, Constructi the P t: Computer-A isted Architectur -Historic R , etc. Utrecht, 1991, p. 78: „Only when trying to po speci c content-related questions will the real bettleneck of each information system emerge: the ‚depth‘ – or extent – and the consistency of the secondary [i.e. our ‚extended‘] information.“

Image Processing and the (Art) Historical Discipline 27 complexity, however, is the t k of providing extended information of the de red Ievel
of detail and consistency.
·
• The problems involved in maintaining that Ievel will decrease when cho to describe relatively homogeneaus material in relatively small research projects6
• It hardly needs to be emphasized that the extended information should be added to the b ic information. In pictorial information systems we obviously ne both.
Two methodological points remain to be raised:
Interpreting d presenting the source
First of all we should make it unambiguously clear that we are deallog with interpre­ tations on each of the three Ievels of information we have distinguished.
• Digitized reproductions of objects are not the same as the objects themselves and in this sense they can be said to be interpretations of the objects. Repr ucing an object, and certainly its digitization, involves a number of thechnical choices, can be inferred from section 2. But even when we Iook at objects them lves, we cannot avoid interpreting them. Any intellectual „appropriation“ of an object involves instantaneously „considering it under me verb description or speci cation.“7
• B ic information about objects, naturally, i s based u p o n interpretation. This may seem to be less obvious for information about their size or materi than about the artist who made them. It neverthele holds on both a practical and theoretical level.
• Precise, detailed and consistent de riptions of the contents of imag are interpreta­ tions of a highly integrated and organized nature.
We moreover assert that the need to embed our interpretations in gener historic research, incre es as we move from the rst Ievel, data, to the third Ievel, extended information. This sertion is consistent with the one that the meaning of im e li outside the image itself. other words, the meaning of an image can only be studied by carefully holding it against an appropriate cultural context from the p t, a context of which (other) images are an important element.
The third Ievel of information provided by a pictorial information system thus be characterized as having a high interpretative „density “ . We create extend information by exercizing „brute force“ on material from the p t. Therefore we have to take into consideration that some users may feel that the data that Iead to a certain interpreta­ tion are hidden by the layers of interpretation themselves. We can call this the iceberg phenomenon: a part of the data oats below the surface and remains invisible.
We can illustrate this with an example from our own research. When we were working on a prototype pictorial information system on Dutch 17th century printer’s devic – the
6 Tbis conclusion may serve to put the theoretical and practical e orts into perspective of projects tbat do indeed try to provide both basic and extended information for heterogeneaus collections. The intellectual achievement of documentation institutes, faced with this t k, often underestimated, by organizations tbat fund them well as by tbe scholars that ma of tbeir information.
7 Cf. Michael Baxandall, Patterns of Intention. On the Historkai Explanation of Pictur . New Haven/London, 1985.

28 Jörgen van den Berg, Hans Brandhorst and Peter van Haissiede
small logo’s witb which these craftsmen identi ed their products – our source consisted of pbotocopied title pages of books printed in the Netherlands during the 17th century. In this period there w no strict distinction between priuters and booksellers. On the title pag botb printer and bookseller e sometimes mentioned. Typically, the device belongs to only one of them. Working with a !arge amount of material we could often sign the device to either tbe printer or the book ller by extrapolating from other occurrences of tbe me device.
So, in tbis c e, tbe u r of the pictorial information system is confronted witb our interpretation, not witb tbe data we found them on the title pages. When the system w publisb (on CD-ROM) it became clear, however, that some researchers were interested in pects of tbe information that we bad ltered out. They would like to know, for example, wbich printers and/or book llers bad collaborated with whom during this period. The info ation to answer tbat question is in our urce. We had been aware of that, but we h decided not to make it explicitly available in the information system.
Obviously, proc i ng a urce always implies „hiding“ aspects of its data by ignoring or lterlog them. Tbis bolds for any source. No matter how faithfully we reproduce a cbarter – its integral text togetber witb scanned images of l of its pages – we do not repl tbe obj t. We can not avoid to „bide“, for instance, cert n signi cant codicolog­ ic cbar teristics. We hope to have made it clear above that if a urce consists – in wbole or in part – of images, simply p sing on all of the images to the user, is no remedy. On the contr y, the more im es are included, the more pre ing the need is to provide patbways to them.
When d igning pieton infonnation systems, one should have a clear view on what info ation should be presented in what way and how the decisions taken in the process can be made transparent po ible. We have called this the conceptual issue. It is a di cult i ue, because one needs to be aware of how other r earchers want to use the materi . me c es that qu tion may evolve into a r earch project of its own.
The centr proble could be de ned : „How does one ensure, given the iceberg phenomenon de ribed above, that one’s research is of maximum use to others?“
There no simple lution to this conceptu problem. In the example cited above we d ided to backtrack. We are going over the source once ag n, surveying all the nam of persans appearing on th title pages. These notes will be incorporated in the forthcoming i ue of the program a kind of worksheets. If we have attributed a printer’s device to a cert n printer, the u r can consult the worksheets to see with whom the printer collaborated using that particular printer’s device. If we can not attribute the printer’s device to one persan mentioned on the title page, we will – by default – give all n of persans mentioned.
It might very well be, however, that researchers are not so much interested in coopera· tion between printers and booksellers, but want to know how these printers and booksellers have c led themselv , what they have said about their shopsigns and the name of their shop, bow they have de ribed their locations and addres s, etc. In short: on we have lved the retrieval b ed on question N, question N+l will be posed.
Anyone describing !arge amounts of visual materi of the p t, concentrating on sourc that combine texts and images, will have to analy that source carefully and
Image Processing and the (A.rt) Historical Discipline
think carefully how to present th� described material. One could imagine that steps are taken to gather data that perhaps will not be presented in the pictorial information system itself, but still can be of use to the researcher posing a N+l question. Data could then be made available in an alternative way, for example in pl n ASCII8.
The status ofa description: the Münchhausen paradox.
The second, even more important issue is the following: what is the status of the iconographical descriptions of images? Are they historical explanations? This is an ex­ tremely important issue, because to make precise, detailed and consistent de riptions, we rely on controlled vocabularies, authority lists and thesauri. These tools have to be a\ ­ able, in whole or part, before the researcher h described the images. How can this be? How can the researcher – to put it sharply – have the terminology available to interpret an image before he knows what it means?
The di culty lies in the exact meaning of the word interpretation. Interpretation in a traditional art historical sense does in fact seem to be de ned the equivalent of historic explanation. This can of course only be the c e if the researcher holds the epistemological view that images contain historical information as weil the keys to decipher this information.
If, instead, we ert that the keys to decipher the historical information of images of the p t lie outside the images, then the status of interpretations on our third Ievel should be seen contemporary, but informed – i.e. based on already proc e d material – descriptions of images. By applying descriptors to images of the p t, we not much explain the image as an element of the past {that is what should be studied), but pre­ process the image in order to get an object that can be studied as an element of the p t in the rst place. The status of our interpretative descriptions is thus by de nition tentative and temporary. By collecting as many of them as possible, however, they can gr u ly begin to pull each other out of the quicksand.
This idea of pre-processing images to get an object for historical research, resulting in labelling descriptions of images as conte porary state ents, seems to be v id so for the image anipulation techniques entioned in section 2. In section 3.3 we will have a Iook at some of the techniques used for pre-processing images with the help of texts.
Summarizing
From an episte ological point of view we emphasized the distance between us, the researchers, and the ele ents fro the past we study, the i ages. We denied the idea that i ages contained historical meaning as weil as the keys to deciper this meaning. Furthermore we argued that in order to study the eaning of i ages we have to d cribe them rst. We stated that these descriptions do not constitute historical explanations of
8 The largest amount of data is by de nition available at the time of data entry, i.e. in the production environment. The design of the query environ ent usually implies tering data. For instance: divergencies the spelling of proper namcs in the source, may be „corrected“ . D igning the production environment means taking decisions about what data will be entered. Moreover, in this environment the data either are in plain ASCII les or they can easily be downloaded into such les. Whether they can indeed be made available in this form, will depend to a !arge extent on the researchers‘ willingness to exchange data in this way…
29
Jö en van den Berg, Hans B ndhorst and Peter van Huisstede
the meaning of th e images, but are an activity we Iabelied as pre-processing. We did not di u the various ways or methods of pre-processing images using di erent (art) theories. The m n idea simply is that the (art) historian clearly indicates which parts or elements of image make it possible to study the image an hsi torical phenomenon. Whether the { ) historian works with iconographic contents or formal pects of images or with a combination of the two is not the main issue here.
we have seen in ction 2, the computer o ers us a Iot of tools to manipulate im . But even with these tools, rese ch of both the form and the iconographic apect of im need text to enable others to use the results of this research. In the following tions we will highlight two pects of ding texts to im es:
• The actual proce of linking various types of information to images. Emph is will be on the production ph e.
• T hniques and tools to help us control the consistency of the information we link to imagPS. Special attention will be p d to iconographical descriptions.
3.2. Creating Access to Images: An Iterative Process
Wordproce ors and editors
Many projects focus on individual images. They collect, or already have information a ut e h ofthe images. The simplest way, then, to structure the information, is to create one record for every image, in which b ic and extended information are combined. The d nt elements of the information may be identi ed by tags or Iabels.
A le that is thus structured, can be quite e ily generated with an ASCII editor or wordpr r . Such a structured text le then combines features of a text and a datab e le. read sequentially, and although it can not make use of records and elds itself, it converted into a genuine datab e f e, if the text is properly marked and one h a DBMS that is able to import ASCII l . Evidently, long the le is kept as a t e, the editing and search facilities of one’s editor are av lable. Data entry can be done mply with a wordproce or.
rather low Ievel appro h h some benefits over directly preparing or making records with the help of a DBMS. First of l one can start a project without the high initi costs of a DBMS. Editors or word processors are relatively cheap, if not ready av lable within institutions. Furthermore preparing data with the help of an editor ves one time to experiment with the data, begin to think about ways to structure them and about the environment needed to proce the data further. So, ideally speaking, the result would a better choice of DBMS because of a better insight in the data the DBMS h to man e.
Th strate so h some drawbacks. First of all, the number of records grows, it will become barder to check the quality and consistency of the descriptions with the help of an editor. The critical le size that can still be managed depends of course on the speci cations of one’s editor. Especially important are its speed and its search facilities. Th e by de nition string matehing facilities, but they can be more or less sophisticated. Some restricted to exact matehing (think of the search possibilities of wordprocessors), others support the use of wild cards and regular expre ions.
Image Processing and the (Art) Historical Discipline 31
Secondly, it is di cult to work with a data model that encompasses multiple les, when the tool you use is an editor. Because a text le is read linearly, one tends to keep information physically close together. The iconographical description typically follows information on maker, present location, provenance, description, etc.; and all are dustered around e.g. the inventory number that uniquely identi es the image. The drawback is clear: this causes redundancy. To give one example: with the name of a painter, some biographical data are often included, like date and place of birth and death. Information like that is not unique to one image. Yet, in a single text le, the only way to link it to the unique element – the image – is to repeat it each time it is applicable. A solution might be to create a second text le where information about artists is collected. A number identifying every single artist could then be used in the rst le. Clearly, this strate can not be maintained very long with complex data.
A third drawback is that using a (single) text le is di cult if one works in a team.
When a group of iconographers describes images, the use of a database management system is a nece ity to enforce uniformity and consistency of the information.
Coming up with the right solution for a particular project will be b ed on trial and error, or it is often called: one will be engaged in an iterative process. In order to think of a data model, one h to have data. In order to acquire data, one has to have a v ue idea of a data model. The important thing is to acknowledge that one must be able to proceed by trial and error, that is to change software or manipulate the data during certain ph es of a project. lt needs no further explanation that correcting errors or changing cour is e ier if the di erent data elements are properly identi ed and tagged.
Te b e programs
There are software products around that make it possible to process the data with the
help of a word processor or editor and overcome the problems mentioned in the rst two drawbacks. These programs, “ kSam“ I urdealist“‚ „Zylndex“ to name just a few, allow the u r very e y import and export of ASCII l . They can, in various degre , handle data structures such elds and records, which they combine with – sometimes very elaborate – text search facilities. They can act a valuable and more or le inexpensive link between editor f es and a production datab c, in particular because they Iet us sort and check the contents of our les, i.e. do a form of data control.
The link will prove to be truly inexpensive when the use of such a program gives the user an opportunity to formulate functional requirements for the production database to be used to handle the data. To give some examples: proper handling of alpha numerical data, variable eld length, repeating elds, c-module to link programs in object format written in C, etc.
Hypertext
It is very di cult to give an exact de nition of the popular term „hypertext“. Its meaning depends entirely on the context in which it is used. Yet, the very fact that it is often referred to the ultimate retrieval tool provokes some comments here.
„Hypertext“ can refer to the original idea of Ted Nelson’s Xanadu project about a digit ly available universe of information. In some other cases „hypertext“ denotes applications that consist of the generalization of text-only documents in which images,
32 J en van den Be , Hans B ndhorst and Peter van Huisstede
moving imag d sound may appear whenever and wherever it seems appropriate (Lucky (1989)). And metimes the word „hypertext“ merely denotes the technique of a network of relations pointing to e h other.
At pr nt, the word is most often u d in the second aod third sense: hypertext applications (or multimedia documents) and the hypertext technique (e.g. interactive help systems of computer programs) .
Ofcourse the hypertext technique is part of a hypertext application. Concerning this techoique a distinction is m e between:
• obj tive lin : for ex ple a link from a table of contents to a particular area of a document; and
• subjective links: editorial links that consist of cross references between relevant ma­ teri or of comments on materi .
Although objective or more or less formal links will form the backhone of a hypertext
application, the subjective or editorial links make an application attractive.
Thr remarks should be made about the hypertext technique and hypertext applica­ tions:
1) Hypertext is a sound and v uable information management technique.
2} To this date the best hypertext applci ations are found in the are of interactive help systems, document man ement systems d teaching systems. The bene ts of hypertext applications in these are are clear: they low the user to follow b own rout through the materi according to bis p ticular interests (subjective linearity).
3) There e me drawb ks:
a) the c led subjective links d value to a hypertext application, tben these applications need a considerable amount of links not to be trivial and th e links have to be provided by informed sta . In short: building hypertext applications is a time consuming and c tly a air.
b) is one thing to give of hypertext applications the possibility of the path of subjective line ity, but then one must provide them with the possibilities of backtracking (where did I come om?) and overview (where in hyperspace I?).
P uction datab
A datab or a datab e man ement system – DBMS it is often called – is u to manage information (data). This information needs to be represented in a well structured form: the datab e. A datab e consists of les, les consist of records, records con st of data elements or elds they are often called, elds consist of entries. adi­ tion ly three typ of DBMS’s are distinguished: the relational model, the hier cbical m el and the network model. With all the Iiterature on DBMS’s around, we will not d ribe th e models here. What is important, though, is that the complexity of the data (the complexity of information in the (art) historical disciplines is not of tbe s e order the information of, let’s say a dentist) incorporated in the datab e often requires creati lutions, re : hybrid systems, li a datab with a thesaurus attached to it or a datab e that is a combination of the relation and t·he hierarchical model.
Image Procesring and the (Art) Historical Discipline
3.3. Aspects o f Vocabulary Control
In various ways basic information and descriptive text can be added to imag to cre e retrieval possibilities. Scholars generally agree about the need to use controlled v abu­ laries in the area of b ic information. There is less consensus in the area of iconograpbic description. Moreover, a tendency can be detected to consider some of the metb s of creating iconographic infonnation – e.g. systematic cl i cation and d ribing im in ee text – to be mutu ly exclusive. We will gue that they are not. Tb c in many c es be used complementary tecbniques.
Control mechanisms and tools
Wordprocessors offer us nearly unlimited freedom to input information in y form. And although database management systems generally make sure that o y cert n formal rules – no text in a numeric eld – they do not restrain our eedom conce g the contents of the entries. The e e with which computers allow us to rt and arch ! quantities of data, however, have made us acutely aware of the necessity to control tb fr dom. The advantages of selecting a preferred variant name for a particular tist, city or object are evident, even if there can be cause for disagreement on what tbat p iewar preferred variant should be. In general, the importance of vocabulary contr h dly be exaggerated.9
Input control can be done in several ways. Here we don’t have to di u them, but it is useful to mention two principles. The rst is to build one’s own Iist of preferred while inputting the data. The list, which grows one progresses, can tben to check new entries and to copy accepted ones. The second one to build on e sting source of terminolo . The existence of these ternative principles do not to m e an ab lute choice between creating an authority Iist or confo ing one. Q often nd a mixture of both: a particular standard is cbo n and then diverg om whenever it judg nec .
Controlled vocabulari may of course include non-preferred terms t , d refer them for the preferred ones. Elaborate systems cross-link terms that e tic ly related. These relationships may also include hierarchical ones. Eventu ly v abula systems can become real thesauri.
Of course, vocabulary control did not start with computerization. It h important element of documentation. The computer, however, p unprec ented f ilities to stimulate or even enforce uniformity of the data enter . Some standard so ware packages support the creation and maintenance of authority l . A few authori l have recently been published in an electronic form, some uf them rt of a ! er system, some of them independent instruments.10 ldeally you consult th and copy from them, without leaving your own application, e.g. your DBMS.
g In the eld of art history important work is being done by various institutio . To name a few: the Witt Library (Courtauld Institute, Londoa), the Bildarchiv Foto M b (M b a/d Lahn), the Inventaire G neral (Paris), tbe Index ofChristian Art (Princeton), the Ge Art Histo formation Program (AHIP, Santa Monica) and the lstit uto Centrale r e Ja cument ione (Rome).
10 AHIP, for example, h publisbed its Union Lists of Artists‘ Names and tbe –
34 Jö en van den Be , Hans Brandhorst and Peter van Huisstede
Status ofauthorities
Controlled vocabularies, dictionaries, and thesauri offer us names and terms which we u in creating b ic and extended information about images, i.e. in creating a subject for historic study. We are aware of the that th authorities themselves are b ed on r arch and thus not really „pre-fabricat “ at l. The terms they propose will be accept until they f si ed d everyone who u s them is – or should be – aware of their provisional nature. In a sen , every time „pre-fabricated“ terms are used to make a new image ces ble, they are t ted ag nst new data.
3.3.1. The Vocab y or Iconography
We now turn to the problems we encounter when we create and try to control the systematic and detailed iconographic acce to images. In analyzing these problems we concentrate on two closely related questions:
• How c m e the con stency of our d criptions?
• How c we optimize the retrieval of iconographic information?
F ussing on th e questions implies that our interest is of a practical rather than a philo phic nature.
Co ency
A historians seldom tr ned in the systematic de ription of !arge amounts of imag . general they focus on the individu image. Yet, the idea that it is not only u ful but po ble to produce !arge quantities of iconographic information h gained more and more cept ce. To me extent th is due to the acqu ntance m y schol s have recently m e with the per n computer. These machines proce , store, retrieve d ind text with eat e . Because of that, the creation of iconographic indices h b ome a much lighter t k. We may even cape making one if we have an electronic ver­ sion of the d riptions at our disp al. The very e e with which text can be manipulated d , h ever, tends to blur some fundament i u .
Undoubt ly, one of the m t important of th i u is consistency. It is es ntial
to the qu ity of the information in iconographic d criptions that:
• a single Ievel ofdet l is m ntained
• a det l mention once, is mentioned every time it is observed afterwards
• the same visu phenomenon is d ribed with the same term every time it is observed .
If the de riptions f l to meet the rst two conditions, the information is unreliable by de nition, and not be made consistent without going back to the source. lf the terminolo varies while the visu elements d cribed do not change, or the same term is to d ribe di erent visu elements, creating consistency c become extremely omplex, because links have to be tablished between the di erent terms.
The importance of consistency – and the di culty of enforcing it – incre es dra­ m ic ly ü:
• the iconographic de riptions in a single project are made by a team of holars
chitecture Theaaurua. Autbority f are available with tbe HIDA/MIDAS DBMS, developed by Bild iv Foto M b g d StarText GMBH.

Image Processing and the (Art) Historica/ Discipline • the iconographic information that is gathered, or rather created, h to made
acce ible to others.
Control procedure
To determine whether we have remained faithful to a once chosen Ievel of det l d to whether a particular detail h been mentioned every time it occurs, we have but one option. After describing a certain number of images, we have to go over the material a second time, and check if every occurrence of a particular phenomenon h been n ic . It is only by creating a complete index of our descriptors and systematic ly going b from this index to the images that we can estimate just how consistent our d criptions are.
Whatever our analysis of the relationship between descriptors and visu phenomena will tell us about the consistency of our selection of details, it will surely re th we have used di erent words to describe the same visual phenomenon. Rever ly, it show that have described di erent visu elements with the same word. Di erent we u for the e element may be synonyms or they may represent different Ie ls of abstraction. In both c es we need to eliminate the divergencies or cross-link our terms.
U n i fo r m i t y
It can be argued that by enhancing the uniformity of our terminology, we do injustice to the features that are unique to each individuai image. To some extent this is botb true and inevitable, because it is impossible – even theoretically – to completely verbalize the contents of an image. In our case, moreover, it is exactly what we want. Our aim is not to replace the image by a description or to try to evoke it in the user’s mind, but to provide it with textual keys by which it can be retri ved. We do have an interest in identi ing similariti and divergencies, but we do not want our choice of words to compücate thin for .
There is not much beyond discipline and common sense with which to minimize v i­ ations and omi ions in the selection of the visu elements we want to de ribe, tbough we can seriously improve our chances by a systematic approach 1 1 . At the same time, it is cruci to Iimit the variation in our terminology. If we do not make an e ort on this point, it will become almost impossible to use the index of our descriptors an instrument to check our descriptions against the images. Even worse, others will hardly be able to retrieve information from our descriptions.
Optimizing the uniformity of our terminolo can be done at di erent times during the process and di erent strategies may be used. We do not intend to exhaustively survey all pects involved. Instead we shall take a shortcut by an yzing the featur of two hypothetic types of description, located at both ends of a theoretical scale that indicat degrees of organization.
It is not very di cult to imagine what is to be found on the end of this ale where the ambition to be consistent is minimal or absent. There we encounter free form p se descriptions that closely follow the variety of the images. No prescriptions other than
11 A heterogeneaus collection may be brok n up in more homogeneaus parts; the order in which the images‘ elements are de ribed may be strictly regulated, etc. …

Jö en van den Be , Hans Brandhorst and Peter an uisstede
the nt of the l guage they are written in, regulate their structure. No controlled v ularyregulatestheirterminology. Noindexgivesacce totheindividualdescriptors. No l ni ks established between bro er, narrower, and related terms. In agreement with this low Ievel of the organization of their contents, the hypothetic descriptions are part of a quential text le, together with the b ic information.
A collection ofsuch descriptions would bear a close resemblance to the typic this­ hibition or oeuvre cat ogue. We would have a hard time defending that they aim to provide extended information de ned above: det led , systematic, and consistent.
lt is more complex to de ne what would nd on the other end of the ale. It wou cert nly not be su cient to simply invert the elements of the de nition we just ga . The absence of a feature, such vocabulary control, does not need to be further q . I pr nce does. So, Iet us have a Iook at the features we have just mentioned d try to picture them constituent elements ofextended information.
t t
First we have to k whether we can u free text a vehicle for extended information? this not a ‚contr ictio in terminis‘? In other words: „ls there a place for free text d riptions at the ‚highly organized end‘ of our sc e?“ We think there is, but only on conditions. Some of these conditions concern the contents of the texts, others have with our retrieval instruments.
The retriev of information from a text le, we have said before, is b ed on string makhing. A string is ven a query argument d the le is sequenti ly scann for t currence of this string. This is a process. But more importantly, trying to nd t information in this way is like trying to nd the light switch in one of the rooms of a pitch-d k house. If we are very lucky, may light up one room, but we still have no what is in the r t of the hou . This holds, irrespective of the sophistication of the with which we can de ne our qu tion, such wild cards, pro mity arches, d regular expre ions.
For ee text to be of any u a vehicle of systematic information, it must be m e more transparent by means of an phabetic keyword index of its descriptors. To u l, an index should not mix di erent kinds of information. For example, it should not the nam of p nters, former owners, t hniques, and iconographic descriptors. In other words: to create meaningful indic , the information h to be broken up into its nstituent data elements and put in parate records and elds (Cf. section 3.2).
other conditions have to be ful lled, if free text is to function a vehicle of extended information. The descriptors have to be submitted to vocabulary control and th ought to be cro -linked.
Keywords
The most obvious alternative to pr descriptions is a string of keywords. Both solu­ tions have their vantages and disadvantages. A string of keywords, for e ple, is not subj t to the syntax of natur language. Thus, we can convey addition information the way we arrange them. For instance, the order in .which we put the descriptors may to distinguish between a whole and its parts. A query for words in a particular order or within a particular distance from each other, may then be more relevant. A prose
Image Processing and the (Art) HiBtorical Discipline 37 de ription, on the other band, is. a more exible instrument, e.g. to supply information
about how visu elements e distributed over the surface of a picture.
Vocabulary control
A controlled v abul y c be a feature of the free text descriptions that we would admit to the hlgh end of our ale. In that c e it would be a custom-m e term , b d upon our previous descriptions and created simultaneaus with them. The terms from such a Iist in turn be u d to build new de riptions. It can so ori nate a Iist of our preferred keywords, which we may have put in a separate keyword field, instead of or in dition to the free text de riptions. Finally, we can borrow our d criptors om an extern source of terminology12
Gontext and cr referenc
With ‚church‘ we may mean a building which c house a Christian congregation d is ‚physical‘ enough to park a bicycle against. lt al refers to an abstraction: an ‚organized Christian society‘. A table can be made of oak; it therefore is a wooden table. We may have seen a c tle in a Iandscape painting; you may see a fortress…
Language is our inescapable medium o f (scholarly) communication; a t the same time it is a potentially unlimited urce of confusion. While creating, consulting or copying om a controlled vocabulary come across many linguistic complications, such homonymity and synonymity. And, what is more, have to guide users through these complications.
When we employ a vocabulary – whether of our own making or a ‚foreign‘ one ­ we n d to know the ex t meaning of the words it contains. If our term Iist is no more than an phabetically arranged, ‚ at‘ Iist of the keywords we low de riptors, no light si shed on their meaning. We would feel uncomfortable when borrowing terms om such a Iist. Meaning can of course be expl ned through de nitions, like in a diction y. The mantics of the terms in a vocabulary can al be clari ed by organizing them into hierarchies. In that way the meaning of a term is explained by the term(s) that surround it hierarchic ly. ‚Table‘ could then be subordinated to ‚furniture‘, ‚oak‘ to ‚wood‘, ‚wood‘ in its turn to ‚plant materi s‘, etc. Using the terms from a vocabulary that is thus organiz , we would ways know exactly what is intended.
Like the word ‚church‘ (a building weil an institution) , the word ‚oak‘ should appear (at le t) twice in a hierarchically organized vocabulary of iconography: once a materi for e.g. furniture, a second time the tree it lf, e.g. part of a land ape. a matter of fact, homonyms should appear often their different meanings are relevant to the purpo of iconographic description. Furthermore, when preference is ven to ‚fortress‘ over its synonym ‚c tle‘ – the building, not the ‚rook‘ (not the bird…) of che – this must be m e clear to users of the vocabulary, who may try to nd ‚fortr s‘ by king for ‚c tle‘.
12 Note: Ex ples of such sources include: the subject headings of botb the Princeton d of Christian Art and the Library of Congre , and the Art and Architectural Th aurus. Tbe ICONCLASS System, and Garnier’s Thesaurus lconographique are tools specifically d igned to facilitate the creation of iconographic de riptions.

38 Jö n van den Berg, Hans Brandhorst and Peter van Hu·isstede
Fin ly, to function properly a hierarchy of iconographic descriptors, our controlled must accomodate concepts of such di erent Ievels of complexity and abstraction ‚table‘ and ‚The Annuuciation to the Virgin‘.
are a subject for study potentially rich the ‚representable‘ world ; maybe even richer, since they can also be a testimony about the visible world of the p t. Building a hle ally organized controlled vocabulary – including cross references – for a eld rich that, is intimidating challenge.
S atic cl i cation
So f , have failed to mention one important expedient to meet this challenge: the systematic cl si cation. The essence of a systematic cl i cation heme is that its erder­ ing principle is not alphabetical but, indeed, systematic. This means that its descriptors r g in hierarchic ly subdivided cl ses. Each descriptor is accompanied by ( pha)numeric code or notation which signs it its place in its d s, by which its context and mantics are clari ed. The concepts, de ned of course in natural langu e, and the cod form inseparable units. Access to them is provided by keywords.
Featur characteric of a systematic cl si cation include:
• i de riptors are absolutely unambiguous, since every code is by de nition unique, though the actual words used in it may occur many times in the vocabulary
• a d riptor can consist of a single word, but it can also consist of a detailed de nition of complex – in our c e iconographic – subjects
• b a its hierarchical arrangement is de t with by the codes, there is no need to g e the keywords that give ce to the de riptors, into hierarchies
• of its ( pha)numeric encoding, a systematic cl i cation can not be freely ended a vocabulary that exists of language only. A very high degree of control must exercised during its construction.
For a d ription of ICONCLASS – a systematic cl si cation system especially de­ r iconography – we may refer you to chapter … of this book. There is no need to go det l now.
Summarizing: at the end on our scale of organization opposite the ‚uncontrolled‘, free
form pr d criptions, we nd a nurober of items on our list of wishes:
• hi y controlled free text. • keywords.
• controlled vocabulary.
• systematic cl si cation.
• hier chic organization of descriptors.
• cr references.
Some of these items may be regarded each other’s alternative, some are comple-
mentary.
.
Image Processing and the (Art) Historical Discipline 39
Optimizing retrie
From what h been said above, we can conclude that to optimize the retrie l of iconographic information the following two conditions should be met:
• lt must be possible to specify the semantics and the iconographic(!) context of the te that is used a search argument. This means that have to be able to distinguish between ‚eagle‘ a predatory bird, ‚e le‘ an anim o en u d in heraldry, ‚eagle‘ symbol of St. John, and ‚eagle‘ an attribute of Jupiter.
• lt must be possible to eliminate or at le t reduce the risk that information actu ly pr ent in the information system is withheld om the u r, becau the term in a query is not identical with that used in the iconographic description. This might occur if the search term is of an hierarchical level di erent from that of the descriptor, or if the search argument a synonym of the d criptor. The proble may be rather str ghtforward: the de riptor is relatively sp c, e.g. ‚eagle‘, while the search argument more general: ‚bird‘. It can be more complicated: ‚the symbol of St. John‘ is used as descriptor, while the search argument is ‚eagle‘. In the context of iconography this constitutes a synonym13
lf the retrieval system fails to meet the condition – i.e. if we can not exclude an ‚eagle‘ we e not interested in – it will retrieve too much. lf it f ls to meet the second condition – retrieve the ‚eagle‘ we did want bu� failed to k for in the right way – it will retrieve too little.
It must be admitted that th e two conditions add up to a demand which is too ambitious to answer with even the most sophisticat text retrieval tools. The re on for this is simple it is paradoxical and conf ing to a computer: when try to retrieve iconographic information through textu keys, we may plicitly k for thin we do not want; at the same time we may not explicitly k for things we do want.
4. Conclusions
this article have moved across three related are of inter t, l relevant to the subj t of Pietorlai Information Systems:
• the, often underestimated, intellectu ch lenge studying images elements of the p t
• the technical issues involved in studying imag with the help of the computer
• the theoretical and practical need to d text to images in order to make them c
sible and retrievable.
lconography is a more or less established art historical method. Techniques to store and manipulate electronic images are readily available. In spite of this, the study of !arge
13 A d ideratumforaspeci callyicono aphicretrie system builtupoftwofeatur : ( a) Iconographic details should be cross referenced for the bro er subject and themes they part of, e.g. om a particular saint’s attribute we should be pointed to the saint’s general icono aphy; (b) Subjects and themes that are related in some way should be cross-linked, e.g. om ‚Hercules killing the Nemean lion‘ to ‚Samson killing the lion‘ Th feature c not be called a condition, because it must be seen belanging to expert system rather than a retrie l system. In spite of , both of them are intrinsic to the ICONCLASS System.

van den Berg, Hans Brandhorst and Peter van Huisstede
oun of visu material from the past in a systematic way poses some new problems, ho to have shown in this articlc. We have dealt rather extensively with problems of ( t) historic method. We have so mentioned a number of practical i ues: nancing proj ts, proj t management, the exchange of information, etc.
A u h of interest that may be seen relevant to our subject – we could refer to it the of „pattern recognition“ – h been excluded from our discussion. This h b n done for various re ons. Firstly, the research on this subject is mainly done in elds like t ci Intelligence and Military Intelligence (if that is not a contradictio in te inis). short, not in are where we can bave ready cess to the result of the studies th ng done. Secondly, to be honest, major tbeoretical and practical problems have to be fore pattern recognition and other, comparable, techniques can be applied in a scienti c ly meaningful way. At this moment, and on their own, th qu do not bring us much closer to our m: the systematic study of images elemen m the p t, with the help of the computer.
We c , bowever, turn the argument around and state that these techniques should be t the amewerk of the systematic study of images from the p t. We should not clude that Cuture developments will turn pattern recognition into a tool with w generate meaningful information about images. In dealing with a subject mplex the im ery of the p t, a Pictorial lnformation System should encomp s m y po ible.

Halbgraue Reihe
zur Historischen Fachinformatik
Herausgegeben von Manfred Thaller
Max-Planck-Institut für Geschichte
Serie A: Historische Quellenkunden Band 14
Erscheint gleichzeitig als:
MEDIUM AEVUM QUOTIDIANUM
HERAUSGEGEBEN VON GERHARD JARITZ
26
Manfred Thaller (Ed.)
Images and Manuscripts in Historical Computing
Max-Planck-Institut für Geschichte In Kommission bei SCRIPTA MERCATURAE VERLAG
St. Katharinen, 1992

© Max-Planck-Institut f Geschichte, Göttingen 1992 Print in Cermany
Druck: Konrad Pachnicke, Göttingen Umschlaggestaltung: B ta Werbeagentur, Göttingen
ISBN: 3-928134-53-1
lntroduction
Table of Contents
Manfred Tballer. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . .. . . . .. . . . .. . 1 I. Basic De nitions
Image Processing and the (Art) Historical Di ipline
.Jörgen den Berg, Hans Brandhorst and Peter van Huisstede ……………. , ..5 II. Methodological Opinions
The Processing of Manuscripts
Manfred Tballer………………………………………………………..41 Pietonal Information Systems and the Teaching Imperative
FrankColsonandWendyHall………………….. ………….. ………….. The Open System Approach to Pictori Information Systems
WendyHallandFrankColson……………………… ……………………87 111. Projects and Case Studies
Tbe Digital Processing of Images in Archives and Libraries
PedroGonz lez………………………………… ……………………..97 High solution Images
AnthonyHamber……………………………………. ………………..123 A Supra-institutional Infrastructure for Image Proce ing in the Humanities?
penS.Ore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 D cribing the Indescribable
GerhardJaritzandBarbaraSchub……………………………………… 143
Full Text / Image DBMSs
RobertRowland………………… ………………………. ……………155

lntroduction
Manfred Th ler
This book is the product of a workshop held at the International Univer ty Institute in Firenze on November 151h, 1991. The intention of that workshop h b n to bring tagether people from m y di erent approaches to „im e pr ing“ p b le. The re on for this „collecting“ approach to the subject w a f ling, th t wbile proce ing in many ways h been the „hattest“ topic in Huma.niti computing 1n r ent years, it may be the le t weil de ned. It seems so much b der to say in this are wbat is speci cally important to historia.ns, tha.n to other people. In that situation it w feit, that a foruin would be helpful, which could sort out what of the various approach can be u ful in historic rese ch.
To solve this t k, the present volume h been produced: in m y ways, it re ts the di u ions which tually have been going on less, than the two comp ion volum on the workshops at Gl gow a.nd Trom do. This is intentional. On the one b d, the p ticipa.nts at the workshop in Firenze did strongly feel the need to have proj represented in the volume, which were not actu ly present at the workshop. On the other, the di u ions for quite me time were engaged in cl i ing what the metbodol i issues were. That is: what tu y e the topics for schol ly di u ion beyond the description of individu projects, when it comes to the processing of imag in historic rese ch?
The situation in the ea is made di cult, because some of the underlying umptions are connected with vigoraus re arch groups, who u fora of schol ly debate, which only slightly overlapping; , what is t itly sumed to hold true in one group of projects may be considered obviously wrang in a.nother one, that it cely d explicit refutation.
We hope, that we have been succes ful in bringing some of these hidden di erenc in opinion out into the open. We consider this extremely import t, because only th cl fication allows for a fair ev uation of projects which may have st ted om di erent sets of sumption. So importa.nt, indeed, that we would like to catalogue here me of the b ic di erences of opinion which exist between image proce ing projects. Tbe re er will rediscover them in many of the contributions; editor I think however, that sum izing tbem at tbe beginning may make the contributions- which, of course, have b n striving for impartiality – more e ily rccognizable parts of one coherent debate.
Three b ic di erences in opinion seem to exist today:
(1) im e processing a genuine and independent eld of Computer b d r cb in the Humanities, or is it an auxiliary too ? Many projects sume tacitly – d me do quite outspokenly- that imag� on the computer act illustrations to more conventional applications. To retrieval systems, illustrations in cat ogues and the like. Proj ts of this type tend to point out, that with currently e ily available equipment d currently clearly understood data proce ing technologies, the an ysis of imag , which c quite e ily be h dled illustrations today, is still costly and of uncertain promi . Wbich is the re n why they u me, that such analytical appro hes, if at all, should be undertaken
2
side efef cts of projects only, which focus upon the relatively simple administration of images. Their opponents think, in a nutshell, that while experiments may be need , their over J outcome is so promising, that even the more simple techniqu of today should be implemented only, if they can later be m e u ful for the advanced t hniques now only p ti ly fe ible.
(2) Connected to this is another con ict, which might be the m t constant one in Humaniti data processing during the l t d es, is particularly d isive, however, when it comes to image proce ing . Shall we concentrate on Ievels of pbistication, which are av lable for many on today’s equipment or shall we try to make use of the m t phisticated tools today, trusting that they will become available to an incre ingly !arge number of projects in the future? This specific battle h been fought since the earliest years of Humanities computing, and this editor h found bimself on th sides at difef rent stag . A „right� answer does not exist: the debate in image processing is probably one of the best occ sions to understand mutu ly, that both positions are full of merit. It is pointl to take permanently restrictions into consideration, which obviously will cease to exist a few years from now. It discredits l of us, if computing in history always promises r ults only on next years equipment and does not deliver here and now. Maybe, that is ind one of the more important t ks of the A ociation for History and Computin to provide a link between both worlds, Jending vision to those of us burdened down by the next funding deadline and di iplining the loftier projects by the question of when metbing be ordable for all of us.
(3) The third major underlying di erence is inherently connected to the previous ones. image such is beautiful, but not very u ful, before it is connected to a description. Sh l such de riptions be arbitrary, formulated in the tr ition ly clouded langnage of a histori , perfectly unsuitable for any phisticat t hnique of retrieval, maybe not even unambigously understandable to a fellow historian? Or shall they follow a prede ned cat ogue of narrow criteria, using a carefully controll vocabulary, for both of which it is mewbat unclear how they will rem n relevant for future r arch questions which have not been ked so far? – All the contributors to this volume have b n much to polite to pbr their opinions in this way: arcely any of them does not have a strong one with reg d to this problem.
More questions than answers. „Image proc sing“, whether appli to images proper or to di t ized manu ripts, ems indeed to be an area, where many methodological qu tions rem n open. Besides that, interestingly, it seems to be one of the most con­ uenti ones: a project like the di talization of the Archivo Gener de Indi will continue to in uence the conditions of historical work for d es in the next century. There e not only many open questions, it is worthwhile and necce ry to di uss them.
While everybody seems to have encountered im e processing in one form or the other alre y, preci knowledge about it seems to be relatively scarce. The volume st ts, therefore, with a general introduction into the eld by· J. v.d. Berg, H. Brandhorst and P. v. Hui tede. While most of the following contributions have been written to be self supporting possible, this introduction attempts to give l readers, particularly those

3
with only a vague notion of the techniques coucerned, a common ground upon which the more specialized discu ions may build.
The contributions that follow have been written to introduce speci c are , where handling of images is useful and can be integrated into a !arger context. All authors h e been ked in this part to clearly state their own opinion, to produce clearcut statements about their methodological position in the di ussions described above. Originally, four contributions were planned: the rst one, di u ing whether the more advanced techniqu of image processing can change the way in which images are analysed and handled by historians, could unfortunately not be included in this volume due to printing deadlin : we hope to present it part of follow up volumes or in one of the next i ues of History and Computing.
The paper of M. Thaller argues that anning and presenting corpora of manu ripts
on a work station can (a) save the origin s, (b) iutroduce new methods for palaeographic
training into university teaching, (c) provide tools for re ing damaged manu ripts, the
comparison of band writing and gener palaeographic studies. He further prop s to
build upon that a new understanding of editori work. A fairly long t hnical discussion
of the mechanisms needed to link images and transcriptions of manu ripts in a wider
context follows.
·
F. Colson and W. H l discuss the role of images in te hing systems in university education. They do so by a detailed description of the mechanism by which imag are integrated into Microcosm I HiDES te hing packages. Their considerations include the treatment of moving images; furthermore tbey enquire about relationships between image and text in typic stages in the di ogue between a teaching package and a user.
W. H l d F. Colson argue in the nal contribution to thi part the gener c of open systems, exemplifying their argument with a di u ion of the various degr which control about the choices a user h is certained in the ways in which navigation supported in a hyper-text oriented system containing images. In a outshell the di erence between „open“ and closed systems can be understood as the following: in an „open system“ the user can dynamic ly develop further the behaviour of an image-b ed or image-related system. On the contrary in static „editions“ the editor h ab lute control, the user none.
Following these general description of approaches, in the third part, several interna­ tion projects are presented, which describe in detail the decisions taken in implementing „re “ image proce ing b ed applications, some of them of almost frigthening magnitude. The contributors of this part were ked to provide a di erent kind of introduction to the subject than those to the previous two: all of them should discuss a relatively small topic, which, however, should be discus d with much greater detail than the relatively broad overviews of the rst two parts.
All the contributions growing out of the workshop came from projects, which had among their aims the immediate applicability of the tools developed within the next 12- 24 months. As a result they are focusing on corpora not much beyond 20.000 (color) and 100.000 (blw) images, which are supposed to be stored in resolutions manageable within 5MB I image (color) and 0.5 MB I image (blw). The participants of the workshop feit strongly, that this view should be augmented by a description of the rationale behind
4

the creation of a !arge scale projt t for the systematic conversion of a complete archive. The resulting paper, by P. Gonza!ez, describes the considerations which Iead to the design of the \rchivo General de Indi projt t and the experiences gained during the completed stages. That description is enhanced by a discussion of the stratrgies selected to make the raw bitmaps acce ible via suitable descriptions I transcriptions I keywords. A critical appraisal, which decisions would be made di erently after the developments in hardware tecbnology in recent years, augments the value of the de ription.
The participants of the workshop feit furthermore strongly, that their view described above sbould be augmented by a description of the techniques used for the handling of images in extremely high resolution. A. Hamber’s contribution, dealing with the V ari project, gives a very thorough introduction into the technical problems rncountered in handling images of extremely high quality and also explains the economic rationale behind an approach to start on purpose with the highest qu ity of images available today on prototypical hardware.
As these huge projects both were related to iustitutions which traditionally collect source material for historical studies, it seemed wise to include also a view on the roJe images would play in the data archives which traditionally have been of much importance in the considerations of the AHC. E.S. Ore discusses what implications this type of machine readable material should bave for tbe infrastructure of institutions speci c ly dedicated to Humanities computing.
Image systems which deal with the archiving of pictorial materi and manuscript systems have so far generally f rly „shallow“ descriptions. At le t in history, moreover, the rely quite frequently on pre-de nt’d terminologies. G. Jaritz and 8. Schuh describe how f and wby historical research needs a di erent approacb to grasp as much of the intemal structure and the content of image possible.
L t not le t R. Rowland, who acted host of the workshop at Firenze, describes tbe considerations which currently prepare the creation of another largesc e archival datab e, to cont n !arge ounts of material from the archives of the inquisition in Portugal. His contribution tries to explore the way in which the more recent developments of image processing can be embedded in the general services required for an archival system.
This series of workshop reports shall attempt to providr a broader b is for thor­ ough di ussions of current methodological questions. ‚fheir main virtue sh l be, that it is produced su ciently quick to become available, before developments in this eld of extremely quick development make them ob lete. We hope we have reached that goal: the editor has to apologize, however, that due to the necessity to bring this volume out in time, proofreading h by neccessity be not intensive it should have been. To which �nother shortcoming is ded: none of the persons engaged in the final production of this volume is a native speaker of English; so while we hope to have kept to the standards of what might be described „International“ or „Conti mtal“ English, the native speakers among the readers can only be ked for their tol rance.
Göttingrn, August 1992