Eric A. Meyer
Peter Murray
Case Western Reserve University
Cleveland OH 44106, USA
eam3@po.cwru.edu
pem@po.cwru.edu
The Borealis image server is intended to address the need for creating a central place to store large numbers of images. The system itself is composed of widely available resources and could be easily reproduced at other sites at very little expense. A Borealis server can be queried from any Web page or browser and delivers an image which has been watermarked. This watermarking is intended to assert ownership of the image in an obvious but unobtrusive fashion. The server is managed using a database which ensures that collection identifier-picture identifier pairs are unique throughout the server. A variety of output styles are supported, including the image in its full size and a reduced version of the image. Images may also be returned in a variety of graphic formats, and a basic search engine allows users to quickly find images relevant to their needs. Through these and other features, the Borealis system is capable of filling a critical gap in the effort to make images available on the Web.
The Borealis Image Server project[1] was launched in late 1995. This initiative was undertaken in response to the realization that there is need for a simple, inexpensive, easily maintained image server which offered basic copyright protection. The result was described in the paper "Borealis Image Server"[2] which was presented at the Fifth International World Wide Web Conference in May 1996.
This paper is intended to be a description of the changes in the project since publication of that paper. It represents another chapter of a work-in-progress, as we are continuing to expand and improve on the Borealis system. Some of the changes described in this paper are the use of the PNG format as an image storage mechanism, Web-based administrative tools, and the addition of a basic search engine. Most of these changes were in response to advances in available technologies, whereas others (such as the search engine) were the result of inquiries and requests from members of the Internet community.
Many of our assumptions remain unchanged since the project began; these include the need for high-speed access and the use of free, proven technologies in the construction of the system. However, there are a few which should be reviewed or explained anew.
In order to keep the image URLs easy to remember and type, images are organized into image collections. Each collection has a collection manager who is able to add, remove, or edit images within the collection, and may create new collections or sub-collections as necessary. This approach, in conjunction with the use of a database to track image locations, keeps the image URLs fairly short and makes it easier for the collections to be managed.
It was decided early in the project that in order to simplify storage issues, the Borealis server would keep a single unmodified copy of each image. From this one file, the Borealis system can produce watermarked images, thumbnail images, scaled images, and converted images in a variety of graphic formats.
As we saw it, the alternative would be to require that each image be stored in its original state, a watermarked state, a thumbnail style (which would limit the system to a predefined thumbnail size), and conversions of the image, in both original and watermarked states, to a variety of graphic formats. By simply storing a single image, the system is given a great deal of flexibility. Addition of new system features, or redefinition of existing features, is accomplished by altering the Borealis system's code. There is no need to alter any pre-generated images, or to create new images, because the original is the only stored image and it does not need to be changed.
The Borealis system attempts to assert ownership through image watermarking-- that is, it creates a display image by combining a small image over the source image, thereby altering the source image enough to make it difficult to remove the watermark but not so much as to make the display image impossible to view. In many ways, watermarking is the core of the Borealis project. It was one of the original design goals, and it makes the system useful not only for academic institutions, but for any organization with a desire to provide large numbers of pictures to its clients.
The system is configured to "tile" a small watermark over a larger image, although it is possible to define a watermark large enough so that it is only used once, as was done in the Atlanta Olympics Picture Gallery[3]. However, the reason for this tiling approach is to watermark as much of the image as possible; this is an attempt to prevent end users from cropping out the watermark. If the watermark is a round seal, using one large watermark will miss the corners of the image, whereas tiling small watermarks across the image will account for most of the visible area.
All access to the images must occur through the Borealis system; in fact, there no way to access the images on the Borealis server without first going through the Borealis system. The watermarking step cannot be bypassed by the user, although it can be suppressed by collection managers, if they choose to exercise this option.
|
First, let's take a look at how a Borealis server delivers a single image. When an image is requested, the server locates the requested image, loads it into memory, converts it to an internal graphics format, scales it if necessary, watermarks it, converts the watermarked file to the requested graphics format, and delivers the final image to the requesting agent (typically a Web browser). The steps taken by the Borealis server are shown by the flow chart in Figure 1.
Here's a concrete example. The server receives a request which looks something like this:
http://images.cwru.edu/full/campus/cmp0011.gif
 |
The server immediately breaks up the URL into the relevant parts: full defines the output style (in this case, full size, or the size of the original file stored by the system); campus indicates the collection of which this image is a part; cmp0011 is the image's identifying name, and .gif specifies the image format in which the display image should be.
The Borealis system performs a search of its database for the collection-image combination campus:cmp0011 and determines the location of the original image. It loads this image into memory, extracts any information stored in the image file, watermarks the file, converts the file into the graphic format requested (for maximum compatibility, the default format is GIF), and delivers this converted and watermarked image to the user [Figure 2].
A full-size version of the source image is watermarked and returned. In other words, no scaling is performed on the source image, and the display image is exactly the same size as the original. The outputFormat type "full" is used in the URL to request this style [Figure 3].
Example: http://images.cwru.edu/full/campus/cmp0094
Figure 3. Full-sized image
Thumbnail versions of the source images can also be generated by the Borealis server. A thumbnail display image is automatically scaled to be 100 pixels on the longest axis, with the other axis scaled proportionally. The thumbnail version of the source image is also watermarked, typically with a smaller-sized watermark file, and no copyright information is included with the display image. The outputFormat type "thumb" is used in the URL to request this style [Figure 4].
Example: http://images.cwru.edu/thumb/campus/cmp0094
Figure 4. Thumbnail image
A user can also request an "image information" page. This is a full HTML page consisting of the title of the image, an inline thumbnail of the image, copyright and author/title information, and a link to a full cataloging record, when such data exists. This output style is useful when a user wishes to get information about a specific image without going through a catalog system first. The outputFormat type "info" is used in the URL to request this style.
Example: http://images.cwru.edu/info/campus/cmp0094
Finally, users can request "contact sheets" of entire collections. These sheets are effectively collection overviews. The browser displays a thumbnail of each image in the collection, along with title and author information, as well as the sizes of the source images (measured in pixels). This output style is very useful for getting a quick idea of the contents of a collection without having to view each image individually. The outputFormat type "contact" is used to request this style, and no image should be specified; the URL should end with a collection identifier [Figure 5].
Example: http://images.cwru.edu/contact/campus/
Figure 5. Contact sheet
In addition to the above output styles and graphic formats, a user can request modifications to the output through the use of parameters which are appended to an image's URL, as described in the HTTP specification[4]. These parameters are:
http://images.cwru.edu/full/campus/cmp0011.gif?scale=0.75
http://images.cwru.edu/full/campus/cmp0011.gif?size=90,65
http://images.cwru.edu/full/campus/cmp0011.gif?rotate=90
The Borealis system runs on a UNIX machine (our current server is a Sun SparcStation 20 with Solaris 2.5) running an HTTP server. A Perl script called imagesrv.pl forms the basis of the request mechanism. The various output styles (full-size, thumbnail, and so on) are aliased to imagesrv.pl using the ScriptAlias directive in the Web server's srm.conf configuration file. When imagesrv.pl receives a request, it checks the validity of the input and makes sure that the proper utilities exist to perform the requested conversion. For this, it must be able to convert the source file from PNG into the PNM internal format, and then after watermarking convert it from PNM into the requested graphic format.
Error control and logging are also performed by imagesrv.pl. After processing the input and determining what step must be taken, the script strings together a series of UNIX commands. These commands are then passed to the shell (via the open command) in order to perform the image processing. The NetPBM utilities and various add-ons are used to convert the source image into the internal PNM format, perform the watermarking and scaling functions, and to convert the internal PNM format to the image format requested by the user.
The following components are used in the Borealis system:
The server maintains an image database called imageDatabase to link collectionID - imageID pairs to a file on disk. Presently this database is in the form of a DBM file on disk. Since the performance of DBM breaks down for large indexes, however, we may need to move to a more formal database product as the collection grow (as summarized in the Future Plans section). The key portion of an entry in the index is a string composed of the collectionID and imageID values; the uniqueness of these pairs is enforced by the Borealis system. The data portion of an entry is the absolute pathname to the source image. This index is updated in real time whenever source images are added to or removed from a collection.
The Borealis system is designed to allow several collection managers, each with his own login-id and password to the server. This ID and password allows complete control over images contained within collection in the manager's area. Collection managers may create and remove collections, add and remove images within these areas, and change watermark, title, and author information for images and collections. In order to maintain a flat URL structure, the Borealis system guarantees that the collectionID for a given collection is unique throughout the system, including those collections under the control of other collection managers.
One goal of the system is to enforce a relatively flat URL space so that the URLs of images are short and easy to remember. This feature should be appreciated by catalogers and users alike. The URL of a Borealis image is in the form:
http://images.cwru.edu/outputStyle/collectionID/imageID.ext?param=value
where outputStyle represents the requested format of the image (thumbnail, full-size, etc.), and is an alias (in both the srm.conf file and the UNIX file system) to imagesrv.pl; collectionID is a unique collection name on the Borealis server containing a group of related images; imageID is the name of an image which is unique within in that collection; and the ext, if any, specifies a particular graphic format (see "Image Conversion" below). param and value can be any one of a number of things as described in the section "URL Parameters."
While it is possible for an image collection to contain thousands of individual images, it is generally not efficient for a UNIX directory to contain thousands of directory entries. An image collection manager can create subdirectories of images based on a logical structure that is meaningful to the collection manager, but which remains invisible to the end user.
This database maintains at least three pieces of information for every image on the server: the collectionID and the imageID, which as a pair form a unique key in the database, and the actual pathname of the image. For example, an image with imageID of "cmp0066" which is part of the collection whose identifier is "campus" might actually have the server pathname /images/dms/jan96/p1/cmp0066.png. The URL of this image in GIF format at full size would be:
http://images.cwru.edu/full/campus/cmp0066.gif
The Borealis system receives the above request from a Web browser (for example). It searches through imageDatabase and finds the following entry:
campus:cmp0066 /images/dms/jan96/p1/cmp0066.png
The system then loads the file into memory, watermarks it, and delivers the resulting display image to the browser.
One of the most popular requests from users of the system, and of attendees at the Fifth International World Wide Web Conference, was the addition of a search engine to the Borealis system. The use of the Harvest search engine to index information imbedded in PNG files has been demonstrated[14] by Dave Beckett, University of Kent at Canterbury. The Borealis system therefore employs a Harvest Gatherer which extracts the information from individual PNG files for the Harvest Index/Search engine, using a locally written Perl routine to do so. A Web form is used to solicit search parameters from users of the Borealis search system. The search can be restricted to certain collections, or set to cover the entire server.
Once the user enters a keyword or set of keywords, the search is performed. The Borealis server keeps track of the images whose information matches the search terms. It then begins to return a page similar to the contact sheet output style. In this case, however, in addition to thumbnails of the images and the author, title, and size information, the results also display the collection of which each image is a part, and a link to the contact sheet for that collection.
All of the source images on a Borealis server are stored in the PNG format. We settled on this approach after examining the PNG specification[15] and finding that it was uniquely suited to our purposes. First of all, PNG offers lossless compression up to 24-bit color, which allows us to conserve disk space without sacrificing the quality of the source images. PNG is also an open standard and therefore free of license restrictions.
Most important, however, is the ability to imbed text information directly into the image files. Using this capability of PNG, we are able to store basic information about each image, such as the copyright text and the title of the image, inside the image itself. This is preferable to the original design, which employed separate text files, and much less processor-intensive than using a separate database to store this information.
A PNG file is made up of chunks[16], each containing information such as the image data, alpha and gamma channels, compression specifications, and other image-related information. The PNG specification also allows for the creation of keyword-value text pairs within the file. These repeatable text fields are labelled "tEXt" chunks. Several keywords used by the Borealis system are already defined in the PNG specification; these include "[Image] Title," "Author," and "Copyright." For PNG files delivered to end users, the standards-defined keyword "Software" is set to a URL and description of the Borealis system. In addition, the Borealis system uses the keywords "Borealis-Watermark" for the full-size image watermark file and "Borealis-tWatermark" for the thumbnail-size watermark file. The keywords are so named in order to avoid possible future conflicts, should watermarks be added to the PNG specification.
A future version of the Borealis system will use the keyword "Borealis-CatalogLink" for linking information into local library catalogs. The PNG specification also allows for text chunks to be compressed (these are labelled "zTXt" chunks), but since the text information we are imbedding is fairly short, little size efficiency would be gained from compressing, and it would come at the expense of computational resources. A locally written Perl library, based on code from the PNG Development Group library, is used to read and write the text chunks in each PNG image.
The operation of watermarking is a non-destructive procedure. It would be simple to merely overwrite the image, but this would defeat the purpose of watermarking. Instead, the light values of the watermark image are used to modify the image so that the watermark appears to have been "stamped" into the surface of the image. This permits the user to see what the file looks like while still asserting ownership of the image.
There are two elements in the watermarking procedure: the source file and a watermark file. In the interests of speed, the watermark file is stored in the PPM graphics format. When an image is requested, the server loads the source file and performs any transformations (scaling or rotating) that have been requested. It then loads the watermark file for the image or collection, and the watermark is applied to the converted image in a grid pattern.
 | plus |  | equals | 
|
Pixels in the watermark file which are darker than 50% gray cause the corresponding pixels in the image to be darkened, whereas pixels lighter than 50% gray have a lightening effect [Figure 6].
It is also important that the copyright information be displayed as part of a full-size image. If there is enough room, the copyright text is watermarked into the image [Figure 7]; if not, the copyright statement is appended to the image inside a black bar [Figure 8]. As with the basic watermarking, the addition of the copyright statement is done after any scaling or resizing of the image itself. Copyrights are not added to thumbnails.
 |
 |
As a way to speed access to popular images, a cache area is reserved on the Borealis server's disk to store watermarked images. This area is implemented as a single directory with the filenames corresponding to the collectionID - imageID pair. Information from sizing and rotating parameters are also included in the filename. When a request for an image is received, the Borealis system first checks the cache area for the image. If it is found, and if the source image and watermark images have not been subsequently modified, the system immediately delivers the image to the user.
The Borealis system uses a time-based cache system where the oldest images are removed as new images are converted and stored in the cache. This weeding process is performed as a cron job so that the removal process does not impact the performance of imagesrv.pl. This job runs periodically throughout each day; at this time, the interval is one hour. Images may be removed based on a number of criteria: creation date, last-accessed date, or file size. Finally, if the Borealis system detects an out-of-disk-space condition, it also forks a background process to clear the cache in order to reclaim needed disk space.
There may be times when the server cannot deliver a requested image. This can happen in circumstances where the image does not exist, either due to a mistyped URL or removal of an image from the server, or if access to the image has been temporarily restricted. If an error occurs, the system instead returns a small image containing text to the effect that the requested image is not available [Figure 10]. Error messages are delivered as images, and not as text, because in many cases the Borealis system will be referenced using an IMG tag in a Web page. Error images are created using the "pbmtext" utility and then converted to GIF (with a transparent background). Error parameters, along with information from the HTTP request such as the Client-IP and Referer: fields, are also written to a file for review by the collection manager and the Borealis system administrator.
 |
By the time this paper is presented, we expect the server to be HTTP/1.1[17] compliant. Of note is that the Borealis system will run as a "non-processed header" (NPH) script, allowing the Borealis system's HTTP server to send and receive all of the HTTP headers. The server will send back the appropriate "content-length" headers, and process byte-range requests to continue transmission of interrupted transactions. The server will also respond to HEAD requests and cache control mechanisms. Lastly, we expect to make use of the "Accept" request header in order to return to the browser the most preferred graphics format when no format is specified in the image's URL.
Another major improvement to the Borealis system is the addition of Web-based administrative tools, referred to collectively as the Collection Management System (CMS). This form-driven interface allows collection managers to add, remove, or transfer images, create or delete entire collections, set collection parameters, and edit information for individual images. Access to the CMS is restricted by a username-password system, so that only authorized collection managers can get access to the collections.
As of this writing, the CMS offers six administrative tools: Add Image, Edit Image, Image Groups, Add Collection, Edit Collection Data, and Watermark Add/Delete. All six are listed on the initial CMS screen, which is presented as soon as the collection manager logs in.
The Add Image screen is made up of two main sections: an area to enter data regarding the image to be added, and a list of collections to which the collection manager has access [Figure 11].
The first input is a file upload entry. This form-upload input type is relatively new[18], so use of the system is restricted to those collection managers who are using recent browsers such as Netscape 3.0. We expect more browsers to support this new element in the near future. Using a file upload input removes the need for creating FTP accounts for the collection managers; this is felt to be an improvement in many ways. The server's security is better protected, and there is also a reduced need for system documentation and the training of collection managers. The uploaded image may be in any one of the graphic file formats supported by the Borealis system. The source image is converted to PNG before insertion into imageDatabase.
Once the file to be uploaded has been designated, the information entered, and the proper collection selected, the collection manager hits the "Add Image" button. He is then presented with a screen which summarizes the information as the system received, and confirmation is requested. If the collection manager confirms the addition of the image, the system then converts the image to the PNG format, imbeds the text information into the image file, and stores it in the appropriate directory on the server.
 |
The Edit Image tool [Figure 10] is quite similar to Add Image in that it displays and allows the editing of the image's imbedded text information and watermark, although there is no ability to upload a file. Instead, there are options to move the image to a different collection, or to delete the image altogether. In addition to the collection list, there is also a list of images in the currently active collection.
The Image Groups tool displays all of the images in a given collection. The collection manager is able to select multiple images and then may choose to either move the entire group to another collection, or delete the selected group. Watermark Add/Delete allows for the uploading of watermark files to, or the deletion of watermark files from, the collection manager's area.
If a collection manager wishes to set up a new collection, he would select the Add Collection tool. The user is prompted for the collectionID, as well as for a more verbose title. Once the system has verified that the collectionID is unique, the collection manager is presented with the Edit Collection Data tool, with the new collection preselected.
Finally, the Edit Collection Data tool permits the collection manager to change the basic parameters of an entire collection, such as the default copyright statement or the default watermark. This screen is also used to set the robot flag. This parameter controls whether or not the contents of the collection may be indexed by automated data gatherers such as WebCrawler. An entire collection may also be deleted from this screen, although this operation requires confirmation from the collection manager.
Although we are fairly happy with the tools we have created, there is room for improvement and expansion. Some possibilities under consideration include displaying a summary of valid watermark files as images instead of a text list, and a tool which will allow the collection manager to index the collection on the fly.
As discussed in the "Server Description" section, we plan to make full use of the content negotiation capabilities of HTTP/1.1. This will allow us to determine which graphic format would be best to return to a given user if no format was specified, instead of always defaulting to GIF, as the system does presently.
Copyright ownership information and author/publisher information is usually stored in the bibliographic record found on a catalog system such as EuclidPLUS. We have considered retrieving this information from the catalog system and then watermarking it into the image. Unfortunately, we have yet to find a satisfactory general solution to this goal, although we continue to see promise in the Z39.50 protocol[19] used to transmit bibliographic data from one system to another.
For those authenticated to do so, the system will offer an output style that is a non-watermarked version of the image. We would anticipate that this would be used by the staff members who maintain the image collections. We also would like to propose a system where a one-time or limited time key can be issued by the manager of an image collection so that a user can retrieve an un-watermarked version of the image (presumably after negotiating a usage agreement with the copyright holder/collection manager of the image archive).
We are considering moving to an object-oriented database, which may offer significant advantages over the current directory-structure approach. This will depend on the versatility and universal availability of the products we study; one product currently under review is mSQL[20], which has the advantage of being free for education and non-profit use.
It was always our intention to continue development of the Borealis system, and this paper is merely the most coherent expression of that effort. We hope that this paper provides an interesting look into the development of a system over time, as well as an update on the project's features. As available technology advances, the Borealis system will advance with it. The use of the PNG format, addition of a search system, and the use of file upload inputs are all examples of the project's growth. We look forward to future improvements such as making the server more efficient by employing the content-negotiation features of HTTP/1.1 and delivering display images in the best graphic format possible to each user. As always, the Perl portions of the Borealis source code are available at http://borealis.cwru.edu/source/, as is the watermarking module, which is written in C. It remains our hope that the Internet community will not only use Borealis, but also help us to improve the Borealis system.
Eric A. Meyer is the Hypermedia Systems Manager at Case Western Reserve University (CWRU in Cleveland, Ohio, USA. Eric is a 1992 graduate of CWRU and holds a Bachelor of Arts Degree in History as well as minors in English, Astronomy, and Artifical Intelligence. His current activities include managing the CWRU campus Web server, direction of campus Web growth, and development of new Web-based technology. It is in this last capacity that he has been involved with the Borealis Image Server project. He is also the author of a critically acclaimed on-line tutorial titled "Introduction to HTML" (http://www.cwru.edu/help/introHTML/toc.html) and maintainer of a set of pages concerning CSS1 support in Macintosh Web browsers.
Peter Murray is the Library Systems Manager at Case Western Reserve University (CWRU) in Cleveland, Ohio, USA. Peter is a 1991 graduate of Miami University with a Bachelor of Science Degree in Systems Analysis. His primary job responsibility at CWRU is managing the library automation system and providing UNIX technical support. Peter has been an active user of the Internet since 1988 and has conducted many regional training sessions. Peter is responsible for much of the Borealis Image Server programming.
