Navigation post

Creating and Using Virtual Reality:
a Guide for the Arts and Humanities

Virtual Reality Case Study Library

Navigation post BibliographyNextGlossaryBackContents

Case Study 3: Virtual Saltburn by the sea: Creative content design for virtual environments
Clive Fencott, University of Teesside

Summary

This case study is based on a programme of ongoing research and a paper presented at the Virtual Systems and Multimedia conference in 1999 (see http://www-scm.tees.ac.uk/users/p.c.fencott). This case study:

  • Uses VRML for Heritage/Tourism websites; the model of the Cliff Lift was subsequently bought by the Saltburn Improvement Company
  • Proposes a content model for the development of Virtual Environments (VEs) which advocates the planning of content in order to induce a feeling of presence for the user
  • In this particular study, in contrast to the archaeological case studies also included in this Guide, the author advocates the inclusion of visual clues rather than accurate representations of objects
  • Is based on over six years of experience in teaching VE design to both graduate and undergraduate students on both creative and technical courses.


Figure 13: Virtual Saltburn by the Sea

Virtual Saltburn by the sea is on-line at: http://www-scm.tees.ac.uk/users/p.c.fencott/Saltburn.

Introduction

What are we designing when we design a VE? Are we designing the scene graph, VRML nodes and so on? Are we designing chairs, tables, machines, creatures and so on? Are we trying to recreate reality or at least some imaginable reality? The answer to the first two is yes and no! The answer to the third is no! Eventually we will create chairs and tables and maybe creatures and whatever else but we we don't start by designing the scene graph let alone coding up VRML nodes. We are designing to communicate. We are designing to communicate the essence of things that visitors can complete for themselves in their own minds. We are also designing for 'the effective communication of causal interaction' (Ellis 1996) because VR is an interactive medium and we have to communicate its possibilities and the consequence of exercising those possibilities. We need to consider the initial stages of VE design at an appropriate level of abstraction that will help us create a model of perceptual content, which will in turn lead us to create a scene graph that the computer system can render as the sort of mediated environment that we want. That is what this paper is all about; the design of perceptual content. But first an example that we will use to illustrate the process.

The Cliff Lift at Saltburn

Saltburn by the Sea is a small Victorian seaside resort which was purpose built in the nineteenth century, having been simply a few seashore cottages used by fishermen and smugglers. As can seen from Figure 14, cliffs rise from the beach and to help the journey between beach and town (at the top of the cliff) a cliff lift was built in 1884. This consists of two small carriages running on an inclined tramway and connected by cables running round pulley wheels situated underneath the operators' hut at the top of the cliff. The motive power is water based. Each car has a tank underneath which can be filled with water. When some passengers are ready to travel, water is run into the tank of the car at the top of the tramway until its weight is greater than the car at the bottom. Letting the brake off a little establishes this fact. As the car at the top of the tramway moves down under the weight of the water and the passengers, so the car at the bottom is pulled up. When the short journey of 207 feet is completed the car at the bottom discharges the contents of its water tank ready to be pulled up on the next run.

The cliff lift viewed from the pier
Figure 14: View of the cliff lift from the pier

Over the years, the seafront at Saltburn has changed little apart from the pier being shortened as a result of a storm in 1974. The buildings at the landward end of the pier, seen white with red geometric patterns in Figure 14, have been enlarged several times and now obscure the ticket office for the cliff lift which is at the foot of the tramway. Today the cliff lift at Saltburn is one of the oldest of its kind in the world and still carries some 70,000 people a year between town and beach and back.

Requirements and conceptual modelling

Requirements modelling parallels very closely the software engineering concept. One of the chief requirements is that purpose should be clearly established here.

Conceptual modelling is effectively the background research activity common to many design projects but in particular those with an aesthetic component. It is the gathering of materials, taking of photographs, sketches, plans, sound and video recordings, etc. It might also include the construction of mood boards as well as potential storyboards etc. Of course the world to be built might have no real-world counterpart and this will of necessity impact on the kinds of activities that might be undertaken here.

Conceptual modelling is where the VE builder gets to know the world to be built. It is also the stage at which major decisions concerning the type of content to be modelled are made.

The principal requirement for the Saltburn VE was that it had to appear fairly realistic but must be focused on trying to give visitors some sense of Saltburn as a seaside town with a strong Victorian heritage. The model also had to run with an acceptable frame rate on average home computers. Thus a few major activities for visitors to undertake, supported by a simple but evocative landscape, was the objective.

The town is quite small and separated from the seafront by steeply sloping, grassy cliffs. The seafront itself is relatively undeveloped by modern standards; there is a pub, a couple of gift shops, a surf hire shop (Yes! Saltburn is an important surfing centre), the pier the only one in the north-east of England, an amusement arcade at the entrance to the pier and the cliff lift itself. Because of its relative solitude even at the height of summer and the high cliffs that rise immediately to the south there is also quite a lot of bird life. Capturing these in some way would offer something of the sense of Saltburn that is required.

One of the challenges in building this type of model is that it is a potentially unbounded, outdoor environment which would therefore constitute a greater test of the effectiveness of the design method. This is because the visitor has to be persuaded to remain in the area modelled rather than being constrained by walls and other physical barriers.

The decision taken was to model the cliff lift and its immediate environs at the foot of the cliff in some detail. The sea, beach, cliffs and cliff top would be kept very simple but modelled on a grand scale so as to make it difficult for visitors to find 'the end of the world'. In terms of interaction as a focus for visitors' attentions the following was decided upon:

  • A working model of the cliff lift that can be operated by the visitor and that will also allow the visitor to ride it up and down
  • Some sort of activity inside the amusement arcade
  • Some bird life that is active in some sense on the beach
  • Something to represent surfing would be desirable.

At this stage a sequence of prototype VRML models was developed which concentrated on investigating activities and interactions using very simple geometry that might be suitable for the final model.

Despite the fact that an apparently realistic model of a real place was being modelled, it soon became apparent that a whole series of choices needed to be confronted as a result of the necessary process of abstraction. In other words, the nature of perceptual reality as opposed to objective reality was being confronted. In the next sections, techniques for considering and modelling perceptual reality are investigated in the context of the Saltburn model.

Perceptual modelling

Perceptual modelling is the act of designing the perceptual opportunities and their inter-relationships. It is of course modelling the intended users' experience of the VE. Perceptual mapping (discussed below) techniques are used to build up a meta-narrative structure analogous to the comprehensible labyrinth of Murray (Murray 1997) of perceptual opportunities which are categorised according to the role they play in the planned scheme of possible user activity. Perceptual opportunities deal not only with conscious experience but also with unconscious experience, sureties, which deliver belief in the VE perceptual realism in (Lombard and Ditton 1997) irrespective of any real world counterpart. The existence and importance of unconscious experience is identified and discussed in e.g. Spiney (Spiney 1998) and Blackmore (Blackmore 1999).

The perceptual opportunities (PO) model of the content of VEs consists of a set of syntactic categories, which can be seen as attributes of any object or group of objects that might conceivably be placed in a VE (Fencott 1999). These attributes specify the way in which the object is intended to function in terms of both its ordinary, everyday meaning (denotation) and its special meaning as part of the interactive processes of the VE it will exist in (connotation). The syntactic categories into which POs can be characterised identify their role in achieving purpose and it is their planned interaction that gives us the overall structure we are looking for. We might thus see POs as a possible characterisation of the lexia basic units of meaning of virtual content rather than its scene graph representation. Figure 15 shows how the range of POs can be broken down into three principal forms that are briefly discussed below.

diagram of perceptual opportunities
Figure 15: Perceptual Opportunities

Because of resource limitations, objects should only be placed in a VE if they provide a clearly identified perceptual opportunity integrated into the VE's perceptual map. They will then support the purpose of the world.

Surprises

The idea for surprises as perceptual opportunities came from the 'appropriately designed infidelities' of Whitelock et al. (Whitelock et al, 1994) who suggested them for emphasis in virtual worlds and thus to precipitate conscious learning. In other words, non-mundane details that are not predictable but they do arise, however surprisingly, from the logic of the space consciously accepted. Surprises, therefore, are designed to deliver the purpose of the VE by allowing visitors to accumulate conscious experience from which narratives can be constructed after the visit.

Surprises can be:

  • implausible but beneficial
  • totally plausible but unexpected

and there are three basic types:

  • attractors
  • connectors
  • rewards.

Note that we have already identified four main foci for interaction in the conceptual modelling phase: the cliff lift, some bird life, something going on in the amusement arcade and something to do with surfing. We will see that these can all be modelled out of the basic set of surprises. We will also see that there are other large structures that are fundamental to VEs that POs can model as well. First, however, we will look at POs on the small scale in terms of units of interaction or agency.

Attractors

Attractors are perceptual opportunities which seek to draw the attention of a visitor to particular places or moments of interest in the VE. They will often be seen or heard from afar. Attractors can be:

  • Mysterious partially obscured/revealed objects, strange or unknown objects, both closed and open doors and doorways
  • Active movement, flashing lights, sounds changing pitch or volume
  • Alien objects that belong to another world,VE, or context altogether, 2-D maps, strange symbols to indicate the end of levels
  • Sensational objects which attract attention through non-visual senses, spatialised sounds, vibrations, smells etc.
  • Awesome large, famous, expansive, etc.
  • Dynamically configured objects that are relocated in space/time
  • Complex made up of a variety of attractors, perhaps seen at a distance
  • etc. this list is not exhaustive.

However, although many attractors rely on people's natural curiosity they are also directly related to other emotional involvements:

  • Objects of desire objects that have some benign significance to the visitor and more particularly to the task at hand
  • Objects of fear objects that have some malign significance to the visitor and to the task at hand.

Animation is a particularly successful form of attractor in that it makes things stand out because of our deep-rooted perceptual affinity for movement. However, attractors may be static and quite local. Doorways as both entrances and exits are examples of static attractors, as are partially obscured objects and localised sounds.

Examples of attractors in the cliff lift are:

  • The complex of buildings and the various sight lines of the horizon and promenade as seen from the initial startpoint (Figure 17), mysterious, awesome
  • Moving cars of the cliff lift 10 seconds after entry into the VE (also Figure 17), active
  • The oystercatchers on the beach in front of the pier from initial startpoint (Figure 16), active, mysterious
  • Open doorways of amusement arcade and ticket office, mysterious
  • Localised sounds in the amusement arcade, sensational.

All attractors rely on people's natural curiosity. Their prime purpose is to draw people into areas of conscious activity, called rewards, which are designed to deliver the main purpose of the VE.

Connectors

Connectors are the means by which visitors are helped to satisfy goals they have set themselves in responding to attractors. This could be through:

  • basic interaction technologies
  • information objects maps, plans, signposts, etc.
  • on-line help
  • degraded reality systematic removal of detail to deflect visitors back to areas of interest
  • and so on.

Examples of connectors in the cliff lift are:

  • the brake controls and the indicator signs over them which indicate which brake is active and what it does
  • the railings, steps and pathways act as connectors by providing sight lines to the main areas of interest
  • in addition, if the visitor travels away from the cliff lift along the promenade or cliff top, degraded reality is used to indicate they are leaving the main area of the VE. In this case the uprights on the railings disappear along with the presence of any other objects.

Rewards

Rewards come in various forms: local, peripatetic and dynamic, and should be designed to as their name suggests reward visitors for their efforts at interaction. The purpose of rewards is to deliver the specifically memorable experiences of the VE as well as ensuring that visitors linger appropriately from time to time as they move around the world. In virtual tourism, for instance, the longer visitors linger overall the more likely they are to find the virtual experience memorable and perhaps retain the desire actually to visit the place the VE is modelling.

Startled Oystercatchers

Startled Oystercatchers
Figures 16a and 16b: Startled Oystercatchers

In the context of the cliff lift, the principal rewards are:

  • Seeing the cliff lift cars change behaviour as a result of clicking the brake controls
  • The oystercatchers flying away when the visitor gets too close to them
  • Sounds changing as the visitor moves around inside the amusement arcade
  • Encountering all the open doors having responded to the complex attractor of the initial viewpoint.

Perceptual mapping

Surprises should work together in patterns to form possible temporal orders or rewards and thus the coherent set of experiences that are intended to deliver the purpose of the world. These patterns are often called Perceptual Maps. The aim of perceptual mapping is to take the basic instances of attractor, connector, reward triples and compose them to build up a picture of the perceptual content of the VE as a whole. We can identify a number of larger structures:

  • Choice points present the visitor with a choice of competing attractors or a choice of alternative goals for the same attractor. Murray (1997) identifies them as a major source of dramatic potential in VEs
  • Retainers are groupings of surprises that constitute major sites of interest and/or interaction that seek to deliver the purpose of the VE as identified by requirements and conceptual modelling
  • Challenge points can be as much conceptual as locational and are obstacles which have to be overcome in order to progress further in the VE
  • Routes implicit or explicit draw visitors round a VE and seek to make sure that all major content is found and made use of.

Attractors should draw attention to sites of retainers and choice points etc. and, if properly designed, lead visitors around the world in a meaningful way using connectors. From the initial entry point the visitor is drawn to the complex of buildings in the centre of the frame. This is reinforced shortly afterwards (10 seconds) by the cliff lift cars moving up and down the tramway. As the visitor appraches the buildings, some steps up to a gap in the railings form a strong attractor, However, at about the same time the dark blobs near the shoreline become recognisable as birds and we have a classic choice point. Once on the promenade the competing attractors of the open doorways to the amusement arcade and ticket office form a powerful choice point.

Attractors may also themselves be retainers. Seen from a distance an animated object may act as an attractor (the cliff lift cars), but when experienced close up the object may be some sort of vehicle to ride in and control, thus becoming a retainer. Retainers are, in fact, patterns of attractors, connectors and retainers, which may be quite localised and, in effect, work as games. The oystercatchers avoiding visitors work in this fashion. They constitute a game in which the visitor chases them to find where they will go next. After a while the visitor realises that the birds always fly to one of three places on the beach and the little game is over, or at least poses no further surprises.

The attractor graph (Figure 18) shows the way POs can be used to construct routes through a VE. There are no challenge points as such in this VE.


Figure 17: View of virtual Saltburn

There are various means of documenting surprises as part of the design process:

  • tables of suprises
  • attractor graphs.

A partial table of surprises for the cliff lift looks like this:

Attractor Connector Retainer
Animated cars at distance Brake controls Operate cars from anywhere in world (outside cars)
Peripatetic
Animated cars at distance Pier and promenade railings, doorways of amusement arcade and partial views of ticket office etc. form axes to the car at the bottom of the cliff, alignment of pier and promenade Ride up and down in the cliff lift 
Local
Unusual red and white pattern on amusement arcade, and partial views through entrance ways Pier and promenade railings, alignment of pier and promenade form axes towards amusement arcade Interactive soundscape inside amusement arcade 
Local
Shore birds on beach Basic navigation controls The bird always flies away when you get too close, you can chase it up and down the beach 
Local
Example of a game as nested pattern of surprises
Partial views of path up/down (denoted by white railings up/down cliff) Basic navigation controls Climb up/down cliff with additional views of seafront etc.
Local

Table 8: Surprises for the cliff lift

We can see that rows one and two constitute a choice point two different goals from the same attractor. The representation of the shore birds attractor at different points on the beach documents a retainer. Retainers can be quite unprepossessing, for instance the path up/down the cliff. Such tables can be constructed in more detail by replacing retainers with individual rewards.

Another way of documenting perceptual mapping is by means of attractor graphs. These quite literally seek to establish the VE to be built as a comprehensible labyrinth of attractors as visible from other attractors. A partial example of such graph for the cliff lift is given in Figure 18 where the numbered nodes in the graph stand for the following attractors:

  1. Buildings at foot of cliff, complex, mysterious
  2. Cliff lift cars moving, active, mysterious
  3. Shorebirds on beach
  4. Ticket office with open doorways, complex, mysterious
  5. Amusement arcade with open doorways, mysterious
  6. Zigzag white lines of path up/down cliff, mysterious, complex.

The nodes in boxes represent retainers that could also be expanded as attractor graphs. The 'Operate cars' retainer has no links because it is peripatetic and can be undertaken almost anywhere and in conjunction with all the other retainers.


Figure 18: A partial attractor graph for virtual Saltburn

The advantage of attractor graphs is that they allow us to consider the main routes through a VE and the design of effective attractors to achieve this. It is also where major choice points should occur as these will be represented by nodes with more than one arrow leading away from them.

Perceptual mapping has much in common with the way painters arrange the composition of a work so as to catch the viewer's attention and lead it around the canvas in a particular way. Although it is not possible to tell a story in a VE in the same way as in a film or TV programme, there is nevertheless an important narrative element to VEs which needs to be built into the design (Fencott 2001). This refers to the purposive accumulation of experience. This is more obvious in 3-D games or Virtual Training Environments (VTEs) such as the classic Hubble Space Telescope VTE used for training the flight team (Loftin and Kenney 1993).

Sureties

Sureties are mundane details that are somehow highly predictable their attraction is their predictability. They arise directly from the architecture of the space and are concerned with the logic of the environment unconsciously accepted.

The following quote gives an insight from photography into the nature of sureties in VEs:

Hence the detail that interests me is not, or at least is not strictly, intentional, and probably must not be so, it occurs in the field of the photographed thing like a supplement that is at once inevitable and delightful. (Barthes 1984)

Sureties are about small things. Navigation for instance, lampposts, incidental furniture. This is because sureties for distance, as people would normally recognise them, are largely absent in VEs. This is also true for the scale of objects and one's own avatar. Space should not be static and sterile but dynamic and messy we are used to the real world being like this so it helps if virtual ones are as well (VEs and mess/clutter don't however go naturally together). A useful aphorism is that in interacting with the real world we are trying to make sense of too much information but that in interacting with VEs we are trying to make sense of too little.

Sound is an important spatial surety in reality. It gives important information about the nature and scale of the space we are currently experiencing, i.e. small, large, inside, outside, etc. We are very susceptible to reflected sound as sureties in this sort of way. We are not very good at locating objects accurately in 3-D space based on the sounds emanating from them. The nature of sound in VEs means that sound can be used for atmosphere etc. but not as well for spatial and directional cues. This depends on the nature of the sound system itself being used, i.e. stereo, surround sound. Sureties are thus concerned with:

  • vection
  • ego scale
  • perceptual noise
  • distance
  • limits.

Some examples drawn directly from the cliff lift are:

  • sound of the cars moving over the metal tracks to reinforce travelling motion
  • railings around the pier, the promenade and cliff tops to indicate avatar scale, vection and to suggest where to go etc. (It is worth noting the fact that none of the railings modelled bears more than a passing resemblance to those actually in place in Saltburn. Comparing Figure 17 above to the photograph of the pier should convince the reader of that. The actual and the virtual do not match. The important point is that there should be railings where railings are expected.)
  • similarly the actual structure of the pier supports are quite complex but only the outer angled supports are modelled in the VE
  • eaves on the amusement arcade and ticket office etc. which, as Figure 19 shows, do not resemble each other yet seem to have passed the scrutiny of visitors to date. Texture files have been scanned at very low resolution but this does not seem to have raised comments among the students who visited the site

Figures 19a and 19b: Real and virtual eaves

  • eaves, railings, pier supports, etc. all add perceptual noise in addition to their primary design justifications.

Some elements have been either modelled very simply or not modelled at all:

  • The sea is a single blue polygon that functions to signify 'sea' rather than model it with graphical accuracy. This is both because of the computational overhead of a more detailed and/or dynamic representation and the risk that such a representation that did not work would draw attention to its inadequacies see shocks below.
  • There are no cables connecting two cars together. Nobody has ever commented on this; what should be a shock goes unnoticed.

Sureties and surprises working together

Sureties and surprises in VEs work together in much the same way jokes do:

My dog has no nose!
How does he smell?
Terrible!

The first two lines are unremarkable and mundane, sureties. The third line comes as a surprise but is plausible from the logic of the first two statements. Jokes all seem to be much like this you set up an imagined and consistent, however fantastic, world and then give it a bizarre, implausible twist which must somehow be derivable from the former. Sureties and surprises in VEs work together, supporting each other and thus the virtuality they inhabit by seeking to catch and retain the attention of the visitor and thus maintain presence and belief. Sureties are the means by which a perceptual map is grounded, virtually, in a believable world.

In the context of the cliff lift we have:

  • the empty interior of the amusement arcade, filled with appropriate sounds but no objects, signals the use of the building
  • the fact that sounds, controls, animation, and vection come together in the cliff lift cars themselves establishes that they are the focus of the VE, particularly in relation to the partially modelled amusement arcade.

It is worth pointing out that many objects will provide a variety of perceptual opportunities as both surprises quite possibly several depending on the context and sureties.

Shocks

Shocks are not perceptual opportunities normally built into VEs but arise as by-products of the design and construction process. They give rise to perceptions that jar, that aren't received as expected in the established context of the VE. They draw attention to the mediated nature of the environment and thus undermine presence. Shocks are thus perceptual bugs which need to be actively sought out and eliminated.

In earlier versions of the cliff lift there were a variety of shocks such as the sea not being big enough so that visitors could see where it ended. The static nature of the sea as currently modelled is also something of a shock.

Conclusions

Despite being generally well received, there are unsolved problems with virtual Saltburn. Communicating the meaning and role of the lift control buttons at the bottom of the VE window has never been achieved. The surfing elements have never been incorporated due to the problem of making them perceptually convincing. The simplicity of the current representation of the sea is related to this. Virtual Saltburn is an empty place, devoid of the bustle of human activity that is the hallmark of the real Saltburn particularly on a hot summer's day. The increased power of even 'entry level' home computers means that many of these problems could now be solved if time permitted.

The perceptual opportunities model is not meant to be a prescriptive design method that tells designers what to do. It is a technique to help designers focus on achieving effective user participation and a coherent set of virtual experiences as a means of delivering purpose. It is also not the intention that designers should spend their time drawing complete attractor graphs or extensive tables of surprises. Once they are aware of the  role of the various perceptual elements and the structures that can be built from them choice points and retainers for instance they can incorporate them into the general task of 'world building'. Over the last six years, student project work has shown this approach to be very effective in focusing minds onto the design of user involvement prior to 3-D graphical modelling and low-level scene graph construction. The result is that students build worlds that work effectively and coherently as virtual environments.

 
 
Kate Allen, Tony Austin, Rachael Beach, Aaron Bergstrom, Sally Exon, Marc Fabri, Clive Fencott, Kate Fernie, Michael Gerhard, Catherine Grout, Stuart Jeffrey, Learning Sites, Anthony McCall, Mike Pringle, Julian Richards, Damian Robinson, Nick Ryan, Melissa Terras 2002

All material supplied via the Arts and Humanities Data Service is protected by copyright, and duplication or sale of all or part of any of it is not permitted, except that material may be duplicated by you for your personal research use or educational purposes in electronic or print form. Permission for any other use must be obtained from the Arts and Humanities Data Service. Electronic or print copies may not be offered, whether for sale or otherwise, to any third party.
 
BibliographyNextGlossaryBackContents