Sound Source Localization - Essay Example

Ubiquitous Computing: Sound Source Localization Partha kar Recent research in ubiquitous computing has focused both on how to infer human activity from a variety of signals sensed in the environment as well as how to use that information to support interactions. In this paper, we examine the feasibility and usefulness of sound source localization (SSL), which is an implicit location system to support monitoring of a remote space as well as to infer key activities. The design and performance description of three architectures and corresponding protocols that use a variation of the Time-of-Flight method for localization three different levels of devices, namely 802.11-enabled PDAs, 3G cell phones and PDAs without network connectivity. CONTENTS Lord Raleigh's Duplex Theory (1907): 11 Head Related Transfer Function (HRTF) 11 Woodworth's Model: 12 Kuhn's Model 12 Larcher & Jot's Model 12 Duda's Ellipsoidal Head Model 12 User's Perspective 15 Service's perspective: 20 Introduction Ubiquitous computing is a model of computing in which computer functions are integrated into everyday life, often in an invisible way. The model requires both small, inexpensive computers and wired and wireless ("dumb") devices connected to larger computers. A household controlled by ubiquitous computing might have remote-controlled lighting, automated sprinklers, a home entertainment center, devices to monitor the health of occupants, and a refrigerator that warns occupants about stale or spoiled food products. The proponents of ubiquitous computing envision a progression in computing functionality from the primacy of desktop computing, with its focus on programming and publishing, to an age of "natural" computing, wherein computers are accepted and utilized in all aspects of work and leisure. Rapid changes in technology, combined with an increasingly mobile society, ensure that the average person is continually challenged to use unfamiliar electrical and mechanical devices. This requires that devices operate in accordance with the intuition of the user, and serving that intuition requires computing power. Ubiquitous computing is, therefore, (arguably) not a dream in need of pursuit, but a predictable outgrowth of technical solutions to societal trends. Modern devices that may serve the ubiquitous computing model include mobile phones, digital audio players, radio-frequency identification tags and interactive whiteboards (Greenfield, 2006 & Hansmann, 2003). Context aware computing is one of the main themes in ubiquitous computing. In practice, certain types of context, such as location, identity, time and activity are more important than others (Dey, 2000). Since the early 1990's much research effort has been Focused on how to acquire, refine, and use location context information (Hightower, 2001). Traditional location-sensing systems rely on either explicit or implicit means of localization. In explicit localization, the user must wear or carry a device or tag which is used to locate them, while implicit localization systems do not require instrumenting the user. Most implicit localization systems use computer vision to track users. We are interested in the use of sound source localization, arguing that understanding the location of sound sources can be valuable for context aware computing. Sound events are often associated with human activities, but little effort in the ubiquitous computing community has tried to exploit this. There are social concerns when sensing video and audio in the home environment. When the actual information retrieved is not the rich signal that a human would see or hear, there is potential for alleviating those concerns. We designed a sound source localization (SSL) system which locates sound events in the environment using microphone arrays (Xuehai Bian and Gregory D. Abowd). The only information extracted in this case is solely the location of sound sources. Our system is based on a standard SSL algorithm which uses the time of delay method and PHAse Transform (PHAT) filtering in the frequency domain to locate sound sources (Knapp, 1976). The system runs continuously (24/7) and feeds the detected sound events into a database which is shared with other applications. 1. Sound source localization: An Introduction Localization refers to our ability to place a sound source in space. We use a number of audible cues for this purpose. One such cue is the interaural time difference or ITD. This refers to the difference in time it take a sound to reach one ear compared to the other. Sounds located directly in front of or behind us will reach both ear simultaneously. If the angle of the source is moved until the difference is greater than 20 microseconds (millionths of a second), a difference in location can be perceived. As a source moves more directly to one side of your head or the other, our ability to discriminate its location using the ITD method diminishes somewhat. A second mechanism, called the interaural intensity difference or IID uses the difference in amplitude caused by the head physically masking sounds coming from one side or the other. Because lower frequencies with longer wavelengths refract more easily around objects, this mechanism is more effective for higher frequencies. The shape of the pinna (outer ear flap) also filters frequencies depending on their angle of incidence. The pinna is also responsible for our ability to place sounds in the vertical plane using this filtering mechanism. Try folding your ear flap over and see how well you can still place sounds. Sound waves reflecting off the shoulder also provide some location cues. All of these mechanisms are ineffective below approximately 270 Hz, as witnessed by the often out-of-the-way placement of subwoofers in surround sound setups. A phenomenon to keep in mind when placing loudspeakers is the precedence effect, in which a listener receiving the same signal from multiple speakers will place it at the closest speaker and not in between unless the time difference between the signals is less than a millisecond. This is why you should try to sit in a central location at an electronic music concert! In judging the apparent size of an acoustic space, the aural cues depend on many factors, including the time elapsed from hearing the source sound to hearing the earliest reflections, the onset of reverberation, the intensity and duration of reverberation, diffusion of high frequencies and the resonant frequencies of the reverberation. With multi-channel sound and control over artificial reverb, many interesting and novel spatial effects can be created (Hass). The importance of sound localization are as follows: 1. Hearing Provides info About our Environment 2. Spatial sounds give detailed info of our surroundings Determine direction and distance to objects 3. Warn of approaching dangers e.g. predators 4. Unlike vision, hearing is omni -directional 5. Can hear in complete darkness! 6. Can guide the more "finely tuned" visual attention 7. Eases the burden of the visual system 2. Sound Localization Functions and Theories Lord Raleigh's Duplex Theory (1907): Assumes a spherical head with no pinnae ITD: difference in time between arrival of sound at each ear - Low frequencies: < 1500Hz ILD: intensity difference between sound at each ear - High frequencies:>1500Hz Shortcomings of the Duplex Theory Front-back Ambiguities Cone of confusion Head Related Transfer Function (HRTF) Filtering of sound spectrum by interactions of sound with head, torso and particularly pinna Pinna: Series of grooves and notches which accentuate or suppress mid & high frequency components in a position dependant manner Each person's pinna differs filtering effects differ. Woodworth's Model: ITD for sound located on azimuth plane Assumes spherical head model, no pinnae Valid for high frequency signals Kuhn's Model ITD for sound located on azimuth plane Based on ITD values obtained from "dummy" head Separate expression for high and low frequencies Larcher & Jot's Model Spherical head model to predict ITD values Accounts for source elevation Duda's Ellipsoidal Head Model 5 parameters measured from listener's head 3. SSL System Architecture and Protocols The following are the SSL architectures and protocols used in general (Lopes): 1. WLANBeep: It assumes that the roaming device is equipped with 802.11 wireless LAN technology. The signal between the device and the sensors includes both audio and 802.11 radio messages. The components of this architecture include: The wireless enabled PDA as the user's roaming device The sensors The WLAN as the communication infrastructure among all components The localization server and The web server which can generate location dependent web pages when it receives a location aware URL from the roaming device. 2. 3GBeep: It assumes that the user's device is a third generation or GPRS mobile phone. In this architecture the user's mobile phone must have access to the internet through the wireless carrier's 3G network, hence making the localization server and the web server accessible to the mobile phone. The communication between the localization server and the sensors is carried out through the WLAN. 3. PureBeep: It assumes no network connectivity on the part of the roaming device. The goal is to inform the system of device's location and not necessarily to send location dependent information back to the device. 4. Ubiquitous Computing: A Technology driven by SSL Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives. Alan Kay of Apple calls this "Third Paradigm" computing (See: http://www.ubiq.com/hypertext/weiser/UbiHome.html). Mark Weiser is the father of ubiquitous computing. Ubiquitous computing is roughly the opposite of virtual reality. Where virtual reality puts people inside a computer-generated world, ubiquitous computing forces the computer to live out here in the world with people. Virtual reality is primarily a horse power problem; ubiquitous computing is a very difficult integration of human factors, computer science, engineering, and social sciences. 5. Issues on Technology Adaptation It can be divided into the following: User's Perspective As user satisfaction is crucial to mobile network operators, user's security requirements must be regarded as one of the most important issues for them. Customers will be happy if they can get the service that they want, almost anywhere at any times in a good price and is of the required quality. On the other hand, user's security requirements should be fulfilled - this is a basic prerequisite to mobile network operators. Any compromise of security that has an effect on user's assets may finally turn out to be a serious problem for mobile network operator's business. User's requirements of security on systems B3G can be categorized as: i. Terminal security: obviously mobile terminal is an important asset of user, requirements on terminal security include: access control: a mobile terminal can be activated and used only by authorized user; virus-proof: terminal should be protected against virus, and network worms, etc.; theft-prevention: stolen terminals should be blocked to access networks; ii. communication and data privacy, includes security of voice and data communications privacy of location, call setup information, user ID, call pattern, etc. service usage privacy: unauthorized partners must not know which services are used by any specific user, its usage pattern and volume, etc. iii. service provision security service availability should be ensured to prevent or mitigate Denial of Service (DoS) attack security against fraudulent service providers e-commerce/m-commerce security To secure user's assets is part of the goal of security architecture of Beyond 3G (B3G) systems. This is a task of, not only, mobile network operators but also manufactures, network operators but also manufactures, regulatory bodies and other participants in future communication environment. Because of competition, mobile network operator will usually enhance its service provision (capacity, quality, variety, etc. of services) to satisfy its customers, possibly by cooperating with other business partners that can be service providers or other network operators. A B3G mobile network will also be more open to 3rd parties that provide services independently. Appropriate security mechanisms and services are required which will be explained in following sections. Security mechanisms and services should also be intelligible and easy to use. Besides the direct requirements from users, some requirements which are related to operation stage, should be considered in system design, too. Security mechanisms should be either transparent to users, or sufficiently usable, i.e. without any difficulty or inconvenience for users. Simple operation is much preferable to users otherwise the mechanism may fail because of user's reluctance to use it. Security mechanisms should not impair service quality apparently, otherwise business competence will be hindered and more important, the mechanisms may be bypassed. Giving users chance to choose between different levels of security through technical configuration may not be a good idea, because inappropriate configurations, which can happen for large number of users, may be exploited by attackers, and normally leads to serious influence on marketing. On the other hand, customer support, like education and consulting are very expensive. Service's perspective: Service and contents to the user can be provided by mobile network operator service providers content providers other businesses or site owners Some of the security requirements that arise here from mobile network operators' point of view are: Service should be provided to the specified set of users (authentication), according to the contractual(authentication), according to the contractual obligation agreed (authorization) and the usage should be accountable. Rogue service or content providers can appear and methods must be developed to deter them. Secure access to services, from any partner, should be provided. The operator should take care that the service providers are correctly charged or if the service provider is paying the operator then the operator should take care that he bills the service provider correctly. Since cooperation with many other service providers are expected in the future communication and service provision systems, non-repudiation will be much important between operators and service providers to prevent and combat frauds. However, appropriate business models may be more efficient than technology means. In future the number of network operators will increase and thus a service that can be provided is openness towards each other which will create positive perception for users. Having openness brings forward several security requirements, level and building trust relationship being one of them. 6. Conclusion To summarize the requirements, the following points, from my point of view, should be carefully treated in future research. provide seamless mobility over heterogeneous networks with sufficient security but no apparent performance compromise mobility vs. location privacy anonymity vs. accountability ensure that service provided to users through 3rd parties are trustful, because user will most possibly complain to mobile network operator when they have a problem special terminal features and re configurability vs. security: user may buy mobile devices directly from vendors instead of from operators, the issue here is security for heterogeneous devices Last but not least we should not forget that, human being and software bugs can be the weakest link in security. References Dey, A.K., Providing Architectural Support for Building Context-Aware Applications, in College of Computing. 2000, Georgia Institute of Technology. Greenfield, Adam (2006). Everyware: the dawning age of ubiquitous computing. New Riders, p.11-12. ISBN 0321384016. Hansmann, Uwe (2003). Pervasive Computing: The Mobile Word. Springer. ISBN 3540002189. Hass, Jeffrey. See: http://www.indiana.edu/ emusic/etext/acoustics /chapter1_ localization.shtml Hightower, J. and G. Borriello, Location Systems for Ubiquitous Computing. Computer,IEEE Computer Society Press, 2001. 34(8): p. 57-66. Knapp, C.H.a.C., G. C., The generalized correlation method for estimation of time delay,''. IEEE Trans. Acoust., Speech, Signal Process. ASSP-24, 1976. 24: p. 320-327. Lopes V. Cristina,Ahaghigh Amir.Localization of Off-the Shelf Mobile Devices Using Audible sound:Architecture, Protocols and Performance Assessment. Xuehai Bian, Gregory D. Abowd, James M.Using Sound Source Localization to Monitor a and Infer Activities in the Home Rehg GVU & College of Computing, Georgia Institute of Technology. Read More