A PASSENGER

Artificial Passenger

CHAPTER 1

INTRODUCTION

Artificial intelligence is the intelligence exhibited by machines or software. It is an

academic field of study which studies the goal of creating intelligence. Major AI researchers

and textbooks define this field as "the study and design of intelligent agents”, where

an intelligent agent is a system that perceives its environment and takes actions that

maximize its chances of success. John McCarthy, who coined the term in 1955, defines it as

"the science and engineering of making intelligent machines"[1].

AI research is highly technical and specialized, and is deeply divided into subfields

that often fail to communicate with each other. Some of the division is due to social and

cultural factors: subfields have grown up around particular institutions and the work of

individual researchers. AI research is also divided by several technical issues. Some

subfields focus on the solution of specific problems. Others focus on one of several

possible approaches or on the use of a particular tool or towards the accomplishment of

particular applications [1].

Artificial intelligence has been the subject of tremendous optimism but has also

suffered stunning setbacks. Today it has become an essential part of the technology industry,

providing the heavy lifting for many of the most challenging problems in computer science.

Thinking machines and artificial beings appear in Greek myths, such as Talos of Crete, the

bronze robot of Hephaestus, and Pygmalion's Galatea. Human likenesses believed to have

intelligence were built in every major civilization: animated cult images were worshiped in

Egypt and Greece and humanoid automatons were built by Yan Shi, Hero of Alexandria and

Al-Jazari. It was also widely believed that artificial beings had been created by Judah Loews

and Paracelsus. By the 19th and 20th centuries, artificial beings had become a common

feature in fiction, as in Mary Shelley's Frankenstein or Karel Capek's R.U.R. (Rossum's

Universal Robots).Pamela McCorduck argues that all of these are some examples of an

ancient urge, as she describes it, "to forge the gods". Stories of these creatures and their fates

discuss many of the same hopes, fears and ethical concerns that are presented by artificial

intelligence [1].

Mechanical or "formal" reasoning has been developed by philosophers and

mathematicians since antiquity. The study of logic led directly to the invention of the

programmable digital electronic computer, based on the work of mathematician Alan Turing

Department of Information Technology, SSBT’s COET, Bambhori, Jalgaon (MS) 1


and others. Turing's theory of computation suggested that a machine, by shuffling symbols as

simple as "0" and "1", could simulate any conceivable act of mathematical deduction. This,

along with concurrent discoveries in neurology, information theory and cybernetics, inspired

a small group of researchers to begin to seriously consider the possibility of building an

electronic brain [1].

An artificial passenger is a device that would be used in a motor vehicle to make sure

that the driver stays awake. IBM has developed a prototype that holds a conversation with a

driver, telling jokes and asking questions intended to determine whether the driver can

respond alertly enough. Assuming the IBM approach, an artificial passenger would use a

microphone for the driver and a speech generator and the vehicle's audio speakers to

converse with the driver. The conversation would be based on a personalized profile of the

driver. A camera could be used to evaluate the driver's "facial state" and a voice analyzer to

evaluate whether the driver was becoming drowsy. If a driver seemed to display too much

fatigue, the artificial passenger might be programmed to open all the windows, sound a

buzzer, increase background music volume, or even spray the driver with ice water. Studies

of road safety found that human error was the sole cause in more than half of all

accidents .One of the reasons why humans commit so many errors lies in the inherent

limitation of human information processing .With the increase in popularity of Telemetric

services in cars there is more information that drivers need to process and more devices that

drivers need to control that might contribute to additional driving errors. This topic is

devoted to a discussion of these and other aspects of driver safety [2].

The AP is an artificial intelligence based companion that will be resident in software

and chips embedded in the automobile dashboard which shown in Figure: 1.1. The heart of

the system is a conversation planner that holds a profile of you, including details of your

interests and profession. A microphone picks up your answer and breaks it down into

separate words with speech recognition software. A camera built into the dashboard also

tracks your lip movements to improve the accuracy of the speech recognition. A voice

analyzer then looks for signs of tiredness by checking to see if the answer matches your

profile. Slow responses and a lack of intonation are signs of fatigue. This research suggests

that we can make predictions about various aspects of driver performance based on what we

glean from the movements of a driver’s eyes and that a system can eventually be developed

to capture this data and use it to alert people when their driving has become significantly

impaired by fatigue [4].



Figure 1.1: Automobile Dashboard

The natural dialog car system analyzes a driver’s answer and the contents of the

answer together with his voice patterns to determine if he is alert while driving. The system

warns the driver or changes the topic of conversation if the system determines that the driver

is about to fall asleep [3]. The system may also detect whether a driver is affected by alcohol

or drugs. If you reply quickly and clearly, the system judges you to be alert and tells the

conversation planner to continue the line of questioning. If your response is slow or doesn’t

make sense, the voice analyzer assumes you are dropping off and acts to get your attention

the system, according to its inventors, does not go through a suite of rote questions

demanding rote answers. Rather, it knows your tastes and will even, if you wish, make

certain you never miss Paul Harvey again. This is from the patent application: An even

further object of the present invention is to provide a natural dialog car system that

understands content of tapes, books, and radio programs and extracts and reproduces

appropriate phrases from those materials while it is talking with a driver. For example, a

system can find out if someone is singing on a channel of a radio station .The system also

includes a recognition system to detect who is speaking over the radio and alert the driver if

the person speaking is one the driver wishes to hear. Just because you can express the rules

of grammar in software doesn’t mean a driver is going to use them. The factor engineering

for example, people using different strategies to talk. In this manner, the individual is guided

to talk in a certain way so as to make the system work.e.g. “Sorry, I didn’t get it. Could you

say it briefly?” Here, the system defines a narrow topic of the user reply via an association of



classes of relevant words via decision trees. The system builds a reply sentence asking: What

are most probable word sequences that could follow the user’s reply [4].

1.1 Voice Control Interface

According to Dmitri Kane sky, a former IBM researcher, currently at Google, The

Artificial Passenger was developed using the Conversational Interactivity for Telemetric

speech system which counts on the driver's natural speech instead of the use of hands [6].

The CIT relies on a Natural Language Understanding system that is difficult to develop

because of the low-powered computer systems available inside cars. IBM suggests that this

system be located on a server and accessed through the cars' wireless technologies. The CIT

system includes another system called the Dialog Manager. The DM takes the load of the

NLU system by interacting with the vehicle, the driver, and external systems such as weather

systems, email, telephones and more [7].

The NLU system receives a voice command from the driver and looks through a file

system to come up with an action to be performed and executes that action .The DM works

with questions asked by the driver such as “How far is The Gallatin Field Airport from

here?” The NLU system will still not be able to understand everything a driver says. Reasons

for that are the different idioms and dialects of different regions. IBM is working on

developing a system that recognizes where the driver is and acknowledge the regional

diction used in that area [7].

Another system used within this technology is the Learning Transformation system

which monitors the actions of the occupants of the car and of the cars around it, learns

patterns within the driver’s speech and stores that data, and learns from such data to try to

improve the performance of the technology as a whole [6].

1.2 Speech Recognition

The speech recognition process relies on three steps. The front-end filters out any

unwanted noise such as noise from the car, background music, or background passengers. It

gets rid of all low energy and high variability signal being recognized [7]. The labeller

breaks apart the speech and searches in a data base to recognize what is being said. It starts

broad by seeing what subject the driver is speaking of. Then goes into more details of what

the driver is truly asking. The decoder next takes all this information and formulates a

response to the driver [6]. IBM states through much experimentation that the speech



recognition is very accurate but the process has not fully been refined and still has kinks

within it [7].

The main part of the Artificial Passenger is the Disruptive Speech Recognition. This

technology keeps a conversation with the driver and analyzes what the driver is saying and

how s/he is saying it. It can recognize fluctuations in the driver’s voice to determine if the

driver is sleepy, upset, or in a good mood through different vibration patterns in the driver’s

speech. It also record the time it takes for a driver to respond in the conversation and from

that determine if the driver is nodding off or being distracted by something[7].

1.3 Driver Drowsiness Prevention

When the computer recognizes that the driver is dozing off, it sends a signal to

interfere. The computer will step in by changing the radio, trying to play games with the

driver, or by opening window to wake the driver up [5]. The computer wants to improve their

alertness by doing these. If it finds that the driver is nodding off over and over, the computer

system is programmed to ask to call a nearby hotel and book a room or suggest the driver

take a break [6].

The Artificial Passenger will try to read jokes, play games, ask questions or read

interactive books to stimulate the driver. Drivers that show more drowsiness will be given

content that is more stimulating than a driver who is not as drowsy [6].

1.4 Distributive User Interface Between cars

IBM recognizes that there are more dangers to a driver than him/herself. Artificial

Passenger is proposed to work between cars by relaying information to one another. The

information could include driving records to show if they have a history of being a bad

driver or on-time analysis of all drivers to show which ones are becoming drowsy and can

interfere through this information. It can also show if a driver is being distracted by games or

wireless devices and interfere with all surrounding drivers[7].

1.5 Definition: Artificial Passenger

The AP is an artificial intelligence based companion that will be resident in software

and chips embedded in the automobile dashboard.

It is a natural language E-companion.

It is a sleep prevention device in cars to overcome drowsiness.

It is a life safety system.



Detects alarm condition through sensors.

Broadcasts pre-stored voice messages over the speakers.

Captures images of the driver.

According to the national survey in UK and USA, it is observed that driver fatigue annually

causes 100,000 crashes,

15000 deaths

71000 injuries

Which cause annual cost of $12.5 billion

A majority of the off- road accidents observed were preceded by eye closures of one half

second to as long as 2 to 3 seconds. A normal human blink lasts 0.2 to 0.3 second.



CHAPTER 2

LITERATURE SURVEY

Table 2.1: Literature survey of Artificial Passenger

Sr No. Year Description

1. 1950

Dr. Jones and Milton developed Eye-tracking hardware is

either mounted on a user's head or mounted remotely. Both

systems measure the corneal reflection of an infrared light

emitting diode (LED), which illuminates and generates a

reflection off the surface of the eye. This action causes the

pupil to appear as a bright disk in contrast to the

surrounding iris and creates a small glint underneath the

pupil [4].

2. 1991

Cognitive psychologists have used remote eye-tracking

systems to study the relationship between cognitive

scanning styles and search strategies (Crosby and Peterson)

[6]. Such eye-tracking studies have been used to develop

and test existing visual search cognitive models. More

recently, human- computer interaction (HCI) researchers

have used remote systems to study computer and Web

interface usability.

3. 2002

Dr. Kanesky led the Conversational Interactivity for

Telematics project that was based on his invention of

Artificial Passenger. Dr. Kanevsky also successfully

applied mathematical methods to speech recognition. His

work on discriminative algorithms allowed the

introduction of a new class of training models which

significantly improved the accuracy of various speech

recognition systems[5].

4. 2003

Remote systems measure the orientation of the eye relative

to a fixed unit such as a camera mounted underneath a

computer monitor. Because remote units do not measure



the pupil glint from multiple angles, a person's head must

remain almost motionless during task performance.

Although head restriction may seem like a significant

hurdle to overcome, Jacob and Karn attribute the

popularity of remote systems in usability to their relatively

low cost and high durability compared with head- mounted

systems [5].



CHAPTER 3

ARCHITECTURE AND WORKING

3.1 Architecture

Figure 3.1: General Architecture

3.1.1 Microphone

As shown in Figure 3.2, The Microphone is used for picking up the words and separate them

by some internally used software for conversation.


Artificial

Passenger

MicrophoneTemperature

indicator Server

Network

Speaker

Odor sensor Key

Door lock

Camera


Figure 3.2: Microphone

3.1.2 Camera

This will track the lips movement of the driver and also used for the improvement for the

accuracy of the speech recognition [3].

■ The following Figure 3.3 show the Camera which track the head and lip movements of the

driver [3].

1. Used for the improvement of accuracy of the speech recognition system.

2. Calculations vary for day and night conditions.3. Night time is determined based on the following conditions:

4. If camera clock is between 18.00 hrs and 07.00 hrs.

5. If the intensity of image is lower than the threshold value.6. If the night time driving headlight is on.

Figure 3.3: Camera



3.1.3 External service provider

Linked to the dialog system by wireless network system coupled with

Car media, driver profile, conversational planner.

Driver analyzer module

It controls interruption of a dialog between the driver and the car dashboard.

3.1.4 Temperature indicator

This component is used to measure the temperature inside the vehicle and it also helps in

maintaining the steady temperature.

1. To analyze temperature of vehicle.

2. In combination with camera, the AP determines whether a child or pet is left in a vehicle

that is beginning to get very hot or cold [4].

3. It sends a message to the owner/driver and takes the correct actions.

4. The AP is able to analyze the situation & executes a corrective action.

5. Opens window or door to allow temperature to moderate.

6. Allow child or pet to leave the vehicle after informing the driver /authorities.

3.1.5 Door lock sensor

This sensor alarm when the door is not locked.

1. Through communication system the owner can open a vehicle door remotely & let the

person out who has been locked.

3.1.6 Odor sensor

This sensor will periodically sprinkle the sweet air inside the vehicle.

1. AP informs the driver/owner about this.

2. AP uses odor detector as well as camera to detect whether groceries were left in the

vehicle.

3.1.7 Speaker

This generally used for the entertainment purpose.

1. Allows the computer to identify the words spoken on to a microphone.

2. Two ASRs used:



3. Speaker independent ASR – used for decoding voice signals of the driver.

4. Operates with voice car media and decodes tapes, audio book, telephone mails etc[3].



3.2 Working

Figure 3.4: Working of Artificial Passenger

A camera built into the dashboard also tracks your lip movements to improve the

accuracy of the speech recognition. Heart of the system is a conversation planner. A

microphone picks up answer and breaks it down into separate words with speech-recognition

software. A voice analyzer then looks for signs of tiredness by checking to see if the answer

matches your profile. Slow responses and a lack of intonation are signs of fatigue. The AP is

not an inflatable automaton that sits in the passenger seat. It's an intelligent presence packed

into the dashboard electronics. The heart of the system is a conversation planner that holds a

profile of you, including details of your interests and profession. When activated, the AP

uses the profile to cook up provocative questions such as, "Who was the first person you



dated?" via a speech generator and the in-car speakers. When the driver is not able to answer

his calls at that moment the mobile indicator automatically accept or decline calls according

to drivers wish [8]. As shown in the Fig 3.4 there are some of the components which support

for the working of the system.

Automatic speech recognition.

Natural language processor.

Driver analyzer.

Conversational planner.

Alarm

External service provider.

Eye tracker.

Camera.

3.2.1 Automatic speech recognition

There are two ASRs used in the system:

Speaker independent:

It will decode the driver voice and the decoded voice signals will output to natural language

processor.

Operates with a voice car media, decodes taps, audio books, telephone mails.

Decoding outputs of the ASR module is analyzed by intelligent text processor and it will

output data to conservational planner [4].

3.2.2 Natural language processor

Processes the decoded signal of textual data from ASR module, identifies semantic and

syntactic content of the decoded message, produces variants of responses and outputs this

data to a text input of the driver analyzer. Identifies related meanings from the contents of

the decoded messages Produces variant of responses. This output goes to the driver analyzer

as an input [5].

3.2.3 Driver analyzer

Receives the textual data and voice data from NLP and measure the time of response using a

clock. This time responses, includes about drivers alertness and it will output to the

conservational planner. This analysis is both objective and subjective. Conversational

Planner instructs the language generator to produce the response. If the driver is in perfect



condition CP instructs the language generator to continue the conversation otherwise it will

be instructed to change the conversation [5].

3.2.4 Conversational planner

This is generally referred as the heart of the system and it instruct the language generator to

produce the response. If the driver continues to be in a perfect condition, then conversational

planner instructs the language generator to continue the conversation otherwise the language

generator is instructed to change the conversation [5].

3.2.5 Alarm

If the conversational planner receives information that the driver is about to fall asleep then it

activates an alarm system.

3.2.6 External service provider

Linked to the dialog system by wireless network system coupled with

Car media, driver profile, conversational planner.

Driver analyzer module

It controls interruption of a dialog between the driver and the car dashboard.

3.2.7 Eye tracker

Collecting eye movement data required hardware and software specifically designed to

perform this function. In Fig 3.5 The Eye-tracking hardware is either mounted on a users

head or mounted remotely. Both systems measure the corneal reflection of an infrared light

emitting diode. This illuminates and generates a reflection off the surface of the eye. This

action causes the pupil to appear as a bright disk in contrast to the surrounding iris and create

a small glint underneath the pupils [6].



Figure 3.5: Eye Tracker



CHAPTER 4

ADVANTAGES AND DISADVANTAGES5.1 Advantages

Artificial Passenger is broadly used to prevent accident

If a driver seemed to display too much fatigue, the artificial passenger might be programmed

to open all the windows, sound a buzzer, increase background music volume, or even spray

the driver with ice water. Studies of road safety found that human error was the sole cause in

more than half of all accidents [3].

If the driver gets a heart attack or he is drunk it will send signals to vehicles nearby

about this so driver there become alert

The natural dialog car system analyzes a driver’s answer and the contents of the

answer together with his voice patterns to determine if he is alert while driving. The system

warns the driver or changes the topic of conversation if the system determines that the driver

is about to fall asleep. The system may also detect whether a driver is affected by alcohol

or drugs [3].

Maintains a database for accident investigation use

This technology is the Learning Transformation (LT) system which monitors the actions of

the occupants of the car and of the cars around it, learns patterns within the driver’s speech

and stores that data, and learns from such data to try to improve the performance of the

technology as a whole [3].

Open and close the window of a car automatically and also answer a call for you

AP determines whether a child or pet is left in a vehicle that is beginning to get very hot or

cold. It sends a message to the owner/driver and takes the correct actions The AP is able to

analyze the situation & executes a corrective action Opens window or door to allow

temperature to moderate Allow child or pet to leave the vehicle after informing the driver

/authorities[4].

Fast Tracker

Now comes IBM (IBM) with a high-tech innovation that makes those gadgets seem, well, so

last-millennium. In May, the company patented Artificial Passenger (AP) a device designed

to substitute for a yakking spouse or a talkative friend in the passenger seat. IBM hopes to

see its invention in most new cars as soon as three years from now. It will converse with the

driver -- about family matters, the weather, or the outcome of the latest Mets game. If you

don't answer quickly enough, AP might turn on the radio. If that fails, the device will start



telling jokes from your favourite Web site. If there is still no response from the driver, AP

can open the car windows. Or it could ask for permission to call a nearby hotel and reserve a

room. There's also a last-resort option: It might sprite the driver with cold water [4].

5.2 Disadvantages

High cost.

Sensors in the front and rear of automobile don’t help if danger comes from sides.

Remote connection to server are not available everywhere, can have delays, and are not

robust.

Some users will produce some phrases that are not represented in collected data or in

grammar that are developed from this data [3].



CHAPTER 5

APPLICATIONS

Interface between neighbouring cars

Determines if a driver present a high safety risk for ex. falling asleep, tired inexperienced or

under the influence of alcohol and signal the car nearby to be careful of the driver [6].

Medical application

The system can monitor the driver and detect if they are sick, for ex. having a stroke or heart

attack. Enhanced, more reliable, and better understood than in the past, artificial passenger

systems can make providing healthcare more accurate, affordable, accessible, consistent, and

efficient. However, AP technologies have not been as well integrated into medicine as

predicted. In order to succeed, medical and computational scientists must develop hybrid

systems that can effectively and efficiently integrate the experience of medical care

professionals with capabilities of AP systems. After providing a general overview of

artificial intelligence concepts, tools, and techniques, Medical Applications of Artificial

Passenger reviews the research, focusing on state-of-the-art projects in the field. The book

captures the breadth and depth of the medical applications of artificial intelligence, exploring

new developments and persistent challenges [8].

Meet Japan's Earthquake Search-and-Rescue Robots

This robot may look like it's trying to eat a mannequin's face, but it's actually practicing for a

controlled rescue operation. The RoboCue, a rescue bot from the Tokyo Fire Department, is

designed to locate and safely retrieve victims from disaster sites specifically bomb sites, but

it could be useful for natural disasters as well. It uses ultrasonic sensors and infrared cameras

to locate trapped humans, and gently loads the injured person onto a cart to be transported to

safety. It even has an onboard oxygen canister [6].

Sleep prevention dialog based car system

An automatic dialog system capable of keeping a drive awake while driving during a long

trip or one that extends into the late evening. The system carries on a conversation with the

driver on various topics utilizing a natural dialog car system. The system includes an

automatic speech recognition module, a speech generation module which includes speech

synthesis or recorded speech, and possibly dynamically combined speech synthesizer and

recorded speech, and a natural language processing module. The natural dialog car system

analyzes a driver's answer and the contents of the answer together with his voice patterns to

determine if he is alert while driving. The system warns the driver or changes the topic of



conversation if the system determines that the driver is about to fall asleep. The system may

also detect whether a driver is affected by alcohol or drugs [8].

This system can also be used in other situations such as

• Security guard.

• Operators at nuclear plants.

• Pilots of airplane.

• Cabins in airplanes.

• Water craft such as boats.

• Trains and subways.

CHAPTER 6



FUTURE SCOPE

Future Implementation will provide us with shortest time routing based on road

conditions changing because of weather and traffic, information about the cars on the route,

destination requirement Information about the cars on the route Provides distributive user

interface between cars.

CONCLUSION



We suggested that such important issues related to a driver safety as controlling

Telematics devices and drowsiness can be addressed by a special speech interface. This

interface requires interactions with workload, dialog, event, privacy, situation and

other modules. We observed that an important application like Artificial Passenger can be

sufficiently entertaining for a driver with relatively little dialog complexity requirements

such as playing simple voice games with a vocabulary containing a few words. Successful

implementation of Safety Driver Manager would allow use of various services in cars

without compromising a driver safety.

BIBLIOGRAPHY



[1] Bach, Joscha, "Seven Principles of Synthetic Intelligence". In Wang, Pei; Goertzel, Ben;

Franklin, Stan. Artificial General Intelligence, Proceedings of the First AGI Conference. IOS

Press. pp. 63–74, 21 December 2006.

[2] Kanevsky,”Telematics: Artificial Passenger and beyond, Human Factors and Voice

Interactive Systems, Signals and Communications Technology Series”, Springer US, pp.

291-325.

[3] "U.S. patent: Sleep prevention dialog based car system", Issue 2300, July 2001.

[4] Eisenberg,”A passenger whose chatter is always appreciated by A” The New York

Times, December 27, 2001, pp.13-45.

[5] Kharif, Olg, "IBM to Drivers: Wake Up”.

[6] Paul Moskowtiz, Ronald Perez, Moninder Singh, Jung-Mu Tang, “Mobile Networks

and Applications” Volume9 issue 5, 2004.

[7] Kanevsky, Dimitri, "Telematics: Artificial Passenger and Beyond".

[8] [Online] Available: http://learnersbook.com/m/imagearticles/view/Artificial-Passenger.


Date post:	26-Nov-2023
Category:	Documents
Upload:	independent
View:	1 times
Download:	0 times

A PASSENGER

Documents