Multi-Agent Systems

1
Multi-Agent Systems

• University Politehnica of BucarestSpring
  2010Adina Magda Florea
• http//turing.cs.pub.ro/mas_10curs.cs.pub.ro

2
Course goals

• Multi-agent systems (MAS) may be viewed as a
  collection of distributed autonomous artifacts
  capable of accomplishing complex tasks through
  interaction, coordination, collective
  intelligence and emergence of patterns of
  behavior.
• By the end of this course, you will know
• what are the basic ideas, models, and paradigms
  offered by intelligent agents and MAS
• build multi-agent systems or select the right MAS
  framework for solving a problem
• use the agent technology in different areas of
  applications
• what do agents bring  as compared to distributed
  processing or object oriented software
  development.

3
Course content

• What are agents and MAS?
• Agent architectures
• Communication
• Knowledge representation
• Distributed planning
• Coordination
• Auctions
• Negotiation
• Agent oriented programming
• MAS learning
• Agents and web services
• Agents and MAS applications

4
Course requirements

• Course grades Mid-term exam               20
  Final exam                     30 Projects 
  30Laboratory                   
  20
• Requirements min 7 lab attendances, min 50 of
  term activity (mid-term ex, projects, lab)
• Academic Honesty Policy It will be considered an
  honor code violation to give or use someone
  else's code or written answers, either for the
  assignments or exam tests. If such a case occurs,
  we will take action accordingly.

5
Lecture 1 Introduction

• Motivation for agents
• Definitions of agents ? agent characteristics,
  taxonomy
• Agents and objects
• Multi-Agent Systems
• Agents intelligence
• Areas of RD in MAS
• Exemplary application domains

6
Motivations for agents

• Large-scale, complex, distributed systems
  understand, built, manage
• Open and heterogeneous systems - build components
  independently
• Distribution of resources
• Distribution of expertise
• Needs for personalization and customization
• Interoperability of pre-existing systems /
  integration of legacy systems

6
7
Agent?

• The term agent is used frequently nowadays in
• Sociology, Biology, Cognitive Psychology, Social
  Psychology, and
• Computer Science ? AI
• Why agents?
• What are they in Computer Science?
• Do they bring us anything new in modelling and
  constructing our applications?
• Much discussion of what (software) agents are and
  of how they differ from programs in general

7
8

• What is an agent (in computer science)?
• There is no universally accepted definition of
  the term agent and there is a good deal of
  ongoing debate and controversy on this subject
• The situation is somehow comparable with the one
  encountered when defining artificial
  intelligence.
• Why was it so difficult to define artificial
  intelligence (and we still doubt that we have
  succeeded in giving a proper definition) and
• Why is it so difficult to define agents and
  multi-agent systems, when some other concepts in
  computer science, such as object-oriented,
  distributed computing, etc., were not so
  resistant to be properly defined.
• The concept of agent, as the one of artificial
  intelligence, steams from people, from the human
  society. Trying to emulate or simulate human
  specific concepts in computer programs is
  obviously extremely difficult and resist
  definition.

8
9

• More than 30 years ago, computer scientists set
  themselves to create artificial intelligence
  programs to mimic human intelligent behaviour, so
  the goal was to create an artefact with the
  capacities of an intelligent person.
• Now we are facing the challenge to emulate or
  simulate the way human act in their environment,
  interact with one another, cooperatively solve
  problems or act on behalf of others, solve more
  and more complex problems by distributing tasks
  or enhance their problem solving performances by
  competition.

9
10

• It appears that the agent paradigm is one
  necessarily endowed with intelligence.
• Are all computational agents intelligent?
• The answer may be as well yes as no.
• Not to enter a debate about what intelligence is
• Agent more often defined by its characteristics
  - many of them may be considered as a
  manifestation of some aspect of intelligent
  behaviour.

10
11
Agent definitions

• Most often, when people use the term agent
  they refer to an entity that functions
  continuously and autonomously in an environment
  in which other processes take place and other
  agents exist. (Shoham, 1993)
• An agent is an entity that senses its
  environment and acts upon it (Russell, 1997)

12

• Intelligent agents continuously perform three
  functions perception of dynamic conditions in
  the environment action to affect conditions in
  the environment and reasoning to interpret
  perceptions, solve problems, draw inferences, and
  determine actions. (Hayes-Roth 1995)
• Intelligent agents are software entities that
  carry out some set of operations on behalf of a
  user or another program, with some degree of
  independence or autonomy, and in so doing, employ
  some knowledge or representation of the users
  goals or desires. (the IBM Agent)

12
13

• Agent a hardware or (more usually) a
  software-based computer system that enjoys the
  following properties
• autonomy - agents operate without the direct
  intervention of humans or others, and have some
  kind of control over their actions and internal
  state
• Flexible autonomous action
• reactivity agents perceive their environment and
  respond in a timely fashion to changes that occur
  in it
• pro-activeness agents do not simply act in
  response to their environment, they are able to
  exhibit goal-directed behaviour by taking
  initiative.
• social ability - agents interact with other
  agents (and possibly humans) via some kind of
  agent-communication language
• (Wooldridge and Jennings, 1995)

13
14

• Identified characteristics
• Two main streams of definitions
• Define an agent in isolation
• Define an agent in the context of a society of
  agents ? social dimension ? MAS
• Two types of definitions
• Does not necessary incorporate intelligence
• Must incorporate a kind of IA behaviour ?
  intelligent agents

14
15

• Agents characteristics
• act on behalf of a user or a / another program
• autonomous
• sense the environment and acts upon it /
  reactivity
• purposeful action / pro-activity
• goal-directed behavior vs reactive behaviour?
• function continuously / persistent software
• mobility ?
• intelligence?
• Goals, rationality
• Reasoning, decision making cognitive
• Learning/adaptation
• Interaction with other agents - social dimension
• Other basis for intelligence?

15
16

• Questions Examples of agents?
• (are they all agents?) Intelligent?
• a thermostat with a sensor for detecting room
  temperature
• electronic calendar
• log-in into your computer you are presented with
  a list of email messages sorted by date
• log-in into your computer you are presented with
  a list of email messages sorted by order of
  importance
• air-traffic control system of country X fails -
  air-traffic controls in the neighboring countries
  deal with affected flights

16
17
Agent Environment
Environment properties - Accessible vs
inaccessible - Deterministic vs
nondeterministic - Episodic vs non-episodic -
Static vs dynamic - Open vs closed - Contains
or not other agents
Agent
Sensor Input
Action Output
Environment
17
18
Multi-agent systems
Many entities (agents) in a common environment
Environment
18
Influenece area
Interactions
19
MAS - many agents in the same environment

• Interactions among agents
• - high-level interactions
• Interactions for - coordination
• - communication
• - organization
• Coordination
• ? collectively motivated / interested
• ? self interested
• - own goals / indifferent
• - own goals / competition / competing for the
  same resources
• - own goals / competition / contradictory goals
• - own goals / coalitions

19
20

• Communication
• ? communication protocol
• ? communication language
• - negotiation to reach agreement
• - ontology
• Organizational structures
• ? centralized vs decentralized
• ? hierarchical/ markets
• "cognitive agent" approach
• MAS systems?
• Electronic calendars
• Air-traffic control system

20
21

• Agents vs Objects
• Autonomy - stronger - agents have sole control
  over their actions, an agent may refuse or ask
  for compensation
• Flexibility - Agents are reactive, like objects,
  but also pro-active
• Agents are usually persistent
• Own thread of control
• Agents vs MAS
• Coordination - as defined by designer, no
  contradictory goals
• Communication - higher level communication than
  object messages
• Organization - no explicit organizational
  structures for objects
• No prescribed rational/intelligent behaviour

21
22

• How do agents acquire intelligence?
• Cognitive agents
• The model of human intelligence and human
  perspective of the world ? characterise an
  intelligent agent using symbolic representations
  and mentalistic notions
• knowledge - John knows humans are mortal
• beliefs - John took his umbrella because he
  believed it was going to rain
• desires, goals - John wants to possess a PhD
• intentions - John intends to work hard in order
  to have a PhD
• choices - John decided to apply for a PhD
• commitments - John will not stop working until
  getting his PhD
• obligations - John has to work to make a living
• (Shoham, 1993)

22
23

• Premises
• Such a mentalistic or intentional view of agents
  - a kind of "folk psychology" - is not just
  another invention of computer scientists but is a
  useful paradigm for describing complex
  distributed systems.
• The complexity of such a system or the fact that
  we can not know or predict the internal structure
  of all components seems to imply that we must
  rely on animistic, intentional explanation of
  system functioning and behavior.
• Is this the only way agents can acquire
  intelligence?

23
24

• Comparison with AI - alternate approach of
  realizing intelligence - the sub-symbolic level
  of neural networks
• An alternate model of intelligence in agent
  systems.
• Reactive agents
• Simple processing units that perceive and react
  to changes in their environment.
• Do not have a symbolic representation of the
  world and do not use complex symbolic reasoning.
• The advocates of reactive agent systems claims
  that intelligence is not a property of the active
  entity but it is distributed in the system, and
  steams as the result of the interaction between
  the many entities of the distributed structure
  and the environment.

24
25

The wise men problem
A king wishing to know which of his three wise
men is the wisest, paints a white spot on each of
their foreheads, tells them at least one spot is
white, and asks each to determine the color of
his spot. After a while the smartest announces
that his spot is white
The problem of Prisoner's Dilemma
Outcomes for actor A (in hypothetical "points")
depending on the combination of A's action and
B's action, in the "prisoner's dilemma" game
situation. A similar scheme applies to the
outcomes for B.
25
26
?

• The problem of pray and predators

?
?
?

• Cognitive approach
• Detection of prey animals
• Setting up the hunting team allocation of roles
• Reorganisation of teams
• Necessity for dialogue/communication and for
  coordination
• Predator agents have goals, they appoint a leader
  that organize the distribution of work and
  coordinate actions

?

• Reactive approach
• The preys emit a signal whose intensity decreases
  in proportion to distance - plays the role of
  attractor for the predators
• Hunters emit a signal which acts as a repellent
  for other hunters, so as not to find themselves
  at the same place
• Each hunter is each attracted by the pray and
  (weakly) repelled by the other hunters

26
27

• Is intelligence the only optimal action towards a
  a goal? Only rational behaviour?
• Emotional agents
• A computable science of emotions
• Virtual actors
• Listen trough speech recognition software to
  people
• Respond, in real time, with morphing faces,
  music, text, and speech
• Emotions
• Appraisal of a situation as an event joy,
  distress
• Presumed value of a situation as an effect
  affecting another happy-for, gloating,
  resentment, jealousy, envy, sorry-for
• Appraisal of a situation as a prospective event
  hope, fear
• Appraisal of a situation as confirming or
  disconfirming an expectation satisfaction,
  relief, fears-confirmed, disappointment
• Manifest temperament control of emotions

27
28
MAS links with other disciplines
Economic theories
Decision theory
OOP
Markets
AOP
Autonomy
Rationality
Distributed systems
Communication
MAS
Learning
Proactivity
Mobility
Cooperation
Organizations
Reactivity
Character
Artificial intelligence and DAI
Sociology
Psychology
28
29
Areas of RD in MAS

• Agent architectures
• Knowledge representation of world, of itself, of
  the other agents
• Communication languages, protocols
• Planning task sharing, result sharing,
  distributed planning
• Coordination, distributed search
• Decision making negotiation, markets, coalition
  formation
• Learning
• Organizational theories
• Norms
• Trust and reputation

29
30
Areas of RD in MAS

• Implementation
• Agent programming paradigms, languages
• Agent platforms
• Middleware, mobility, security
• Applications
• Industrial applications real-time monitoring and
  management of manufacturing and production
  process, telecommunication networks,
  transportation systems, electricity distribution
  systems, etc.
• Business process management, decision support
• eCommerce, eMarkets
• Information retrieving and filtering
• Human-computer interaction
• CAI, Web-based learning - CSCW
• PDAs - Entertainment

30
31
Agents in action

• NASAs Earth Observing-1 satellite, which began
  operation in 2000, was recently turned into an
  autonomous agent testbed.Image Credit NASA
• NASA uses autonomous agents to handle tasks that
  appear simple but are actually quite complex. For
  example, one mission goal handled by autonomous
  agents is simply to not waste fuel. But
  accomplishing that means balancing multiple
  demands, such as staying on course and keeping
  experiments running, as well as dealing with the
  unexpected.
• "What happens if you run out of power and you're
  on the dark side of the planet and the
  communications systems is having a problem? It's
  all those combinations that make life exciting,"
  says Steve Chien, principal scientist for
  automated planning and scheduling at the NASA Jet
  Propulsion Laboratory in Pasadena, Calif.

31
32
TAC SCM

• Negotiation was one of the key agent capabilities
  tested at the conference's Trading Agent
  Competition. In one contest, computers ran
  simulations of agents assembling PCs. The agents
  were operating factories, managing inventories,
  negotiating with suppliers and buyers, and making
  decisions based on a range of variables, such as
  the risk of taking on a big order even if all the
  parts weren't available. If an agent made an
  error in judgment, the company could face
  financial penalties and order cancellations.

32
33
Exemple de agenti Buttler agent

• Imagine your very own mobile butler, able to
  travel with you and organise every aspect of your
  life from the meetings you have to the
  restaurants you eat in.
• The program works through mobile phones and is
  able to determine users' preferences and use the
  web to plan business and social events
• And like a real-life butler the relationship
  between phone agent and user improves as they get
  to know each other better.
• The learning algorithms will allow the butler to
  arrange meetings without the need to consult
  constantly with the user to establish their
  requirements.

33
34
Robocup agents

• The goal of the annual RoboCup competitions,
  which have been in existence since 1997, is to
  produce a team of soccer-playing robots that can
  beat the human world champion soccer team by the
  year 2050.
• http//www.robocup.org/

34
35
Swarms

• Intelligent Small World Autonomous Robots for
  Micro-manipulation
• A leap forward in robotics research by combining
  experts in microrobotics, in distributed and
  adaptive systems as well as in self-organising
  biological swarm systems.
• Facilitate the mass-production of microrobots,
  which can then be employed as a "real" swarm
  consisting of up to 1,000 robot clients. These
  clients will all be equipped with limited,
  pre-rational on-board intelligence.
• The swarm will consist of a huge number of
  heterogeneous robots, differing in the type of
  sensors, manipulators and computational power.
  Such a robot swarm is expected to perform a
  variety of applications, including micro
  assembly, biological, medical or cleaning tasks.

35
36
Intelligent IT Solutions
Goal-Directed Agent technology.
AdaptivEnterprise Solution Suite allow
businesses to migrate from traditionally static,
hierarchical organizations to dynamic,
intelligent distributed organizations capable of
addressing constantly changing business demands.
Supports a large number of variables, high
variety and frequent occurrence of unpredictable
external events.
36
37
True UAV Autonomy

• In a world first, truly autonomous, Intelligent
  Agent-controlled flight was achieved by a Codarra
  Avatar unmanned aerial vehicle (UAV).
• The flight tests were conducted in restricted
  airspace at the Australian Armys Graytown Range
  about 60 miles north of Melbourne.
• The Avatar was guided by an on-board JACK
  intelligent software agent that directed the
  aircrafts autopilot during the course of the
  mission.

37
38
Information agents

• Personal agents (PDA)
• provide "intelligent" and user-friendly
  interfaces
• observe the user and learn users profile
• sort, classify and administrate e-mails,
• organize and schedule user's tasks
• in general, agents that automate the routine
  tasks of the users
• Web agents
• Tour guides Search engines
• Indexing agents - human indexing
• FAQ finders - spider indexing
• Expertise finder

38
39
Agents in eLearning

• Agents role in e-learning
• Enhance e-learning content and experience
• give help, advice, feedback
• act as a peer learning
• participate in assessments
• participate in simulation
• personalize the learning experience
• Enhance LMSs
• facilitate participation
• facilitate interaction
• facilitate instructors activities

39
40
Agents for e-Commerce

• E-commerce
• Transactions - business-to-busines (B2B)
• - business-to-consumer (B2C)
• - consumer-to-consumer (C2C)
• Difficulties of eCommerce
• Trust
• Privacy and security
• Billing
• Reliability

40