Multi-Agent Systems (Chapter 9) Adapted with permission from Adina Magda Florea [e-mail]

1
Multi-Agent Systems(Chapter 9)Adapted with
permission from Adina Magda Floreaadina_at_wpi.edu
2
Benevolent vs.. self-interested agents

• Benevolent cooperative distributed systems.
  (CDPS) Simplifies the task enormously.
• Self-interested- potential for conflict

3
Distributed problem solving

• Group coherence - agents want to work together -
  cooperative agents
• Competence - agents must find ways to work
  together - coordinate to cooperate
• Task and result sharing - an agent has many tasks
  to do and asks other agents to do some of its
  tasks then it should integrate the results
• Distributed planning - the problem to be solved
  is to design and execute a plan in a distributed
  manner, by many agents

3
4
Distributed Problem Solving

• Motivations
• Speed up through parallelization
• Distribution of expertise
• Distribution of Data, features change
• Problem is inherently distributed
• Distribution of Results
• General Steps
• Task decomposition
• Task allocation
• Exchange sub problem solutions
• Task accomplishment
• Results Synthesis (make whole)

5
Task Decomposition

• Partitioning of a task into sub-tasks for
  possible allocation to another agent
• Goal is to make sub-tasks independent Minimize
  coordination (so communication costs dont
  outweigh gain)
• Minimize shared data
• Minimize share resources
• Task decomposition is a hard problem and
  generally performed a priori by system designers.

6
Task Allocation

• Homogenous Systems
• Agents identical, allocation simple since each is
  equally qualified to work on sub-tasks
• Heterogeneous Systems
• Sub-task requirements - matched to agent skills
• Potentially difficult problem (perfect match
  problem)

7
Which kind of system to build?

• Homogenous systems are simpler
• Only one kind of agent to build
• Dont have to consider agent skills when
  distributing sub-tasks
• Homogenous systems considered unsuitable for
  complex problems
• Low overall utilization of skills and resources

8
Agent Roles in Task Allocation

• Agents can assume two roles
• Servers Agents capable of providing a service
• Clients Agents requiring a service
• Agents can be both
• I.e. An agent may use the services of other
  agents to complete a service is to providing to
  another agent
• Task allocation systems must provide a way to
  match clients with servers

9
Centralized Allocation Systems

• Centralized
• 3rd party manages client-server matching
• Hierarchical Subordination
• Superior agents order subordinates to carry out
  task.
• Typically a static, pre-defined agent
  organization
• Egalitarian - all agents considered equal
• Requires special broker or trader agents to
  manage client requests and server bids
• Allows centralized allocation techniques

10
Egalitarian Allocation System
RejectC
C
RequestA
RequestA
A
B
AcceptD
AcceptD
D
Servers
Trader
Client
11
Distributed Allocation Systems

• Each agent individually attempts to obtain
  required services
• Acquaintance Network
• Direct Allocation
• Agents can only use the services of the agents it
  knows about
• Potentially serious scalability issues
• Delegated Allocation
• Agents can ask other agents to use their
  acquaintances to find an agent capable of
  providing a particular service
• Requires strongly connect acquaintance network
• Both methods require accurate knowledge of agent
  skills
• May use various caching strategies to maintain
  and age acquaintance information

12
Distributed Allocation Systems (cont)

• Contract Net
• Market Place approach
• Clients issue description of tasks
• Servers reply with bids
• Client chooses the best bidder
• Server affirms its commitment
• Proven approach from other disciplines/simple
• Well suited for dynamic environments
• Concurrent and many-to-many nature of the
  protocol creates challenging race conditions

13
Task Allocation System Tradeoffs
Distributed
Centralized
Acquaintance
Contract Net
Trader

• Benefits
• Coherence
• Drawbacks
• Bottleneck
• Fault Intolerance

• Benefits
• No Bottleneck
• Fault tolerance
• Drawbacks
• Coherence
• Scalability
• Latency

• Benefits
• Proven/Simple
• Flexibility
• Drawbacks
• Message volume
• Temporal Spatial Ignorance

14
Types of Tasks

• Independent
• Tasks are self-contained
• Can be performed in any order and concurrently
• Interdependent
• The solutions of some sub-tasks are required for
  the solution of other sub-tasks
• Coordination possible if dependencies known
  before
• Possible dependencies only become apparent at
  runtime
• A Results Sharing mechanism is needed to solve
  these dependencies

15
Motivations for Results Sharing

• Confidence
• Independent derivations affirm/challenge previous
  results leading to more confidence
• Completeness
• Combination of partial results leads to a larger
  set of results
• Precision
• Sharing of results allows for iterative
  refinement (agents come to see interface)
• Timeliness
• Obvious performance benefits via parallel
  processing

16
Result Sharing

• Problem solving proceeds by agents cooperatively
  exchanging information as the solution is
  developed.
• Results may be shared
• proactively - one agent sends another agent some
  information because it believes that the other
  will be interested in it.
• reactively an agent sends information to
  another in response to a request.

17
Result Sharing Benefits

• Confidence (checking solutions)
• Completeness/precision share local views
• Timeliness may get results faster (even if agent
  could do it himself)

18
What about inconsistency?

• Ignore it but are you throwing away the true
  information (the part that doesnt fit the
  expectation)?
• Resolve it through negotiation
• Degrade gracefully
• progress opportunistically (not in strict
  predetermined order)
• communicate high level results, not raw data
• inconsistency resolved as you go (not at end)
• no single solution route (if one is problematic,
  try another)

19
The Coordination Problem

• Managing the interdependencies between the
  activities of agents. e.g.
• You and I both want to leave the room. We
  independently walk towards the door, which can
  only fit one of us. I graciously permit you to
  leave first.

20
Coordination Techniques

• Organisational Structures
• Multi-agent Planning
• Norms and social laws
• Coordination Models based on human teamwork
• Joint commitments (Jennings)
• Mutual Modelling

21
Organizational Structuring

• Organizes agents into an organization
• May be based on how the task was decomposed
• Agents use knowledge of the organization to
• Determine with whom to communicate
• Prioritize tasks
• Agents only need to know about the local
  organizational structure (coherence)
• Choosing an organization structure can, itself,
  be a difficult problem!

22
Organizational Structuring
Geographicallydistributed cells
23
Organizational Structures

• A pattern of information and control
  relationships between individuals.
• Responsible for shaping the types of interactions
  among the agents.
• Aids coordination by specifying which actions an
  agent will undertake.
• Organizational structures may be
• Functional (based on skills)
• Spatial (based on physical location)
• Temporal (based on time relationship)

24
Organizational Structure Models

• A pattern for decision-making and communication
  among a set of agents who perform tasks in order
  to achieve goals. e.g.
• Automobile industry
• Has a set of goals To produce different lines of
  cars
• Has a set of agents to perform the tasks
  designers, engineers, salesmen

Reference Malone 1987
25
Alternative Coordination Structures 1Product
Hierarchy
26
Alternative Coordination Structures 2Functional
Hierarchy
Product Manager (several products)
27
Alternative Coordination Structures 3Centralised
Market
Product Manager 3
Product Manager 2
Product Manager 1
Functional Managers
28
Alternative Coordination Structures
4Decentralised Market
Product Manager 3
Product Manager 2
Product Manager 1
Designers
Salesmen
Engineers
29
Comparison of Organization Structures the
Issues!
Production cost Coordination cost Vulnerability cost
Product hierarchy H L H-
Funtional hierarchy L M- H
Centralised market L M H-
Decentralised market L H L
30
Organizational Structures - Critique

• Useful when there are master/slave relationships
  in the MAS.
• Control over the slaves actions mitigates
  against benefits of DAI such as reliability,
  concurrency.
• Presumes that atleast one agent has global
  overview an unrealistic assumption in MAS.

31
Partial Global Planning (PGP)

• A DAI testbed Distributed Vehicle Monitoring
  Testbed (DVMT) to successfully track a number
  of vehicles that pass within the range of a set
  of distributed sensors (agents).

• Each agent monitors a dedicated area
• There could be overlapping areas

32
Partial Global Planning (PGP)

• Main principle cooperating agents exchange
  information in order to reach common conclusions
  about the problem solving process.
• Why is planning partial?
• The system does not generate a plan for the
  entire problem.
• Why is planning global?
• Agents form non-local plans by exchanging local
  plans and cooperating to achieve a non-local view
  of problem solving.

33
Partial Global Planning (PGP)

• Starts with the premise that tasks are inherently
  decomposed.
• Assumes that an agent with a task to plan for
  might be unaware as to what tasks other agents
  might be planning for and how those tasks are
  related to its own.
• No individual agent might be aware of the global
  tasks or states.
• Purpose of coordination is to develop sufficient
  awareness.

34
Partial Global Planning (PGP)

• PGP involves 3 iterated stages
• Each agent decides what its own goals are and
  generates short-term plans in order to achieve
  them.
• Agents exchange information to determine where
  plans and goals interact.
• Agents alter local plans in order to better
  coordinate their own activities.

35
Partial Global Planning (PGP)

• Partial Global Plan a cooperatively generated
  datastructure containing the actions and
  interactions of a group of agents.
• Contains
• Objective the larger goal of the system.
• Activity map what agents are actually doing and
  the results generated by the activities.
• Solution construction graph a representation of
  how the agents ought to interact in order to
  successfully generate a solution.

36
Partial Global Planning (PGP)

• A DAI testbed revisited.

Agenti
Overlapping area
Vehicle track
j
i
Agentj
37
Coordination Techniques

• Organisational Structures
• Multi-agent Planning
• Norms and social laws
• Coordination Models based on human teamwork
• Joint commitments (Jennings)
• Mutual Modelling

38
Multi-agent Planning

• Agents generate, exchange and synchronise
  explicit plans of actions to coordinate their
  joint activity.
• They arrange apriori precisely which tasks each
  agent will take on.
• Plans specify a sequence of actions for each
  agent.
• It is a trade-off between specificity and
  reactive.

39
Multi-agent Planning

• Two basic approaches
• Centralised plans of individual agents analysed
  by a central coordinator to identify
  interactions.
• Distributed a group of agents cooperate to form
  a
• Centralized plan
• Distributed plan
• Big difference between them!

40
Multi-agent Planning

• Distributed Planning for centralised plans
• e.g. Air traffic control domain (Cammarata)
• Aim Enable each aircraft to maintain a flight
  plan that will maintain a safe distance with all
  aircrafts in its vicinity.
• Each aircraft send a central coordinator
  information about its intended actions. The
  coordinator builds a plan which specifies all of
  the agents actions including the ones that they
  should take to avoid collision.

41
Multi-agent Planning

• Distributed Planning for distributed plans
• Individual plans of agents, coordinated
  dynamically.
• No individual with a complete view of all the
  agents actions.
• More difficult to detect and resolve undesirable
  interactions.

42
2 Distributed planning

• What can be distributed
• The process of devising a plan is distributed
  among agents
• Execution is distributed among agents
• Planning
• State representation and plan representation
• Search vs. planning
• representation of changes to the world state
• representation of and reasoning about the plan
  (steps/actions)

Planning ? Search
42
43

• 2.1 Centralized planning for distributed plans
• Operators
• move(b,x,y) move b from x to y ?
  movetotable(b,x)
• Precond on(b,x) ? clear(b) ? clear(y)
  Precond on(b,x) ? clear(b)
• Postcond on(b,y) ? clear(x) ? Postcond
  ob(b,T) ? clear(x) ? ?on(b,x)
• ?on(b,x) ? ?clear(y)

work backward from each on goal
I'm Bill Agent1
I'm Tom Agent2
on(A,B) on(C,D) on(E,F) on(B,T) on(D,T)
on(F,T)
on(B,A) on(F,D) on(A,E) on(D,C) on(E,T)
on(C,T)
1. Given a goal description, a set of
operators, and an initial state
description generate a partial order plan
43
44

• S1 move(B,T,A) To satisfy the preconditions,
  we have
• S2 move(A,B,E) S2 lt S1, S3 lt S4
• S3movetotable(E,F) S6 lt S4, S6 lt S5
• S4 move(F,T,D) Also
• S5 move(D,T,C) S2 threat to S3 ? S3 lt S2
• S6 movetotable(C,D) S4 threat to S5 ? S5 lt S4
• Then the partial ordering is S3 lt S2
  lt S1
• S6 lt S5 lt S4
• S3 lt S4
• Any total ordering that satisfies this partial
  ordering is a good plan for Agent1

2. Decompose the plan into sub problems so as to
minimize order relations across plans 3. Insert
synchronization 4. Allocate sub plans to agents
44
45

• What if we have 2 agents?
• DECOMP1
• Subplan1 S3 lt S2 lt S1
• Subplan2 S6 lt S5 lt S4
• and S3 lt S4
• Agent1 S3 lt send(clear(F)) lt S2 lt S1
• Agent2 S6 lt S5 lt wait(clear(F)) lt S4

46

• S3 movetotable(E,F) S2 move(A,B,E) S1
  move(B,T,A)
• S6 movetotable(C,D) S5 move(D,T,C) S4
  move(F,T,D)
• DECOMP2
• Subplan1 S3 lt S5 lt S4
• Subplan2 S6 lt S2 lt S1
• and S3 lt S2 and S6 lt S5
• Agent1 S3 lt send(don't_care(E)) lt wait(clear(D))
  lt S5 lt S4
• Agent2 S6 lt wait(don't_care(E)) lt wait(clear(D))
  lt S2 lt S1
• Obviously, DECOMP2 has more order relations among
  sub plans than DECOMP1
• Therefore, we choose DECOMP1
• S3 lt send(clear(F)) lt S2 lt S1
• S6 lt S5 lt wait(clear(F)) lt S4
• But
• then back to DECOMP2

lt
lt
4. If failure to allocate sub plans then redo
decomposition (2) If failure to allocate sub
plans with any decomposition then redo
generate plan (1) 5. Execute and monitor sub plans
I know how to move only D, E, F
I know how to move only A, B, C
46
47

• 2.2 Distributed planning for centralized plans
• generate separate plans, then merge
• parallel result sharing
• may involve negotiation
• Agent 1 - is specialized in doing
  movetotable(b,x)
• Agent 2 - is specialized in doing move(b,x,y)
• PAgent1 S3 movetotable(E,F) satisfies
  on(E,T)
• S6 movetotable(C,D) satisfies on(C,T)
• no ordering
• PAgent 2 S1 move(B,T,A), S2 move(A,B,E)
  satisfies on(B,A) ? on(A,E)
• S4 move(F,T,D), S5 move(D,T,C) satisfies
  on(F,D) ? on(D,C)
• ordering S2 lt S1 and S5 lt S4
• Merge PAgent1 with PAgent2 by checking
  preconditions and threats
• S3 lt S2, S6 lt S5, S3 lt S4, S2 lt S1 and S5 lt S4
• one agent executes (as is centralized)

47
48

• The problem is decomposed , given to specialize
• similar to task sharing
• may involve backtracking
• Agent 1 - knows only how to deal with 2-block
  stacks
• Agent 2 - knows only how to deal with 3-block
  stacks

48
49

• 2.3 Distributed planning for distributed plans
• a) Plan merging
• How much effort on coordinating issues?
• Agents formulate local plans to satisfy their
  goals
• Local plans are exchanged
• Local plans are combined analyzing for positive
  and negative interaction
• Add messages and/or timing commitments to resolve
  negative plan interactions and to exploit
  positive plan interactions
• Interacting situations
• Positive interactions between plans
• redundant actions
• beneficial actions
• Negative interactions between plans
• harmful actions
• exclusive actions
• incompatible actions

49
50

• movehigh(b,x,y)
• Precond have_lifter ? clear(b) ? clear(y) ?
  on(y,z) ? z? T
• Postcond on(b,y) ? clear(x) ? ?on(b,x) ? ?
  clear(y) ? free_lifter
• pick_lifter
• Precond free_lifter
• Postcond have_lifter ? ?free_lifter
• Agent1 S1move(B,T,A) lt S2 pick_lifter lt S3
  movehigh(E,T,B)
• Agent2 R1move(C,T,D) lt R2 pick_lifter lt R3
  movehigh(F,T,C)

Negative interactions what type?
if both select same lifter
R1
S1
need_l
S2
S3
Sf1
free_l
R2
R3
50
51
Positive interactions

• Give examples of positive interactions
• redundant
• beneficial
• Problems with the approach?

• b) Iterative plan formation
• build all feasible plans
• build partial order plans to facilitate plan
  merging
• build abstract plans to be iteratively refined

51
52

• c) Hierarchical distributed planning
• Each agent stores plans on several levels of
  abstraction
• Use abstract plans (hides details)
• Abstract operator - a kind of macro-operator
  sequence of applicable operators

Write paper
Type content
Read references
Organize ideas
Edit text
..
Locate Computer
Check for errors
Edit figures
52
53
Hierarchical behavior-space search
algorithm 1. Level ? 0 (current level of
abstraction), Agent_List Agent1, ,
AgentN 2. for i1,N do if Pi is compatible
with PJ, j1,N, j?i then Agenti
removes itself from Agent_List (no
problems) 3. if Agent_list then exit 4. Let
N be the new number of agents in Agent_List 4.1
Determine conflicts between Pi 4.2 if
conflicts to be resolved at a lower level
then (a) Level ? Level 1 (b) go to step
2 5. 5.1 Sort agents in Agent_List 5.2 for
i1,N-1, in current ordering do (a) make
Agenti the current superior (b) send Pi to
each AgentJ, ji1, N (c) for ji1, N do
- AgentJ checks compatibility of PJ with Pi and
replan - AgentJ checks compatibility with
PK, k1,i-1 and replan

• A kind of CSP
• - backward checking
• - forward checking
• Ordering
• - what heuristic?

Add exit condition for no solution
53
54

• 2.4 Distributed planning and execution
• Real world incomplete and incorrect information
• a) Contingency planning
• Conditional planning - constructing a conditional
  plan that accounts for each possible situation or
  contingency that could arrive

Start
on(A,B)?clear(C)?clear(A)
Checkarm(Ag1)
Ask Ag2 to move(A,B,C)
?armbroken(Ag1)
armbroken(Ag1)
move(A,B,C)
Context ?armbroken(Ag1)
Negotiate with Ag2 for it to achieve move
Plan to achieve on(B,A)
Finish
on(B,A)?on(A,C)
54
55
Multi-agent Planning

• Critique
• Agents share and process a huge amount of
  information.
• Requires more computing and communication
  resources.
• Difference between multi-agent planning and PGP
• PGP does not require agents to reach mutual
  agreements before they start acting.

56
Multi-agent Planning

• Sometime Plans can also become obsolete very
  quickly. i.e. Short life-span.

57
Lets take a minute

• Can you think of a situation where multi-agent
  planning will not be appropriate?
• Discuss with your neighbours.

58
Comparing Common Coordination TechniquesA Look
at the Issues
59
Coordination Techniques

• Organisational Structures
• Multi-agent Planning
• Norms and social laws
• Coordination Models based on human teamwork
• Joint commitments (Jennings)
• Mutual Modelling

60
Social Norms and Laws

• Norm an established, expected pattern of
  behaviour.
• e.g. To queue when waiting for the bus (not
  always in Norway!!)
• Social laws similar to Norms, but carry some
  authority.
• e.g. Traffic rules.
• Social laws in an agent system can be defined as
  a set of constraints
• Constraint gt ?E,? ?,
• E ? E is a set of environment states
• ? ? Ac is an action, (Ac is the finite set of
  actions possible for an agent)
• if the environment is in some state e ? E,
  then the action ? is forbidden.

61
Social Norms and Laws

• Example Feature interaction in
  telecommunications
• Uses deontic logic (model obligations)

62
Coordination Techniques

• Organisational Structures
• Multi-agent Planning
• Norms and social laws
• Coordination Models based on human teamwork
• Joint commitments (Jennings)
• Mutual Modelling

63
Coordination Cooperation 1

• Can we have coordination without cooperation?
• A group of people are sitting in a park. As a
  result of a sudden downpour, all of them run to a
  tree in the middle of the park because it is the
  only source of shelter.

64
Coordination Cooperation 2

• How does an individual intention towards a goal
  differ from being a part of a team (a collective
  intention towards a goal)?
• Responsibility
• e.g. You and I are lifting a heavy object.
• Individual goal ?? team responsibility

65
Coordination Based on Human Teamwork

• Some agent coordination models are inspired by
  human teamwork models, e.g. Joints intentions
  (Jennings).
• Intentions are central to the concept of
  practical reasoning.
• Practical reasoning deliberation means-end
  reasoning
• Deliberation deciding what state of affairs to
  achieve
• Means-end reasoning deciding how to achieve
  these states of affairs

66
Mutual Modelling

• Build a model of the other agents their beliefs
  and intentions.
• Put ourselves in the place of the other
• Coordinate own activities based on this model.
• Coordination without cooperation game-thoery
  can be used.

67
Joint Intentions

• Proposed by Jennings
• Based on human teamwork models
• When a group of agents are engaged in a
  cooperative activity, they must have a joint
  commitment to the overall aim as well as their
  individual commitments.
• Distinguishes between the commitment that
  underpins an intention and the associated
  convention.

68
Joint Commitments

• Commitment a pledge or promise (e.g. to lift
  the heavy object).
• Commitment persists if an agent adopts a
  commitment, it is not dropped until for some
  reason it becomes redundant.
• Commitments may change over time, e.g. due to a
  change in the environment
• Main problem with joint commitment
• Hard to be aware of each others states at all
  times

69
Conventions

• Convention means of monitoring a commitment
• e.g. specifies under what circumstances a
  commitment can be abandoned.
• Need conventions to describe when to change a
  commitment
• When to keep a commitment (retain)
• When to revise a commitment (rectify)
• When to remove a commitment (abandon)

70
Convention - Example

• Reasons for terminating a Commitment
• Commitment Satisfied
• Commitment Unattainable
• Motivation for commitment no longer present
• Rule R1
• If Commitment Satisfied OR
• Commitment Unattainable OR
• Motivation for Commitment no longer present
• then
• terminate Commitment.

71
Social Conventions

• Conventions describe how an agent should monitor
  its commitments, but not how it should behave
  towards other agents.
• Asocial
• Sufficient for goals that are independent.
• For inter-dependent goals
• Need social conventions
• Specify how to behave with respect to the other
  members of the team.

72
Teamwork Definition

• American Heritage Dictionary
• Cooperative effort by the members of a team to
  achieve a common goal.

73
Teamwork Example

• Two vehicles travelling in a convoy
• Consider two agents Bob and Alice. Bobs wants to
  drive home, but does not know his way. He knows
  that Alice is going near there and that she does
  know the way. Bob talks to Alice and they both
  agree that he follows her through traffic and
  that they drive together.

Ref Cohen Levesque, 1991
74
Teamwork 1

• Important distinction
• Coordinated action that is not cooperative, e.g
• Individual drivers in traffic following traffic
  rules
• Coordinated cooperative action, e.g
• A convoy of drivers

75
Teamwork 2

• How does an individual intention towards a
  particular goal differ from being a part of a
  team with a collective intention towards a goal?
• Responsibility towards the other members of the
  team.

• Agents i, j and k are a team and have a common
  goal G.

76
Teamwork 3

• Joint action by a team involves more than just
  the union of simultaneous individual actions.
• Joint intentions and mutual beliefs (Cohen
  Levesque, 1991)
• Joint commitment (Jennings, 1996)
• When a group of agents are engaged in a
  cooperative activity, they must have
• Joint commitment to the overall activity
• Individual commitment to the specific task that
  they have been assigned to

77
Joint Intentions (Jennings) RevisitedSocial
Conventions

• Team members must be aware of the convention that
  govern their interactions. e.g.

• Both Ai and Aj must fulfill their commitments to
  achieve G.

• Either Ai or Aj must fulfill their commitment.

• There is a need for all agents in a team to
  inform other members of the status of their
  commitments!

78
Teamwork Model Based on CDPS

• Recognition
• Agent has a goal and recognises the potential for
  cooperative action.
• Team Formation
• Finds a group of agents that have a commitment to
  joint action.
• Plan Formation
• Agree upon course of action, (through a process
  of negotiation).
• Team Action
• Execute agreed plan of joint action.

G
79
Team Selection

• The process of selecting a group of agents that
  have complimentary skills to achieve a given
  goal(s). (Ref Tidhar et. al., 1996)
• Agents exchange their skills, goals, plans,
  current beliefs.
• Done at runtime.