Over the past few years, a number of industrial applica-tions have deployed agents. However, a substantial gapstill exists between the cutting-edge research carried outmainly in university laboratories and research institutesand the domain-specific industrial applications that com-mercial organizations develop.
The articles in this department intend to give some indi-cation of agent technology’s readiness for commercial de-ployment, based primarily on the presentations and discus-sions at the inaugural Industry Track of AAMAS 2005—theFourth International Joint Conference on AutonomousAgents and Multiagent Systems.
Opportunities for agent deploymentMultiagent systems and autonomous-agent technologies
provide a design-and-implementation paradigm for softwaresolutions based on collective decision making in a commu-nity of autonomous, loosely coupled computational entities.Many agent development environments and agent integra-tion platforms are available either commercially or in opensource format ready to deploy in commercial applications.
The agent research community has consolidated signifi-cantly in the last few years. In particular, the formalizationof agent-based computing’s foundations has positioned thedomain in relation to adjacent fields of theoretical researchsuch as formal logic, game theory, theorem proving andmodel checking, distributed and parallel computing, scala-bility, and complexity theory. The community is alsoinvolved in much research that’s closer to potential businessapplications, such as the Semantic Web, open systems, andubiquitous computing. The achievements in these fieldsform a solid foundation for technology transfer from univer-sity labs and research institutes to industrial applications.
The agent paradigm and the available agent techniquesperform well in five types of domains. The first is compet-
itive and noncooperative domains, where information-sharing restrictions prevent a centralized decision-makingarchitecture—for example, e-commerce applications, sup-ply-chain management, and e-business. In such domains,the agent paradigm is employed to design and describemainly Web-based systems.
In the second type of domain, the data required for auto-mated decision making aren’t centrally available. The usualreasons for this are the geographical distribution of knowl-edge (for example, logistics, collaborative exploration,mobile and collective robotics, or pervasive systems) orenvironments where communication is partially or tem-porarily inaccessible. Other reasons include temporal distri-bution (for example, satellite networks where satelliteshave different views of the earth at different times of theday) and conceptual distribution (for example, in layeredhierarchies, where entities at one layer might have noknowledge of events or processes at other layers, as in theInternet or supply chains).
The third type of domain requires survivable time-criti-cal response and high robustness in distributed scenarios.Example domains include time-critical manufacturing orindustrial-systems control that requires replanning or fastlocal reconfiguration to handle problems instantly.
The fourth type of domain involves simulation and model-ing. Using agents for simulation has been common. Agentscan be deployed either in simulations requiring easy migra-tion to the real environment or where traditional simulationtechniques are expensive.
The final type of domain involves open-systems engi-neering. Early agent deployment projects emphasized suchdomains, but the reality of the implementations deliveredso far hasn’t met expectations. Even though using ontolo-gies and FIPA (Foundation for Intelligent Physical Agents)standards has addressed many syntax issues, semantic-integration issues remain problematic. Web services andWeb technologies in general seem to have taken the lead inapplications in this area.
In our experience, industrial organizations frequentlyrequest (and agent technology developers frequently pro-vide) these functionalities:
Agent technology provides industrial-applications
developers with new abstractions for distributed-
system development, new methodological tools, and a set of
algorithms for creating autonomous, collaborative systems.
Agents in Industry: The Best from the AAMAS 2005Industry Track
Michal Pechoucek, Czech Technical UniversitySimon G. Thompson, BT
• planning,• scheduling,• resource and strategic decision making,• diagnostics,• control and real-time replanning,• software systems integration,• interoperability,• knowledge integration,• ontologies, and• simulation and modeling.
Despite some successful case studies inindustry, agent technology has suffered fromhype and a loss of momentum. This loss hasseen many unique properties of the earlytools and techniques subverted by changes inthe commercial environment (for example,the Web’s emergence as the key corporateapplication platform, the dot-com crash, andMicrosoft’s emergence as the dominantprovider of desktop-personal-productivitysoftware) or by the development of rival tech-nologies such as Enterprise JavaBeans, over-lay networks, and Web services. Recently,however, the momentum has reversed courseas academic agent research programs haveborne fruit and particularly as agents haveincorporated advanced techniques from otherAI areas. Key examples of such advancesare the
• utilization of efficient, powerful plan-ning algorithms,
• development of OWL,• development of algorithms to reason
about action in teams,• development of efficient market-clearing
mechanisms, and• refinement of the general principles and
architectures of agents—specifically,BDI (belief-desire-intention).
These techniques have enabled the imple-mentation of applications having a clearadvantage over traditional systems.
In addition, improved computer hardwareand increased availability of open sourcecomponents, especially for networking andsoftware development, have made develop-ment of effective agent applications cheaper,faster, more reliable, and, above all, easier.
Another reason for agent technologies’increased momentum is that the human cap-ital available to practitioners has developedrapidly as generations of students who havebeen exposed to the principles and possibili-ties of agents and AI have joined the work-force. Clearly, without the skills and vision
to implement these techniques into practi-cal solutions, progress is impossible.
The AAMAS Industry TrackConferences organized by the agent
research community frequently discuss “bluesky” research ideas (ideas that aim beyondimmediate application), theoretical- andempirical-research results, and agent tech-nology’s potential and actual applicability.For example, AAMAS is an annual meeting ofagent technology researchers and practition-ers that has become the canonical forum forthe presentation of new results in the field.The conference resulted from the merger ofthree successful conferences (the Interna-tional Conference on Multiagent Systems;Agent Theories, Architectures, and Lan-guages; and the International Conferenceon Autonomous Agents). In an encouragingdevelopment, for the first time, a specialtrack at AAMAS 2005 covered research onthe industrial application of agents.
The Industry Track featured reports ondefense and exploration applications andreports from commercial business operations.1
In the aerospace applications session,NASA presented a monitoring agent forspace shuttle launching criteria, and the JetPropulsion Laboratory presented an auton-omous science agent flying onboard theEarth Observing One spacecraft. Two pre-sentations covered defense applications onautonomous control and teamwork of un-manned aerial vehicles, and one covered an agent-based simulation application fornaval training.
In the logistics-and-transport session, theCatholic University of Lueven presenteda decentralized approach to autonomous-guided-vehicle control for warehousing.Whitestein Technologies and Magenta Tech-nology offered their solutions for transportoptimization in industrial logistics. Univer-sity Jaume I Castellón researchers describedtheir concept of agent deployment for trafficmanagement and control. The ECN (EnergyResearch Center of the Netherlands) reportedon the successful application of agent tech-nology in electricity infrastructure control.
In the manufacturing session, RockwellAutomation’s presentation on agent-basedindustrial control represented a traditionalmanufacturing industry. Another presenta-tion detailed a system that the DFKI (Ger-man Research Center for AI) developed forplanning and monitoring steel production,and a Singapore Institute of Manufacturing
Technology presentation dealt with semi-conductor assembly. Two presentations inthis session focused on general conclusionsinstead of specific applications. The CzechTechnical University compared the successand potential of agent deployment in defenseand manufacturing applications. Tom Wag-ner, Les Gasser, and Mike Luck gave a panel-like presentation on agent technology’s po-tential impact.
The following short articles summarizewhat we consider to be the four best contri-butions to that track. Our goal is to conveythe event’s key contributions to a wideraudience than just the attendees or thosewho have time to read the entire proceed-ings. Unfortunately, we couldn’t includeevery significant presentation or mentionevery important discussion point, and manyattendees might disagree with our perspec-tive. However, we hope that this personalview serves as evidence of agent technol-ogy’s real and considerable impact.
Reference
1. M. Pechoucek, D. Stainer, and S. Thompson,eds., Proc. 4th Int’l Conf. Autonomous Agentsand Multi-Agent Systems—AAMAS 2005 Indus-try Track, ACM Press, 2005.
Variable-Autonomy Control of Teams of Uninhabited Air VehiclesJeremy W. Baxter and Graham S. Horn,QinetiQ
Uninhabited air vehicles are of particu-lar interest to the defense sector becausethey could significantly reduce risk to air-crews. Current UAV systems typicallyrequire multiple operators to control a sin-gle platform. QinetiQ has been developingan approach that lets one operator controlmultiple platforms.
The basic concept is a decision-makingpartnership between a human operator andan intelligent uninhabited capability. (Acapability is a collection of platforms, sen-sors, and weapons.) The human providesmission-level guidance to the pool of coop-erating UAVs and takes on a largely super-visory role. The UAVs self-organize toachieve the goals the operator sets, such asto observe an area or to locate and destroy ahigh-value mobile target. Owing to regula-tory or liability issues, a human must make
MARCH/APRIL 2006 www.computer.org/intelligent 87
some critical decisions, such as weapon re-lease. So, the uninhabited capability mustrefer such decisions to the operator. We’veimplemented this concept using a variable-autonomy interface onto a multiagent sys-tem, as part of a larger trials system.
The trials systemWe use the trials system (see figure 1) to
evaluate potential concepts of use and tech-nologies. It includes a synthetic-environment(SE) simulation that models real-worlddynamic interactions. Human-in-the-looptrials let us capture the key requirements forthe decision-making partnership. The sys-tem elements have evolved in response tofeedback from trials (subjective commentsand objective performance measures) andchanges to the concepts of use.
A multiagent system provides a natural,powerful way to represent multiplatformtasks and sets of coordinated, cooperatingagents. The trials system contains four typesof agent (see figure 1). The user agent allo-cates individual UAVs to the tasks that theoperator sets, and provides the operator withinformation. Group agents plan and coordi-nate a task’s execution, sometimes callingon the capabilities of specialist planning
agents. UAV agents interact directly withindividual platforms, commanding theautopilot to undertake specific maneuversand receiving status and sensor informa-tion. The user agent routes requests forcritical decisions to the variable-autonomyinterface. Depending on the autonomylevel (set for each request type), the inter-face will either automatically grant permis-sion to continue or defer to the operator.
Group agents embody the knowledge ofhow to plan and execute coordinated teamtasks using a framework based on joint-intentions theory.1 We originally designedthe framework to enable robust execution ofuser orders by teams of entities in ground-based battlefield simulations.2 It provides asolid grounding for the communication nec-essary to keep a team task coordinated.Originally, it contained only group and vehi-cle agents and didn’t let an operator issuenew tasks during execution (at start-up, itprovided each team with a single order thatcould be decomposed into orders for sub-groups). Adding the user agent allows foroperator interaction and parallel tasking.
The trials scenarioThe scenario for the trials is a time-criti-
cal targeting mission against a high-valuemobile target. The system deploys a pack-age of four UAVs, containing a variety ofsensors and weapons, to locate and destroythe target. The operator is the pilot of asingle-seat fighter. The mission consists oftwo main phases: search and attack.
One specialist planning agent producesplans for the search phase. It expands a set ofpossible target positions into regions that amoving target could reach in the next fewminutes. The agent plans routes that let theUAVs efficiently search these regions withshort-range sensors and take images of poten-tial targets that the operator will classify.
The attack phase can begin when the op-erator has classified a ground entity as thehigh-value target. Another specialist plan-ning agent provides access to a dynamicscheduler3 that allocates UAVs to the tasksthey must execute during the attack phase:release the weapon and gather images of thetarget to see if the weapon has destroyed it.
We’ve used the multiagent system inthree trials. A single pilot was able to suc-cessfully control the UAV team to com-plete the missions. Including the agents inthe trials system has allowed the quickercompletion of more complicated missions,with reduced operator workload.
AcknowledgmentsThis research was part of the UK Ministry of
Defence Output 3 research program on behalf ofthe Director Equipment Capability—Deep Tar-get Attack. We gratefully acknowledge their sup-port. We’re part of a QinetiQ team that’s devel-oping and implementing the decision-makingpartnership concept; we focus on the multiagent-system element.
References
1. H. Levesque, P. Cohen, and J. Nunes, “OnActing Together,” Proc. 8th Nat’l Conf. Arti-ficial Intelligence (AAAI 90), AAAI Press,1990, pp. 94–99.
2. J.W. Baxter and G.S. Horn, “Executing GroupTasks despite Losses and Failures,” Proc. 10thConf. Computer Generated Forces andBehavioral Representation, 2001, pp.205–214.
3. M.J.A. Strens and N. Windelinckx, “Com-bining Planning with Reinforcement Learn-ing for Multi-Robot Task Allocation,” Adap-tive Agents and MAS II, D. Kudenko et al.,eds., LNAI 3394, Springer, 2005, pp.260–274.
88 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS
Useragent
Operator interfacesInformationand requests
Commands andoperator approvals
Status andsensor data
Platformcommands
Groupagent
UAV1agent
Groupagent
Groupagent
Groupagent
Specialistplanning agent
UAV2agent
UAV3agent
UAV4agent
Multiagent system
UAV2 platformcontroller
Synthetic environment
Figure 1. The main components of a trials system for controlling uninhabited air vehicles. Each UAV agent interacts with a platform controller that’s connected to thesynthetic environment.
The PowerMatcher: MultiagentControl of Electricity Demandand SupplyKoen Kok, Cor Warmer, and René Kamphuis,Energy Research Center of the Netherlands
Distributed generation of electricity isproviding an increasing part of the world-wide energy supply. DG consists of differ-ent sources of electric power connected tothe distribution network or to a customersite. This approach is distinct from the tra-ditional central-plant model for electricitygeneration and delivery. Examples of DGare photovoltaic solar systems, small andmedium-scale wind turbine farms, and thecombined generation of heat and power(CHP).
When the share of DG increases in a geo-graphical area, clustered control of DG bycommon ICT (information and communica-tion technology) systems can add value. Asa result, distribution networks are expectedto evolve from a hierarchically controlledstructure into a network of networks, inwhich a vast number of system parts com-municate with and influence each other.The number of components actively involvedin coordination will be huge. Centralizedcontrol of such a complex system will reachthe limits of scalability and communicationoverhead.
A key technology for solving this prob-lem is market-based control. In market-based control, many control agents com-petitively negotiate and trade on an elec-tronic market to optimally achieve theirlocal control action goals. Use of market-based control in the electricity infrastruc-ture opens the possibility for distributedcoordination in addition to the existingcentral coordination.
The PowerMatcherThe PowerMatcher method provides mar-
ket-based control for supply-and-demandmatching (SDM) in electricity networkswith a high share of DG. It’s based partly onearlier research by Fredrik Ygge and HansAkkermans;1 Hans Akkermans, Jos Schrein-emakers, and Koen Kok;2 and Per Carlsson.3
In this method, a control agent representseach device. The agent tries to operate thedevice process in an economically optimalway, within the process’s constraints. Theagents negotiate their electricity consumptionor production on an electronic exchange mar-ket. The resulting market price determines the
power volume allocated to each device.From the viewpoint of controllability,
devices that produce or consume electric-ity fall into six classes, each having a spe-cific agent strategy. We look at three in thisarticle. The first class consists of stochas-tic-operation devices, such as solar andwind energy systems, where the powerexchanged with the grid behaves stochasti-cally. The second class is shiftable-opera-tion devices, which must run for a certainamount of time regardless of the exact mo-ment and thus are shiftable in time. Anexample of such a device is a ventilationsystem in a utility building that needs torun for 20 minutes each hour. The thirdclass comprises user action devices, whoseoperations result from a user’s direct ac-tion. Examples include audio and videodevices, lighting, and computers.
Local agents’ self-interested behaviorcauses electricity consumption to shifttoward moments of low electricity pricesand causes production to shift toward mo-ments of high prices. So, SDM emerges onthe global-system level.
A simulationTo investigate distributed SDM’s impact
for a residential area, we simulated a clus-
ter of 40 houses, all connected to the samesegment of a low-voltage distribution net-work. Heat pumps (electricity consumers)heated 20 of the dwellings; micro-CHPunits heated the other 20. The simulationtreated washing machines as shiftable-operation devices with a predefined opera-tional time window, photovoltaic solar cellsas stochastic-operation devices, and light-ing as user action devices.
Figure 2 shows the result of a typicalsimulation run. In both plots, a single plot-line indicates the total consumption andproduction, and we treat production asnegative consumption. In figure 2a, alldevices are free running; in figure 2b, themarket-based control agents match supplyand demand.
This simulation shows that our methodcan exploit flexibility in device operationthrough agent bids in an electronic powermarket. The peak in electricity demand is substantially lower in the controlledcase. From the viewpoint of network oper-ations, this result is important, because thehighest expected peak demand determinesthe needed network capacity (transformersand cables). Reducing this peak reducesnetwork investments. Furthermore, intro-ducing SDM results in a flatter, smoother
MARCH/APRIL 2006 www.computer.org/intelligent 89
20
10
0
–10
–20
Load
(kW
)
0:00 03:00 06:00 09:00Time12:00 15:00 18:00 21:00
20
10
0
–10
–20
Load
(kW
)
0:00 03:00 06:00 09:00Time12:00 15:00 18:00 21:00
Total cluster consumptionTotal cluster productionFeed in from midvoltage net
Total cluster consumptionTotal cluster productionFeed in from midvoltage net
Peak power30% lower
Flat feedin profile
(b)
(a)
Figure 2. Results of a simulation of residential electricity distribution: (a) free-runningdevices; (b) market-based control agents match supply and demand. Multiagent control leads to peak load reduction and power profile smoothing.
90 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS
profile of the electricity fed in from themidvoltage network. This result is inter-esting from the viewpoint of electricitytrading, where increased predictability ofboth production and consumption addsvalue.
Field testingWe’re investigating the PowerMatcher in
real-life environments for two different busi-ness cases. One aims to automatically reducethe imbalance in a commercial trader’s real-world portfolio by aggregating medium-sized industrial electricity producing andconsuming installations (see figure 3). In thisexperiment, overproduction and underpro-duction of wind parks induce price changeson the cluster’s electronic market. The otherdevices’control agents react to this withcounteractions, which restore the cluster’senergy balance. The first test results indicatea decrease of the total power imbalance byapproximately 25 percent. Reduction ofunpredictability in the trade portfolio reducesimbalance costs charged to the trader by theindependent network operator.
The other field test, on a cluster of micro-CHP units operating as a virtual power plant,demonstrates their ability to contribute to acommon control goal. This experiment uses
15 domestic heating systems at consumerpremises. The virtual power plant can pro-vide value through electricity trading or local-grid-operation support.
AcknowledgmentsThe European Commission partially supported
this research in the context of the EUSUSTDEV Proj-ect ENK5-CT-2002-00673, called CRISP (Distrib-uted Intelligence in Critical Infrastructures forSustainable Power).
References
1. F. Ygge and J.M. Akkermans, “Resource-Oriented Multi-Commodity Market Algo-rithms,” Autonomous Agents and MultiagentSystems, vol. 3, no. 1, 2000, pp. 53–71.
2. J.M. Akkermans, J.F. Schreinemakers, andJ.K. Kok, “Emergence of Control in a Large-Scale Society of Economic Physical Agents,”Proc. 3rd Int’l Joint Conf. Autonomous Agentsand Multiagent Systems (AAMAS 04), IEEECS Press, 2004, pp. 1230–1231.
3. P. Carlsson, “Algorithms for Electronic PowerMarkets,” PhD thesis, Dept. of InformationTechnology, Uppsala Univ., 2004.
Manufacturing Agents at Rockwell AutomationVladimír Marík and Pavel Vrba, RockwellAutomation Research Center, PragueKenwood H. Hall and Francisco P. Maturana, Rockwell Automation AdvancedTechnology Laboratory
As the complexity of manufacturingbusiness environments grows, multiagent-systems (MAS) technology is becomingincreasingly important for development ofhighly distributed, robust, and flexible in-dustrial-control architectures. From an MASviewpoint, the manufacturing system is acommunity of highly distributed, autono-mous, efficiently cooperating, and asyn-chronously communicating units—agents—integrated by the plug-and-operate approach.
Rockwell Automation Inc. manufacturesindustrial-automation technology, leadingthe US market in discrete automation andcontrol products. In 1995, its first indus-trial-agent project optimized machine loadbalancing and increased reliability of asteel rod bar mill. It is currently applyingMAS technology to its flagship product,ControlLogix programmable logic con-trollers (PLCs), and developing the MAST(Manufacturing Agent Simulation Tool)
Local agent
Wind turbine park 1
Wind turbine park 2
Residential heat production
Cold storage
Emergency generator
Test dwelling
PowerMatcher aggregator
Local agent
Local agent Local agent
Local agent Local agent
Datacommunications
network
Figure 3. Control agents balancing a real-world commercial-trade portfolio.
MARCH/APRIL 2006 www.computer.org/intelligent 91
agent simulation infrastructure to supportdesign and validation.
Real-time control agents and simulationRockwell Automation’s control agent
architecture usually implements each agentas a module that encapsulates both the real-time control subsystem and the softwareagent (see figure 4). The RT control subsys-tem directly handles the information fromphysical sensors and actuators in real timeand is programmed in a low-level language(usually the ladder logic programming lan-guage). The software agent is implementedin a higher-level programming language(usually C++ or Java) and handles decisionmaking and negotiation.
The important part of this solution is anefficient runtime interface allowing bothinformation transfer from the RT controlsubsystem (I/O and other control or diag-nostics data) to the software agents andpropagation of the agents’ control actionsto the RT control subsystem. To simplifythis system’s integration with existing in-dustrial-automation-control architecturesbased on PLCs, we gave the agents directaccess to the PLC’s data memory so thatthey can observe and influence the RT con-trol subsystem directly.
When thinking of a real industrial de-ployment of agents that requires high reli-ability and strict adherence to real-timeconstraints, you must abandon the idea ofhosting the software agents on a PC andinterfacing them to PLCs. Therefore, wehave modified the ControlLogix PLC’sfirmware so that the C++ agents can rundirectly inside the PLC in parallel with theladder logic code. Rockwell has developedthe Autonomous Cooperative System as aC++-based agent platform dedicated toControlLogix PLCs. The ACS lets us dis-tribute the agents across several PLCs (onePLC usually hosts several agents). It alsosupports agent management services (reg-istration, deregistration, services lookup,and so on) and ensures the transport ofmessages conforming to FIPA (Foundationfor Intelligent Physical Agents) standardsamong the agents.
The ACS’s first application was the de-velopment of a reconfigurable control sys-tem for a US Navy ship’s chilled-watersystem1 that increased the system’s surviv-ability. An individual agent controls eachelement of the physical CWS equipment(valve, cooling unit, piping section, and so
on). When an agent’s built-in diagnosticmodule detects a failure, the agent initiatesnegotiation with other agents to reconfig-ure the CWS—for example, finding analternative path for water in the piping sec-tion to avoid a broken part.
To test and validate the agent-based con-trol system before deploying it in a manu-facturing environment, simulation is indis-pensable. The simulation must emulate themanufacturing equipment or processes; forthis, we strongly prefer commercially avail-able simulators such as Matlab or Arena.Once the simulation proves that the agent-based control system is mature enough todeploy, we replace the simulation with thereal physical system. This shift must be assmooth as possible, preferably without anymodifications to the agent code developedfor the simulation. Because the agents willinteract with the physical system by shar-ing the control data in the PLC memory, weuse this mechanism also to share data withthe simulation.
For example, for agent-based control ofthe CWS, we implemented the simulationin Matlab and Simulink. After verificationand testing, we successfully deployed theunchanged agent-based control system tocontrol the valves, cooling units, and otheractual equipment of a scaled-down physi-cal model of the ship.
MASTThis simulation environment, which
Rockwell Automation developed andimplemented in Java, serves mainly as an
agent-based demo implementation formaterial handling in flexible manufactur-ing. The developed agent library representsmaterial-handling systems’ basic compo-nents such as work cells, conveyor belts,and switches (diverters). Agent cooperationfocuses on finding the optimal transporta-tion routes in the system. The proposedsolutions provide fault tolerance and struc-tural flexibility. You can emulate any com-ponent failure (for example, any conveyorbelt failure), which causes the agents tonegotiate an alternative route to avoid thebroken component. You can add new com-ponents (representing new transportationcapabilities) to the system or remove exist-ing ones on the fly.
Recently, Rockwell Automation has ex-tended the MAST environment to simulatethe holonic-packing-cell testbed at the Uni-versity of Cambridge’s Centre for Distrib-uted Automation and Control (see figure5). They have extended MAST’s agent li-brary with a set of agents to represent andcontrol particular components of the lab’sequipment such as a Fanuc M6i robot, astorage unit, a gate in a Montech conveyorsystem, a gantry robot, rack storage, andRFID (radio frequency identification) read-ers. More important, an agent representseach manufactured product—in this case, acustomized Gillette gift box. This productagent autonomously and proactively con-trols its own production process by negoti-ating with the other agents. In this case, theprocess involves packing a box with differ-ent grooming items such as gels, deodor-
Agent(C++ or Java)
Ladder logic
Low
-leve
lco
ntro
lHi
gh-le
vel
cont
rol FIPA
communication
IEC 1131-3communication
ControlLogix PLC
FIPAAgent(C++ or Java)
Data tableTags
Ladder logic
Figure 4. A real-time control agent architecture for the ControlLogix programmablelogic controller. (FIPA stands for the Foundation for Intelligent Physical Agents; IEC stands for the International Electrotechnical Commission.)
ants, and razors. The agents negotiate oversuch issues as which storage location canprovide the requested items and which ro-bot will pack them.
The industrial case studies in this articleillustrate that you can effectively employMAS technology to design the next gener-ation of large-scale, robust, and flexiblemanufacturing control systems. Featuressuch as fully decentralized decision mak-ing, dynamic lookup for suitable serviceproviders, or embedded support for simu-lations go far beyond the capabilities ofclassic centralized and hierarchical indus-trial-control systems.
Reference
1. F.P. Maturana, R.J. Staron, and K.H. Hall,“Methodologies and Tools for IntelligentAgents in Distributed Control,” IEEE Intelli-gent Systems, vol. 20, no. 1, 2005, pp. 42–49.
Adaptive, Dynamic TransportOptimizationKlaus Dorer and Monique Calisti,Whitestein Technologies
Logistic networks’ increasing complex-ity and dynamic nature motivates a cost-sensitive rethinking of process and opti-
mization strategies.1 This goal requires notonly efficient processes but also IT solu-tions that can deliver the required flexibil-ity and dynamically respond to change andcustomization.
Living Systems Adaptive TransportationNetworks is a comprehensive agent-basedsolution for optimization and dispatchingof full and partial truckloads, including track-ing and real-time event handling. LS/ATNincludes
• a real-time route optimizer,• an event management system that in-
forms dispatchers about a wide range ofevents as they occur (or, proactively, ifexpected events don’t occur),
• a tracking facility that provides accuratedata about the progress of orders, and
• a simulation mode that assists in tacticaland strategic decision making.
Agent-based optimizationFinding optimal routes for serving trans-
portation requests from a (usually large) setof customers is a complex problem. A lim-ited number of available trucks must pickup and deliver transportation orders at spe-cific customer locations. The trucks can beof different types and capacities and are usu-ally available at different locations. Truckdrivers must observe drive time restrictions.
Pickup and delivery must occur within spe-cific time windows, even though time con-straints can potentially be violated withinsome tolerated degree (soft constraints).The problem is highly dynamic, not onlybecause transportation requests aren’t allknown in advance but also because variousunpredictable events can affect previouslydefined plans. Trucks might be delayedowing to traffic jams or other unforeseenproblems or even become temporarilyunavailable.2
You can distribute the solving of thistransport optimization problem amongmultiple interacting software agents to
• achieve scalability with growing sizes ofproblem instances,
• directly reflect the distributed nature oftransportation organizations and deci-sion-making centers,
• facilitate the handling of local deviationswithout having to propagate local changesand recompute the whole solution, and
• increase robustness (avoiding a singlepoint of failure).
In particular, the LS/ATN architecturereflects how logistics companies managethis domain’s increasing complexity. Atransportation business is usually dividedinto dispatching regions. Transportationrequests arriving at a region are first tenta-tively allocated and possibly optimized inthat region. If orders’ pickups or deliveriesoccur in different regions, these other re-gions are also informed and asked to han-dle the request in case they can provide acheaper solution to transport the order.
In LS/ATN, distinct software agents rep-resent different regions. A local AgentRe-gionManager manages trucks starting in itsregion. A centralized AgentDistributor dis-tributes incoming transport requests ac-cording to their pickup location. Whenreceiving a new order, an AgentRegion-Manager generates a valid solution (that is, a transportation plan specifying whichorders to combine into which routes, andwhich trucks will handle those routes). Todo this, LS/ATN uses a contract net proto-col to sequentially insert transportationrequests.3 The system checks all availabletrucks in that region both to verify theircapability to transport the order and to de-termine the cost. However, this approachcould produce suboptimal plans—for ex-ample, because the “best fitting” truck is
92 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS
Figure 5. A simulation of the Cambridge packing cell in the MAST simulation environment.
MARCH/APRIL 2006 www.computer.org/intelligent 93
already full. So, to improve the solution,the system schedules cyclic transfers be-tween trucks.4,5 A cyclic transfer is an ex-change of orders between routes—transferrequests among regions are triggered whentrucks have routes spanning different re-gions. A simple strategy to select an ordertransfer is a hill-climbing approach thatselects the most cost-saving transfers froma neighborhood of possible transfers. Thishill-climbing process continues with allchanged routes until the system can’t per-form any more cost-saving exchanges.
The LS/ATN design’s main advantagestems from its direct mapping to today’stransport business organizations and itsgood scalability. Moreover, its computa-tional overhead is also lower than a fullydistributed solution using one agent pertruck.6 Its main drawback is degradation ofthe solution, compared to a fully central-ized approach (from a global-optimumperspective). However, a fully centralizedsolution often wouldn’t be feasible in real-world scenarios.
Figure 6 illustrates details of a route thatLS/ATN generated.
ResultsWe ran extensive empirical tests for ABX,
a European logistics company, to determinewhat cost savings LS/ATN could provide.The cost model and constraint checker tookinto account real-world costs and constraints,thereby enabling comparison of the agent-based solution’s optimization results with realtransport plans that professional dispatcherscreated manually. The analyzed data set con-tained roughly 3,500 real business transporta-tion requests (orders).
LS/ATN decreased costs 11.7 percent; 4.2percent of this stemmed from fewer drivenkilometers. Another 2.2 percent of the sav-ings came from significantly increasing thenumber of consecutive routes that are cheaperto sell on the spot market. The rest of the sav-ings stemmed from the LS/ATN solutionpreferring routes starting in regions wheretrucks are cheaper to buy. An additional im-portant achievement is that the LS/ATN solu-tion used 25.5 percent fewer trucks than themanual solution. This is due to higher utiliza-tion of the trucks and longer utilization, onaverage, of each truck. Today, ABX usesLS/ATN in its day-to-day operations.
LS/ATN draws its strength from a multi-agent system core. Built on a bottom-up opti-mization philosophy, goal-directed agents
interact to solve subproblems that, whenconsolidated, result in a solution to the over-all problem. Similar to human decision mak-ing, solutions to problems arise from theinteraction of individual decision makers(represented by software agents), each withits own local knowledge. Traditional IT sys-tems’ centralized, rule-based nature imposesintrinsic limits on dealing successfully withunpredictability. Multiagent systems don’thave this limitation because collaboratingagents quickly adapt to changing circum-stances and operational constraints.
References
1. “UK Consumer Products Industry Cites CostReduction as Its Biggest Logistics Challenge,”Exel News, 10 Sept. 2002, www.exel.com/exel/home/media/news/newsreleases/2002/pressreleasecostreduction.htm.
2. M.W.P. Savelsbergh and M. Sol, “The Gen-eral Pickup and Delivery Problem,” Trans-portation Science, vol. 29, no. 1, 1995, pp.17–29.
3. J.-J. Jaw et al., “A Heuristic Algorithm for theMulti-Vehicle Advance Request Dial-a-RideProblem with Time Windows,” Transporta-tion Research, vol. 20 B, no. 3, 1986, pp.243–257.
4. P.M. Thompson and H.N. Psaraftis, “CyclicTransfer Algorithm for Multivehicle Routingand Scheduling Problems,” Operations Re-search, vol. 41, no. 5, 1993, pp. 935–946.
5. S. Mitrovic-Minic, Pickup and Delivery Prob-lem with Time Windows: A Survey, tech. reportTR 1998-12, School of Computing Science,Simon Fraser Univ., 1998.
6. K. Fischer, “Cooperative TransportationScheduling: An Application Domain forDAI,” Applied Artificial Intelligence, vol. 10,1996, pp. 1–34.
Conclusions and Lessons LearnedMichal Pechoucek and Simon G. Thompson
The AAMAS 2005 Industry Track includedseveral discussions in various formats, rang-ing from formal debates to philosophicaldiscussions in the small hours of the morn-ing. These discussions explored how theagent community could improve its rele-vance and impact to build on successes sofar. While a reasonable amount of interac-tion occurs between agent researchers andindustry, industrial adoption of agent-basedsolutions faces these main bottlenecks:
• Limited awareness about the potential ofagent technology in industry. Agents areused in a few specialized disciplines andremain unused in others where they mightbe appropriate.
• Limited publicity of successful indus-trial projects with agents.
• Misunderstandings about agent technol-
Figure 6. Details of a route that LS/ATN (Living Systems Adaptive Transportation Networks) computed, including route and order information, a map of the route, andschedule information in tabular and graphical form.
ogy’s capabilities, which led to earlyindustrial adopters’ unrealistic expecta-tions and subsequent frustration.
Some common unrealistic expectationsand inappropriate uses of agent technologyfall into seven main categories:1
• Complexity. People often expect thatagent technology can help solve verycomplex (perhaps NP-hard) problems. In our experience, this is obviously incor-rect, although partitioning problems usingthe agent abstraction can often lead toapproximate solutions with lower com-putational demands.
• Black box. People often view agenttechnology as a black-box technology
(like neural networks or genetic algo-rithms) that you can insert to solve aparticular complex problem. However,agent technology provides primarilysystem concepts and design paradigmsthat are useful in well-defined classesof problems.
• Intelligence. People sometimes thinkthat agents can directly deal with prob-lem solving and domain-specific intelli-gence. However, agent researchers’ primeconcern is the agents’ collective behaviorand decision making, and agent researchoften overlooks the technology’s appli-cation to real-life problems.
• Agentification. People think that you canfully automate agent integration and leg-acy system encapsulation. However, no
sophisticated mechanism exists that canencapsulate any legacy system fullyautomatically. Common current solu-tions involve alternative technologies(for example, Web services).
• Learning. People frequently overesti-mate multiagent systems’ potential for learning. They often think that anagent should be superadaptable andable to accommodate to any requestedbehavior (this expectation is closelyconnected to those of intelligence andagentification).
• Interoperability. Standards and interop-erability are computationally expensive.It isn’t wise to use full FIPA (Foundationfor Intelligent Physical Agents) compli-ance in systems where full openness
94 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS
Michal Pechoucek isthe principal investigatorand head of the GerstnerLaboratory Agent Tech-nology Group, an asso-ciate professor in artifi-cial intelligence at theCzech Technical Univer-sity in Prague, and a
part-time senior consultant for CertiCon. Con-tact him at [email protected].
Simon G. Thompson isa principal research sci-entist and research groupleader at the IntelligentResearch Center in BT’sresearch department.He’s also a visiting re-search fellow at the Uni-versity of Southampton.
Jeremy W. Baxter is a lead researcher atQinetiQ. Contact him at [email protected].
Graham S. Horn is aresearcher at QinetiQ.Contact him at [email protected].
Koen Kok is a scientificresearcher in intelligentenergy management atthe Energy ResearchCenter of the Nether-lands. Contact him [email protected].
Cor Warmer is a scien-tific researcher in intelli-gent energy managementat the Energy ResearchCenter of the Nether-lands. Contact him [email protected].
René Kamphuis is ascientific researcher inintelligent energy man-agement at the EnergyResearch Center of theNetherlands. Contact himat [email protected].
Vladimír Marík is themanaging director of theRockwell AutomationResearch Center, Prague,Czech Republic. Contacthim at [email protected].
Pavel Vrba is the leadof the Agent TechnologyGroup at the RockwellAutomation ResearchCenter, Prague, CzechRepublic. Contact him [email protected].
Kenwood H. Hall is the vice president forArchitectures and Sys-tems Technology atRockwell Automation.Contact him at [email protected].
Francisco P. Maturanais the Agent Infrastruc-ture Lead at the RockwellAutomation AdvancedTechnology Laboratory,Cleveland. Contact himat [email protected].
Klaus Dorer is a seniorresearcher at WhitesteinTechnologies. Contacthim at [email protected].
Monique Calisti is thevice president of R&D at Whitestein Technolo-gies. Contact her [email protected].
isn’t necessary (for example, in simula-tion and modeling).
• Mobility. People often claim that agentmobility is inevitable and more essentialthan is actually the case. Often, migra-tion of data or simple communication issufficient, rather than migration of anagent’s code and state.
While the Industry Track attendees ap-preciated the presentations’ high technicalquality, they frequently expressed one spe-cific concern. The presented applications
demonstrated only a technical, revenue, orefficiency advantage. These application’sdevelopers can rightly claim that theirsolutions are superior to those that werethe previous state of the art. However, inbusiness, this isn’t the fundamental test ofvalue. Businesses assess an advantage by itsreturn on investment, but the Industry Trackpresented no evidence of this. This point (ina slightly different form) also applies tofields such as defense and medicine, where aprecise commercial quantification of agenttechnology isn’t possible. These fields have
well-known benchmarks and metrics forevaluating system performance, but agent-based systems rarely, if ever, prove superioraccording to these criteria.
Reference
1. M. Pechoucek, M. Rehak, and V. Marik,“Expectations and Deployment of Agent Tech-nology in Manufacturing and Defence: CaseStudies,” Proc. 4th Int’l Conf. AutonomousAgents and Multi-Agent Systems—AAMAS 2005Industry Track, ACM Press, 2005.
MARCH/APRIL 2006 www.computer.org/intelligent 95
Healthcare is a vast open environment characterized by sharedand distributed decision making and management of care. Itrequires the communication of complex and diverse forms ofinformation among a variety of clinical and other settings, as wellas coordination among groups of healthcare professionals withvery different skills and roles. It has been argued in recent yearsthat intelligent agents’ basic properties (autonomy, proactivity,and social ability) and multiagent systems’ main features (man-agement of distributed information, and communication andcoordination between separate autonomous entities) make thesetechniques good options for solving problems in this domain. Thisspecial issue aims to provide empirical confirmations of this claim,by presenting successful applications of agent technology in anyhealthcare-related area (for example, diagnosis, monitoring,scheduling, and decision support systems).
Submitted papers should address at least one of these issues:
• Successful application of agents and multiagent systems inhealthcare (this is the preferred topic).
• Cooperation between intelligent agents to improve patientmanagement (for example, distributed patient scheduling).
• Agents that deliver remote or elderly care (for example, homecare).
• Agents that provide information about medical services.• Multiagent systems for patient monitoring and diagnosis.• Multiagent systems that improve medical training or educa-
tion (for example, tutoring systems).• Medical agent-based decision support systems.• Information agents that gather, compile, and organize med-
ical knowledge available on the Internet.• Solutions to the basic methodological and technological prob-
lems associated with deployment of agent-based healthcaresystems:
– Security and privacy of medical data.– Social acceptance of agent-based systems.– Lack of common medical ontologies.– Lack of centralized control.– Communication standards.– Integration with other types of software.– Legal and ethical issues.
• A survey of the state of the art in healthcare agents.
Important Dates
Submissions due for review: 9 June 2006
Notification of acceptance: 25 Aug. 2006
Final version submitted: 8 Sept. 2006
Issue publication: Nov./Dec. 06
Submission GuidelinesAuthors who plan to submit a paper for this special issue are
encouraged to send a one-page text summary before 20 Aprilto Antonio Moreno, the special issue editor, at [email protected]. He will then provide feedback regarding the paper’sadequacy for the special issue.
Submissions should be 3,000 to 7,500 words (counting a stan-dard figure or table as 200 words) and should follow the maga-zine’s style and presentation guidelines (see www.computer.org/intelligent/author.htm). References should be limited to 10citations. To submit a manuscript, access the IEEE Computer Soci-ety Web-based system, Manuscript Central, at http://cs-ieee.manuscriptcentral.com/index.html.
