An efficient knowledge transfer solution to a novel SMDP formalization of a broker's decision problem


This paper introduces a new technique for optimizing the trading strategy of brokers that autonomously trade in re- tail and wholesale markets. Simultaneous optimization of re- tail and wholesale strategies has been considered by existing studies as intractable. Therefore, each of these strategies is optimized separately and their interdependence is generally ignored, with resulting broker agents not aiming for a glob- ally optimal retail and wholesale strategy. In this paper, we propose a novel formalization, based on a semi-Markov deci- sion process (SMDP), which globally and simultaneously op- timizes retail and wholesale strategies. The SMDP is solved using hierarchical reinforcement learning (HRL) in multi- agent environments. To address the curse of dimensionality, which arises when applying SMDP and HRL to complex de- cision problems, we propose an ecient knowledge transfer approach. This enables the reuse of learned trading skills in order to speed up the learning in new markets, at the same time as making the broker transportable across market envi- ronments. The proposed SMDP-broker has been thoroughly evaluated in two well-established multi-agent simulation en- vironments within the Trading Agent Competition (TAC) community. Analysis of controlled experiments shows that this broker can outperform the top TAC-brokers. More- over, our broker is able to perform well in a wide range of environments by re-using knowledge acquired in previously experienced settings.

Divisions: Engineering & Applied Sciences > Computer Science
Event Title: 14th International Conference on Autonomous Agents and Multi Agent Systems
Event Type: Other
Event Location: Istanbul Congres Center
Event Dates: 2015-05-04 - 2015-05-08
Uncontrolled Keywords: artificial intelligence,learning,knowledge transfer,reinforcement learning,MDP,SMDP,broker agent,Artificial Intelligence,Software,Control and Systems Engineering
ISBN: 978-1-4503-3771-7
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
http://www.aama ... aamas/p1735.pdf (Publisher URL)
PURE Output Type: Conference contribution
Published Date: 2015-05-04
Authors: Talla Kuate, Rodrigue
Chli, Maria ( 0000-0002-2840-4475)
Wang, Hai H. ( 0000-0002-4192-5363)



Version: Draft Version

Export / Share Citation


Additional statistics for this record