An efficient knowledge transfer solution to a novel SMDP formalization of a broker's decision problem

Talla Kuate, Rodrigue, Chli, Maria and Wang, Hai H. (2015). An efficient knowledge transfer solution to a novel SMDP formalization of a broker's decision problem. IN: AAMAS'15 International Conference on Autonomous Agents and Multi Agent Solutions. TUR: ACM.

Abstract

This paper introduces a new technique for optimizing the trading strategy of brokers that autonomously trade in re- tail and wholesale markets. Simultaneous optimization of re- tail and wholesale strategies has been considered by existing studies as intractable. Therefore, each of these strategies is optimized separately and their interdependence is generally ignored, with resulting broker agents not aiming for a glob- ally optimal retail and wholesale strategy. In this paper, we propose a novel formalization, based on a semi-Markov deci- sion process (SMDP), which globally and simultaneously op- timizes retail and wholesale strategies. The SMDP is solved using hierarchical reinforcement learning (HRL) in multi- agent environments. To address the curse of dimensionality, which arises when applying SMDP and HRL to complex de- cision problems, we propose an ecient knowledge transfer approach. This enables the reuse of learned trading skills in order to speed up the learning in new markets, at the same time as making the broker transportable across market envi- ronments. The proposed SMDP-broker has been thoroughly evaluated in two well-established multi-agent simulation en- vironments within the Trading Agent Competition (TAC) community. Analysis of controlled experiments shows that this broker can outperform the top TAC-brokers. More- over, our broker is able to perform well in a wide range of environments by re-using knowledge acquired in previously experienced settings.

Divisions: Engineering & Applied Sciences > Computer Science
Event Title: 14th International Conference on Autonomous Agents and Multi Agent Systems
Event Type: Other
Event Location: Istanbul Congres Center
Event Dates: 2015-05-04 - 2015-05-08
Uncontrolled Keywords: artificial intelligence,learning,knowledge transfer,reinforcement learning,MDP,SMDP,broker agent,Artificial Intelligence,Software,Control and Systems Engineering
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
http://www.aama ... aamas/p1735.pdf (Publisher URL)
Published Date: 2015-05-04
Authors: Talla Kuate, Rodrigue
Chli, Maria ( 0000-0002-2840-4475)
Wang, Hai H.

Download

[img]

Version: Draft Version


Export / Share Citation


Statistics

Additional statistics for this record