Beyond goal-rationality:Traditional action can reduce volatility in socially situated agents

Abstract

Systems that pursue their own goals in shared environments can indirectly affect one another in unanticipated ways, such that the actions of other systems can interfere with goal-achievement. As humans have evolved to achieve goals despite interference from others in society, we thus endow socially situated agents with the capacity for social action as a means of mitigating interference in co-existing systems. We demonstrate that behavioural and evolutionary volatility caused by indirect interactions of goal-rational agents can be reduced by designing agents in a more socially-sensitive manner. We therefore challenge the assumption that designers of intelligent systems typically make, that goal-rationality is sufficient for achieving goals in shared environments.