“Negative reinforcement” in early TAMER work (2008-2010)

In psychology, negative reinforcement is the removal of an aversive stimulus, causing an increase in associated behavior. In some early work on TAMER, I used this term to mean a negatively valued feedback signal, which is more akin to punishment and typically decreases associated behavior. AI and machine learning often borrow terms from psychology and give them new meanings (e.g., “shaping rewards”), so this usage isn’t necessarily an error, but I think it’s bad practice nonetheless.

Typo in derivation of belief update in The nature of belief-directed exploratory choice in human decision-making (2012)

During typesetting, a typo was introduced in the derivation in the upper right-hand column of this paper. The first line should read