Baked on Brains: Encephalon number 62

The latest Encephalon Carnival is out at the Mouse Trap. Sandy G has done a fine job, and whether you snort it, smoke it, shoot it up, or spoon it right onto the cortex, you should head on over.

Other Encephalon related news is that there is now a dedicated feed.

The next edition of this carnival comes out on February 2nd and is hosted by Shelley Batts at Of Two Minds. The carnival archive and schedule is here.

Testosterone shifts the balance between sensitivity for punishment and reward in healthy young women

Ah, men. We just keep on finding out ways that we're crazy, on average, as compared to women. I've got a little pile of papers on male idiocy, and I'm going to try to write up a couple of them. This one is first for a bunch of reasons, among them:

(1) The journal it is in has a very cool name: Psychoneuroendocrinology. I really would like a paper in that on my CV.

(2) It reports a pretty cool experiment.

(3) I'd already read it.

In a nutshell, this team found that administration of testosterone made healthy young women demonstrably less risk averse and punishment sensitive. Here, as usual, is the full abstract:

Animal research has demonstrated reductions in punishment sensitivity and enhanced reward dependency after testosterone administration. In humans, elevated levels of testosterone have been associated with violent and antisocial behavior. Interestingly, extreme forms of violent and antisocial behavior can be observed in the psychopath. Moreover, it has been argued that reduced punishment sensitivity and heightened reward dependency are crucially involved in the etiology and maintenance of psychopathy. A task that has been proven to be capable of simulating punishment-reward contingencies is the IOWA gambling task. Decisions to choose from decks of cards become motivated by punishment and reward schedules inherent in the task. Importantly, clinical and subclinical psychopaths demonstrate a risky, disadvantageous pattern of decision-making in the task, indicating motivational imbalance (insensitivity for punishment and enhanced reward dependency). Here, in a double-blind placebo-controlled crossover design (n = 12), whether a single administration of testosterone would shift the motivational balance between the sensitivity for punishment and reward towards this tendency to choose disadvantageously was investigated. As hypothesized, subjects showed a more disadvantageous pattern of decision-making after testosterone compared to placebo administration. These findings not only provide the first direct evidence for the effects of testosterone on punishment-reward contingencies in humans, but they also give further insights into the hypothetical link between testosterone and psychopathy.

The paper does pretty much what it says on the box. The experiment was partly occasioned by existing work showing that testosterone affected punishment sensitivity and aggression in animals, which in turn suggests that it may play a role in psychopathy. Jack van Honk (of Utrecht University) and fellow researchers rounded up "12 healthy young women ranging in age from 20 to 25 years" and established absence of psychopathology and substance abuse by interview. Testing was conducted during the follicular phase of the menstrual cycle, "because of the low and stable levels of sex hormones during this period". Each subject received a single dose (0.5mg) of testosterone or a placebo, with the testosterone administration leading to a "10-fold increase in total testosterone". This dosage had been previously established to lead to "significantly elevated physiological responsiveness (vaginal pulse amplitude) in healthy young women after about 4 hours".

Yes, that's right: vaginal pulse amplitude. In case you were wondering (and who wouldn't be) this is the "only physiological response known to possess a
non-habitual nature, thus allowing multiple measures throughout the day". This non-habituating response also justified the 4 hour delay from administration to the other assessment. They don't say how they measured it.

Besides the physiological measure just described, subjects completed a mood assessment by self-report, and the IOWA gambling task. In this task subjects select cards from four decks paying hypothetical rewards and giving (sometimes) hypothetical penalties. Subjects get to chose 100 cards. Two decks are net advantageous, two net disadvantageous, but the 'bad' decks give larger payouts sometimes. Many take it that persistence with the 'bad' decks indicates low sensitivity to punishment or risk. Scoring usually (and here) divides the 100 draws into 5 blocks (of 20) and reflects the relative fraction of good and bad choices for the block.

I've got my doubts about the IOWA task, of which more shortly. That aside, the team found the following:

So more testosterone makes you madder, in the sense of sticking with high payout courses of action that on balance are net punishers, as compared to those less hopped up on the knacker water. Bear in mind that the 10-fold increase left the healthy young women with less of the stuff than the average male of the same age.

I should also mention that van Honk's group has done a pile of other work on testosterone and decision making. If there's a useful web page listing the work, it's been cunningly hidden, but I'm going to blog more of it, and citation indexes will help find it too.

Getting back to my worries about the IOWA task, I don't see why there are four decks instead of two (although this may not matter). I don't see why the instrument combines so many things in such a dirty way, since there are varying magnitudes and frequencies of both rewards and punishments, and the contingencies are unknown to the subject. As a good behaviourist I want to know why individual assessments for sensitivity to delay, and risk, and punishment aren't performed separately and rigorously, and why there isn't something real at stake for the subjects (whose choices don't in fact lead to real reward or punishment). There are some odd results with the IOWA task as well - Chiu and colleagues (2005) found that making new decks with the same net rates of reward but different frequencies of payment led to 'normal' subjects chosing differently, and higher levels of education leads to worsened performance on the IGT (Evans and Colleagues 2004).

Finally, here's a semi-serious question, although it means I'll never be President of Harvard: Does the same protocol lead to any measurable difference in mathematical ability?

References:

Chiu, Y-C., Lin, C-H., Huang, J-T., Lin, S., Lee, P-L, and Hsie, J-C. (2005). Immediate gain is long-term loss: Are there foresighted decision makers in Iowa Gambling Task? (Presentation at the third meeting of the Society for Neuroeconomics, Kiawah Island, September, 2005.)

Evans, C., Kemish, K., and Turnbull, O. (2004). Paradoxical effects of education on the Iowa Gambling Task. Brain and Cognition 54: 240–244.

J van Honk, Dennis J.L.G. Schuttera, Erno J. Hermansa, Peter Putmana, Adriaan Tuitena, Hans Koppeschaar (2004). Testosterone shifts the balance between sensitivity for punishment and reward in healthy young women Psychoneuroendocrinology, 29 (7), 937-943 DOI: 10.1016/j.psyneuen.2003.08.007

Postscript 24 January 2009: Here's a related piece on BBC News, by Simon Baron-Cohen. I doubt anyone knows any more about autism. Or that's what my mate Dave says.

Reversal of cocaine addiction by environmental enrichment

Addiction is one of those topics where there's a lot of morally hysterical yelling that generates plenty of heat but no illumination. It's also an area where we're making rapid and interesting scientific progress. One battle that was -- to my mind -- won a long time ago involved getting addiction taken seriously as in key respects a behaviour like any other. That doesn't mean addictive behaviour isn't odd in some respects, but rather that like any other behaviour it is sensitive to opportunity cost.

This was a tough sell partly because much muddled thinking about addiction sees the activity of the addict as a departure from her normal agency - the addict was roughly supposed to be periodically and unconditionally enslaved to something else. A body of empirical work was done showing that consumption by addicts was sensitive to price - when booze costs more, drunks drink less. And also sensitive to opportunity cost - when booze costs the same, but additional substitutable products become available, so that what is being given up by drinking is more, drunks also drink less. The introduction to Rudy Vuchinich and Nick Heather's 2003 anthology Choice, Behavioral Economics and Addiction (Pergamon) provides a terrific overview of the relevant theory and results.

This paper adds very importantly to what we know. The paper reports that mice addicted to cocaine lose addictive symptoms when exposed to an "enriched environment" during withdrawal. Here's the abstract:

Environmental conditions can dramatically influence the behavioral and neurochemical effects of drugs of abuse. For example, stress increases the reinforcing effects of drugs and plays an important role in determining the vulnerability to develop drug addiction. On the other hand, positive conditions, such as environmental enrichment, can reduce the reinforcing effects of psychostimulants and may provide protection against the development of drug addiction. However, whether environmental enrichment can be used to "treat" drug addiction has not been investigated. In this study, we first exposed mice to drugs and induced addiction-related behaviors and only afterward exposed them to enriched environments. We found that 30 days of environmental enrichment completely eliminates behavioral sensitization and conditioned place preference to cocaine. In addition, housing mice in enriched environments after the development of conditioned place preference prevents cocaine-induced reinstatement of conditioned place preference and reduces activation of the brain circuitry involved in cocaine-induced reinstatement. Altogether, these results demonstrate that environmental enrichment can eliminate already established addiction-related behaviors in mice and suggest that environmental stimulation may be a fundamental factor in facilitating abstinence and preventing relapse to cocaine addiction.

So what did they do? Well, the addicted a group of mice to cocaine, and the split them into two sub-groups during withdrawal. Those in the "enriched environment" had larger enclosures with nicer shelters, a running wheel, and access to toys that were changed weekly.

They studied three different measures of addiction, each in a separate experiment:

(1) Behavioural sensitisation, which is a measure of the increase in effects of cocaine following chronic administration.
(2) Location preference, which is a measure of the extent to which a previously learned spatial association continues to elicit drug-seeking behaviour.
(3) The propensity of cocaine administration to lead to relapse after withdrawal.

The result was dramatic - after 30 days of exposure to the "enriched environment" addiction behavior of all three kinds had disappeared.

In contrast, the mice in the non-enriched environment still showed measurable addiction on each model after 30 days.

What has been found here isn't that availability of substitutable 'alternatives' makes a difference to the course of addiction. First, we knew that already. Second, the mice in the "enriched environment" weren't chosing between play and cocaine, so that the availability of play raised the opportunity cost of cocaine. They had more options during withdrawal.

The authors suggest two possible mechanisms. First, the "enriched environment" was less stressful, and stress is know to mediate relapse and drug-seeking. Second, the "enriched environment" provided opportunities to learn a wider range of reward-seeking behaviours, which reduced the power of the previously learned patterns relative to controls which had fewer such opportunities. These are not mutually exclusive, and both seem highly plausible (there's also independent evidence for each).

There's plenty more to be found out, but this is a very useful addition to our understanding. For another brief notice of the same research see this on Science Daily.

M. Solinas, C. Chauvet, N. Thiriet, R. El Rawas, M. Jaber (2008). From the Cover: Reversal of cocaine addiction by environmental enrichment Proceedings of the National Academy of Sciences, 105 (44), 17145-17150 DOI: 10.1073/pnas.0806889105

Exposure to Scientific Theories Affects Women’s Math Performance

Hosting the December 2008 Praxis Carnival got me thinking again (partly courtesy of submissions by PodBlack Cat and Dr Isis) about the representation of women in science. When a seasonal clean out of one of the many folders filled with PDFs that I "really ought to read some time" turned up this paper, I figured to blag it.

It's not controversial that in many sciences, including mathematics, women are under-represented compared to the ratios in the population at large. There's a lot of discussion over the questions of what precise mix of factors might explain this imbalance, and what sorts of policies might reduce it.

This study looked at women's performance at mathematics. The subjects completed a test including two mathematics sections separated by a reading comprehension section. The middle section was an experimental manipulation, with one of the following four essays:

(G) This essay argued that there were mathematics-related sex differences, and that the explanation was genetic.

(E) This essay argued that there were mathematics-related sex differences, and that the explanation was experiential.

(NS) This essay argued that there were no mathematics-related sex differences.

(S) This essay primed the question of sex without making reference to differences in mathematical ability.

The hypothesis was that in the second test participants in condition (G) and (S) would underperform those in condition (E) and (NS). This is just what they found, a result that was replicated in a schematically similar study where the manipulation was heard, rather than read. See the figure below.


This isn't surprising at all - it's consistent with a pile of established social psychology on the effectiveness of stereotypes. But it's definitely important.

Among other notices of this little paper, see The CIRTL Cafe.

I. Dar-Nimrod, S. J. Heine (2006). Exposure to Scientific Theories Affects Women's Math Performance Science, 314 (5798), 435-435 DOI: 10.1126/science.1131100

Friday science clip: Robot with "Rat Brain"

This is very, very, very cool:

Neural encoding of the concept of nest in the mouse brain

We're finding out cool stuff about the Hippocampus all the time. This paper, by Lin and colleagues, in PNAS last year, is a fine example. As it says on the box, they find evidence of interestingly selective responses - responses relating to nests - in the mouse brain. Thanks to Michael over at Ionian Enchantment for drawing my attention to this paper.

The neurons in question were in the hippocampus. This brain region is known to be important for a variety of memory related, but also decision making, functions. (See my earlier posting on hippocampal representation of physical locations forward of actual movement in the rat.) The authors of this study describe this as evidence of encoding of the 'concept of nest'.

Aside: Some researchers have tiresomely mantained that conceptual thinking is distinctively human. It doesn't help that philosophers and psychologists don't use 'concept' univocally, and rarely in the same ways as each other. Appropriate behaviour is going to be reasonably responsive to interesting equivalence classes (predators, escape routes, possible mates, nest materials, etc., etc.) and the cognitive processes relating to these classes is at least necessary for 'conceptual' cognition, even if some might want to demand additional features (like compositionality).

Back to the main story: This crowd stuck the tools for ensemble recording into region CA1 of the hippocampus of each of the subject mice, and the exposed the mice to a variety of nests and objects that were in some sense nest-like, but not appropriate (being upside down, far too big, of poor materials, etc.) as well as unusual but suitable nests (like oddly shaped ones). The mice also approached the target objects from different directions. In the luxurious world of the lab-mouse, a nest is a paper or plastic cup about the right size to curl up in (see the images with the supplemental information.)

Their analysis found three categories of interesting activity:

• "Transient-on" cells responded temporarily to, for example, initial encounter with the home nest, but not to a similar circular object (a water cup) or to a nest when upside down. (These cells are associated with activity amounting to "there's a nest!") See the movie below.
  
• "Persistent-on" cells responded persistently to presence in the home nest, irrespective of direction of original approach. (These cells are associated with activity amounting to "I'm in a nest!")
• "Persistent-off" cells were the opposite of the above, going quiet outside nests. ("I'm not in a nest!")

Here is one of the movies from the supplemental information, showing a transient on cell about its business as the mouse goes about its:




Additional sub-experiments placed appropriate nests under odd conditions (e.g. being under a glass floor) and found that the 'nest cells' weren't keen on nests that weren't readily useable. This complements the finding that the cells didn't like upside down (otherwise appropriate) nests either.

This is very cool. Encodings like this are efficient resources for other processing, like selecting responses (whether to "poop here" or not), and populating more plainly spatial memory with meaning ("here be a nest"). The Johnson and Redish paper I blogged previously suggests that "here be good stuff" is encoded in some way that indexes to spatial memory, although their main point is to show something about the use of that memory in action selection. It also wasn't obvious in advance that the encoding would be, as it seems to be, primarily functional, although I'd have been surprised if it wasn't.

It would be good to know more about how these encodings are constructed.

I wonder if any of those Heideggereans, prone to letting off clouds of portentous bull about how science can't tell us anything about the real nature of embodied being in the world, or how it is that we come to relate in an immediate way to objects that are appropriate for our purposes, are paying attention. I doubt it, and doubt that they read PNAS.

Lin, L., Chen, G., Kuang, H., Wang, D., Tsien, J.Z. (2007). Neural encoding of the concept of nest in the mouse brain. Proceedings of the National Academy of Sciences, 104(14), 6066-6071. DOI: 10.1073/pnas.0701106104

Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point

A terrific paper by Adam Johnson and A. David Redish in The Journal of Neuroscience from November 2007 shows that neurons in the hippocampus transiently represent paths forward of an animal when the animal is deciding between more than one path.

We've known since the late 1970s that some hippocampal cells are spatially selective. The cells came to be called 'place cells' and the spaces to which they responded their 'place fields'.

We're still finding out what exactly they're up to, and how they support the computational demands of different sorts of problem. Johnson and Redish trained rats to navigate two kinds of maze for reward, in order to ask some specific questions about what part of the hippocampus was up to at short time scales. The kind I'll focus on here is illustrated here: a 'multiple-T maze', with a sequence of choices. (The picture is figure 1 from the paper.)

Each rat also had multiple tetrode recording devices implanted into the CA3 region of his hippocampus. Using Bayesian decoding, movement of the rats through the maze was related to measured activity in the hippocampus, to identify place fields and corresponding patterns of neuronal firing. (The analysis associated places with ensembles of spikes, rather than individual cells.)

The upshot of this analysis was, partly, a representation of hippocampus activity associated with the physical space of the maze.

The really exciting part of the work was relating those two (physical space and hippocampus activity) with a third stream of data relating to the changing physical position of the rat, and the specific contingencies of the task faced by the rat. Rats are known sometimes to pause and look around when they're at decision points in mazes. In a properly designed maze where observed behaviour tells us that they're pretty good at taking the turn that gives them better reward, we have a decent idea what information they're drawing on when they eventually start moving.

Johnson and Redish observed that at the decision points the firing patterns in the hippocampus showed that place fields forward of the animal, and down the available paths, were active. This very clearly suggests that the hippocampus is involved in mediating memory functions relating to decision making.

There's a telling graphical representation of the key finding, that conceals all the analysis that went into this. (My second figure is one panel of figure 7 of the paper.) The white circle shows the physical position of the rat as it moves through the maze, and the blue the place fields arrayed to correspond to the physical space of the maze. The coloured pixels on the place field show activity as recorded by the tetrodes at the time - mostly corresponding to the space where the rat is, but clearly sweeping ahead (especially at 120-280 milliseconds) while the rat pauses at the decision point.

This is even more vivid when you see an animation, and there is one in the supplemental materials to the paper - you can also view it (this is slowed down, not actual speed) here:



This animation is a really terrific example of how very complex data can be made totally compelling when presented in a coherent way, and especially one that makes effective use of the number of dimensions in the graphic. Physical space, the position of the rat, and the activity corresponding to place fields are co-ordinated, and then updated over time so that the the fact that some cells "Transiently Encode Paths Forward of the Animal at a Decision Point" is totally clear. Bravo.



Johnson, A., Redish, A.D. (2007). Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point. Journal of Neuroscience, 27(45), 12176-12189. DOI: 10.1523/JNEUROSCI.3761-07.2007

Marketing actions can modulate neural representations of experienced pleasantness

A paper published in PNAS in January of this year has been getting a fair bit of attention. The paper, by Plassman and colleages, is called Marketing actions can modulate neural representations of experienced pleasantness, and the bottom line result is that experimental subjects rated the very same wines as more pleasant, and had greater activity in some brain regions, when that wine was presented as being more expensive.

The subjects of the experiment had been told that they were tasting five different wines, with known prices, where in fact two of the wines were repeated, each time at a different price (one at USD5 and 45, the other at USD10 and 90).

Among the other accounts of this research in the blogosphere see Ben Goldacre, Epistemist, Neurocritic and Mindhacks.

The result isn't particularly surprising, and it seems to me that some fairly obvious variations on the task would have helped get clearer on what is going on. Also, I find some of the ways the results are presented a little unhelpful.

First, let me say a bit more about how the study was conducted. Subjects rated wines (all Cabernet Sauvingnons) without price information to establish that ratings of the wines were reasonably consistent. They were. Under scanning, with squirts of wine delivered through plastic tubes, they made a series of ratings (of either 'pleasantness' or 'intensity' on a 6 point scale) interspersed with mouth rinsing. Every wine was presented along with its 'price', and subjects held the wine in their mouth for 10 seconds before rating it. One of the repeated wines was presented half of the time at actual price of USD5 and otherwise marked up to USD45. The other repeated wine was presented at its real price of USD90, and marked down to USD10.

Subjects rated the same wine as more pleasant when more expensive, and, as hypothesised, there was more activity in their medial orbitofrontal cortext (mOFC), because other studies suggest that it is in the business of representing 'experienced pleasantness'.

So far so good. But the authors open by saying that "A basic assumption in economics is that the experienced pleasantness (EP) from consuming a good depends only on its intrinsic properties and on the state of the individual." They give a citation for this claim, but as so often the citation is to another publication endorsing the same prejudice, rather than a piece of evidence from the history of economics or utility theory. Actually, even Bentham was quite emphatic that as well as direct determinants of the utility from an experience (intensity, duration, certainty and propinquity) but also three important relational properties - fecudity (the tendency of a pleasure to be accompanied by others), purity (the tendency of a pleasure not to be followed pain) and extent (the number of persons who share a pleasure).

So price could well be an indicator of fecundity or purity (suggesting fine rather than lousy food, toadying service, well-dressed dates, etc.) even to a Benthamite. I wonder if there will ever be a day when people can manage to report results like this without feeling the need to announce a caricature or worse outright misrepresentation of the history of economics.

An earlier (2004) paper in Neuron, called Neural Correlates of Behavioral Preference for Culturally Familiar Drinks by McClure and colleagues, is clearer, and covers related ground. McClure et al found that on average subjects liked Coke more when they knew what they were drinking, than when they didn't know whether it was Pepsi or Coke. In their case the neural correlates of the marketing related preference weren't the mOFC, but a set of areas including the hippocampus and the ventro-medial PFC.

We need more experiments to try to isolate effects of scarcity (for example telling subjects that what they were tasting came from one of two classes, set up so that one was much scarcer), as well as Bentham style fecundity, purity and extent. That would also help shed light on how the different components of prior beliefs are neurally implemented and how they interact.

Two more quibbles, then a speculation.

Quibble 1: The abstract of the Plassman paper, and some remarks in the paper, suggest that this has something to do with "decision making". Indirectly it probably does, but the subjects in the experiment didn't make any decisions at all - they merely reported ratings. The subjects in the McClure et al study did make decisions forced choices between Coke and Pepsi, where there was a definite opportunity cost to each choice.

We just don't know whether subjects would have kept on buying the "UDS90 wine" over the "USD10" wine having tasted both and when spending actual money. We know that people like free wine more when they think it's expensive, and that's not the same thing at all.

Quibble 2: There's some plain sloppiness here, whether the fault of the authors or PNAS. On p1051 the text refers to figure 1D as representing data from the 'postscanning blind test' whereast the figure caption indicates that it is panel 1E that represents that data. On page 1053 it says that "Two of the three wines were administered twice, once identified by their actual retail price and once by a 900% markup (wine 1: $5 real retail price, $45 fictitious price) or a 900% reduction (wine 2: $90 real retail price, $10 fictitious price)." Yeah right, a 900% reduction of USD90 leaves you with ten dollars. There I was thinking that 10 was about 11 percent of 90.

Then the speculation. I'd really like to see a roughly analogous experiment in the case of politics. Hotelling effects drive the content of at least some policies held by opposing parties in democracies towards near-indistinguishability. But citizens typically hold strong preferences between parties. Why not have subjects rate their level of agreement with policy claims and vary whether they're told that the policies are those of a Democrat or Republican, or a Tory or Labourite, etc.? I'd bet that the framing would make a difference, but I'm curious as to whether any coherent brain data would come along with that.

Oh, and one last thing. My occasional wine society (a bunch of friends who get together to eat and drink) are doing the opposite tomorrow night - a blind tasting to see if we can pick the expensive wines out of a set with 50% of the cheapest wines we can find. I'll report back. Odds are we'll drink too much and not discover anything new. Also, sadly, we don't have a brain scanner.

Reward prediction, yes. Stimulus categorization, no.

Here's a follow up to an earlier posting, my first stab at blagging a piece of proper (peer-reviewed) research. The paper, by Pan et al, is published in Nature Neuroscience, and is called Reward prediction based on stimulus categorization in primate lateral prefrontal cortex.

As I said before, this paper is a welcome addition to our knowledge in the field of neuroeconomics, but the results are presented in a way that I regard as partly confused or misleading.

The subjects of the experiments were monkeys. The 'reward prediction' referred to in the title is their making choices, by means of saccades to specific targets, that correctly led to greater reward while activity of lateral prefrontal cortex (LPFC) neurons was. There's nothing new about this - monkeys have been saccading to targets that predicted reward with wires stuck in their LPFCs for a while. What's relatively new is the specific challenge they faced identifying the correct target.

First they were trained that each of two different sequences of visual stimuli interspersed with saccade tasks (A1-A2-A3, and B1-B2-B3 - see figure) led to equal reward after the final saccade and stimulus. Then they were separately trained that A3 predicted greater reward than B3.

Then they were offered a 'novel' choice between A1 and B1, and 'correctly' chose A1 most of the time.

Again, as I said before, the behavioral result is cool if not surprising (it's old news in animal learning). The monkeys 'could' have showed no relative preference in the final task, given that the first stimulus in each sequence had not been directly associated with different relative reward at the end of the sequence. And the recordings from large numbers of neurons in the LPFCs of the monkeys found that some were preferentially active for rewards, some for stimuli (A1 or B1), and some for stimulus-reward interactions (e.g. associations between A1 and the differential reward associated with A3 over B3).

The title of the paper, the abstract, and various remarks in it, suggest that we should read this as having something to do with 'categorization', and that the work presents some kind of problem for temporal-difference learning approaches to reward prediction. I don't think either point is correct, even though the work is interesting.

On the first point, categorization has an established meaning in cognitive science. According to the first sentence of the entry on 'Categorization' in the MIT Encylopedia of the Cognitive Sciences, "Categorization, the process by which distinct entities are treated as equivalent, is one of the most fundamental and pervasive cognitive activities." But this paper isn't about treating distinct entities as equivalent except in the minimal sense that numerically distinct instances of A1 are treated as equivalent. If that's all it takes to make a categorization experiment, just about every neuroeconomics experiment has been a categorization experiment. (Pan and colleagues don't give a definition of categorization, so it's not clear enough what they mean by it.)

On the second point, predictor-valuation (or temporal difference) models can cope with sequences of predictors where reward only follows the final predictor. (Montague and Berns (2002) report a sequence with two stimuli, where the middle one was sometimes unpredictably timed relative to the first and the reward.) Predictors are predictors of the values of states, and associations establish informational links between states. So making a state have a higher value, which is what the middle stage of the Pan et al study did by making A3 worth more than B3, should raise the values of predictors of ways of getting to those states. This new paper sheds interesting light on how the brain handles predictor valuation in cases where predictors are chained together, but it isn't reason to think that we've found something that temporal difference learning can't deal with.

Or am I missing something?

Reward prediction based on 'stimulus categorization' in primate LPFC

A new paper in Nature Neuroscience adds something genuinely interesting to our growing understanding of neuroeconomics, but I've got reservations about how the authors describe the significance of their own work. The paper is called Reward prediction based on stimulus categorization in primate lateral prefrontal cortex.

The authors first trained monkeys that reward was delivered after each of two different sequences of three stimuli (see picture below) as long as the correct saccades were made in between the stimuli. (Saccadic eye movements are a common choice action in these experiments - they don't require head movements, can easily be tracked, and some important early work in the field related economic values of targets to the activity of neurons involved in making the different saccades.)

In the main experiment, following the above learning, monkeys first learned that one of the two final stimuli from the initial task was associated with a larger reward than the other, and then offered a novel choice between the first member of the two sequences. They mostly 'correctly' opted for the first stimulus of the sequence that ended with the stimulus that was now associated with relatively greater reward.

The behavioral result is cool if not surprising (it's old news in animal learning). The monkeys 'could' (according to a certain simplistic behaviourism they would) initially have showed no relative preference in the final task, given that the first stimulus in each sequence had not been directly associated with different relative reward at the end of the sequence. (The magnitudes of the rewards in the first training stage had been equal for both sequences.) And there's more - the authors recorded large numbers of neurons in the lateral prefrontal cortex (LPFC) and found that some were preferentially active for rewards, some for stimuli, and some for stimulus-reward interactions.

But the title of the paper, the abstract, and remarks dotted around in it, suggest that the authors think this has something to do with the question of whether "LPFC predicts reward by means of temporal-difference learning or by a model-based method", and to have something to do with "categorization" or "category members that have not been linked directly with any experience of reward."

This seems to suppose a conflict that isn't necessary. As Montague and Berns (2002) showed predictor-valuation (or temporal difference) models cope just fine with sequences of predictors where reward only follows the final predictor (their sequence had two stimuli with varying timing of the second stimulus and the reward). Predictors are predictors of the values of states, and associations establish informational links between states. So making a state have a higher value should raise the values of predictors of ways of getting to those states. This new paper sheds interesting light on how the brain handles predictor valuation in cases where predictors are chained together, but it isn't reason to think that we've found something that temporal difference learning can't deal with.

And 'categorization' refers to a large and complex literature in cognitive science. It would be better not to invoke it imprecisely, or as though all that was needed to make things clear was a bald contrast with "simple associative theories".