Sunday, March 18, 2018

Universal Linguistic Decoders are Everywhere

Pereira et al. (2018) - click image to enlarge

No, they're not. They're really not. They're “everywhere” to me, because I've been listening to Black Celebration. How did I go from “death is everywhere” to “universal linguistic decoders are everywhere”? I don't imagine this particular semantic leap has occurred to anyone before. Actually, the association travelled in the opposite direction, because the original title of this piece was Decoders Are Everywhere.1 {I was listening to the record weeks ago, the silly title of the post reminded me of this, and the semantic association was remote.}

This is linguistic meaning in all its idiosyncratic glory, a space for infinite semantic vectors that are unexpected and novel. My rambling is also an excuse to not start out by saying, oh my god, what were you thinking with a title like, Toward a universal decoder of linguistic meaning from brain activation (Pereira et al., 2018). Does the word “toward” absolve you from what such a sage, all-knowing clustering algorithm would actually entail? And of course, “universal” implies applicability to every human language, not just English. How about, Toward a better clustering algorithm (using GloVe vectors) for inferring meaning from the distribution of voxels, as determined by an n=16 database of brain activation elicited by reading English sentences?

But it's unfair (and inaccurate) to suggest that the linguistic decoder can decipher a meandering train of thought when given a specific neural activity pattern. Therefore, I do not want to take anything away from what Pereira et al. (2018) have achieved in this paper. They say:
  • “Our work goes substantially beyond prior work in three key ways. First, we develop a novel sampling procedure for selecting the training stimuli so as to cover the entire semantic space. This comprehensive sampling of possible meanings in training the decoder maximizes generalizability to potentially any new meaning.”
  • “Second, we show that although our decoder is trained on a limited set of individual word meanings, it can robustly decode meanings of sentences represented as a simple average of the meanings of the content words. ... To our knowledge, this is the first demonstration of generalization from single-word meanings to meanings of sentences.”
  • “Third, we test our decoder on two independent imaging datasets, in line with current emphasis in the field on robust and replicable science. The materials (constructed fully independently of each other and of the materials used in the training experiment) consist of sentences about a wide variety of topics—including abstract ones—that go well beyond those encountered in training.”

Unfortunately, it would take me days to adequately pore over the methods, and even then my understanding would be only cursory. The heavy lifting would need to be done by experts in linguistics, unsupervised learning, and neural decoding models. But until then...

Death is everywhere
There are flies on the windscreen
 For a start
 Reminding us
 We could be torn apart

---Depeche Mode, Fly on the Windscreen


1 Well, they are super popular right now.


Pereira F, Lou B, Pritchett B, Ritter S, Gershman SJ, Kanwisher N, Botvinick M, Fedorenko E. (2018). Toward a universal decoder of linguistic meaning from brain activation. Nat Commun. 9(1):963.

Come here
Kiss me
Come here
Kiss me


Subscribe to Post Comments [Atom]


At March 19, 2018 9:17 AM, Anonymous Anonymous said...

It is pretty spectacular how little understanding of language one must have to think you can get anywhere interesting with meaningfully decoding brain activity, as if there is a common source code across people that would allow to go much beyond the crude level that these attempts result in. I blame Chomsky.

At March 22, 2018 5:43 AM, Blogger Francisco Pereira said...

As the person behind the wheel, I must say this was the one review I was looking forward to. People are people, and will write suggestive titles, even if they follow a policy of truth. That said, I am grateful for the mercy in you, and assure you that it is just a question of time till we publish something better than this; there's just always something to do, if one is aiming for the sweetest perfection. Everything counts, in large amounts, and this is particularly the case when reprocessing data...

At March 22, 2018 1:14 PM, Blogger The Neurocritic said...


I will thank you most of all for the respect you have for me. Your work is a lot like life but a game with added reality. Can you say what you're trying to play anyway? It's a competitive world and going backwards is not an option. Words are meaningless and forgettable. Forgetting all you couldn't do today -- I'll drink to that tonight. I will leave it there on a note full of hope not despair.


Post a Comment

Links to this post:

Create a Link

<< Home

eXTReMe Tracker