AI Debate 2: Night of a thousand AI scholars

A year ago, Gary Marcus, a frequent critic of deep learning forms of AI, and Joshua Bengio, a leading proponent of deep learning, faced off in a two-hour debate about AI at Bengio’s MILA institute headquarters in Montreal.

Wednesday evening, Marcus was back, albeit virtually, to open what is now the second installment of what has become a planned annual debate on AI, under the title “AI Debate 2: Moving AI Forward.” (You can follow the proceedings from 4 pm to 7 pm on Montreal.ai’s Facebook page.)

Vincent Boucher, president of the organization Montreal.AI, who had helped to organize last year’s debate, opened the proceedings, before passing the mic to Marcus as moderator. 

Marcus said 3,500 people had pre-registered for the evening, and at the start, 348 people were live on FaceBook. Last year’s debate had 30,000 by the end of the night, noted Marcus.

Bengio was not in attendance, but the evening featured presentations from sixteen scholars: Ryan Calo, Yejin Choi, Daniel Kahneman, Celeste Kidd, Christof Koch, Luis Lamb, Fei-Fei Li, Adam Marblestone, Margaret Mitchell, Robert Osazuwa Ness, Judea Pearl, Francesco Rossi, Ken Stanley, Rich Sutton, Doris Tsao and Barbara Tversky. 

“The point is to represent a diversity of views,” said Marcus, promising a three hours that might be like “drinking from a firehose.”

Marcus opened the talk by recalling last year’s debate with Bengio, which he said had been about the question, Are big data and deep learning alone enough to get to general intelligence?

This year, said Marcus, there had been a “stunning convergence,” as Bengio and another deep learning scholar, Facebook’s head of AI, Yann LeCun, seemed to be making the same points as Marcus about 

There was also a Jürgen Schmidhuber paper recently that seemed to be part of that convergence, said Marcus. 

Following this convergence, said Marcus, it was time to move to “the debate of the next decade: How can we move Ai to the next level.”

Each of the sixteen speakers spoke for roughly five minutes about their focus and what they believed AI needs. Marcus compiled a nice reading packet you can check out for the scholars.

Fei-Fei Li, the Sequoia Professor of computer science at Stanford University, talked about what she called the “north star” of AI.

The north star, she said was interaction with the environment. 

Following LI, Luís Lamb, who is professor of computer science at the Universidade Federal do Rio Grande do Sul in Brazil, talked about “neurosymbolic AI,” a topic he had presented a week prior at the NeurIPS AI conference.

Calling Marcus and Judea Pearl “heroes,” Lamb said the neurosymbolic work from Marcus’s concepts in Marcu’s book, “The Algebraic Mind,” including the need to manipulate symbols, on top of neural networks. “We need a foundational approach based both on logical formalization” such as Pearl’s “and machine learning,” said Lamb. 

Lamb was followed by Rich Sutton, distinguished scientist at DeepMind. Sutton recalled neuroscientist David Marr, who he called one of the great figures of AI. He described Marr’s notion of three levels of processing that were required: computational theory, representation and algorithm, and hardware implementation. Marr was particularly interested in computational theory, but “there is very little computational theory” today in AI, said Sutton.

Thing such as gradient descent are “hows,” said Sutton, not the “what” that computational theory needed.

“Reinforcement learning is the first computational theory of intelligence,” declared Sutton. Other possibilities were predictive coding, bayesian inference, and such. 

“AI needs an agreed-upon computational theory of intelligence,” concluded Sutton, “RL is the stand-out candidate for that.”

Next came Pearl, who has written numerous books about causal reasoning, including the Times best-seller, The Book of Why. His talk was titled, “The domestication of causality.”

“We are sitting on a goldmine,” said Pearl, referring too deep learning. “I am proposing the engine that was constructed in the causal revolution to represent a computational model of a mental state deserving of the title ‘deep understanding’,” said Pearl. 

Deep understanding, he said, would be the only system capable of answering the questions, “What is?” “What if?” and “If Only?”

Next up was Robert Ness, on the topic of “causal reasoning with (deep) probabilistic programming.”

Ness said he thinks of himself as an engineer, and is interested in building things. “Probabilistic programming will be key,” he said, to solving causal reasoning. Probabilistic programming could build agents that can reason counter-factually, a key to causal reasoning, said Ness. That was something he felt was “near and dear to my heart.” This, he said, could address Pearl’s question for the “If only?” 

Next was Ken Stanley, who is Charles Millican professor of computer science at the University of Central Florida. Stanley took a sweeping view of evolution and creativity.

“We layer idea upon idea, over millennia, from fire and wheels up to space stations, which we would call an open-ended system.” Stanley said evolution was a parallel, a “phenomenal system,” he said, that had produced intelligence. “Once we began to exist, you got all the millennia of creativity,” said Stanley. “We should be trying to understand these phenomena,” he said. 

Following Stanley was Yejin Choi, the Brett Helsel Associate Professor of Computer Science at the University of Washington.

She recalled Roger Shepard’s “Monsters in a Tunnel,” examples form “The Enigma of Reason,” led hear to talk about the importance of language, and also “reasoning as generative tasks.” “We just reason on the fly, and this is going to be one of the key fundamental challenges moving forward.”

Choi said new language models like “GPT-4, 5, 6 will not cut it.” 

Marcus took a moment to identify convergence among the speakers: on counter-factuals, on dealing with the unfamiliar and open-mindedness; on the need to integrate knowledge, to Pearl’s point about data alone not being enough; the importance of common sense (to Choi’s point about the “generative.”)

Marcus asked the panel, “Okay, we’ve seen six or seven different perspectives, do we want to put all those together? Do we want to make reinforcement learning compatible with knowledge.” 

Lamb noted that neurosymbolic systems are not about “the how.” “It’s important to understand we have the same aims of building something very solid in terms of learning, but representation precedes learning,” he said. 

Marcus asked the scholars to reflect on modularity. Sutton replied that he would welcome lots of ways to think about the overall problem, not just representation. “What is it? What is the overall thing we’re trying to achieve” with computational theory, he prompted. “We need multiple computational theories,” said Sutton.

Among audience questions, one individual asked how to reason in a causal framework with unknown, or novel variables. Ness talked about COVID-19 and studying viruses. Prior viruses helped to develop causal models that were able to be extended to SARS-CoV-2. He referred to dynamic models such as the ubiquitous “SEI” epidemiological model, and modeling that in probabilistic languages. “You encounter it a lot where you hit a problem, you’r unfamiliar, but you can import abstractions from other domains.  

Choi pointed out “Humans are capable of believing strange things, and do strange causal reasoning,” she said, “do we want to build a human-like system?” Choi said one thing that is interesting about humans is the ability to communicate so much knowledge in natural language, and to learn through natural language. 

Marcus asked Li and Stanley about “neuro-evolution.” “What’s the status of that field?” he asked.

Stanley said the field might use evolutionary algorithm, but also might just be an attempt to understand open-ended phenomena. He suggested “hybridizing” evolutionary approaches with learning algorithms.

“Evolutiona is one of the greatest, richest experiments we have seen in intelligence,” said Li. She suggested there will be “a set of unifying principles behind intelligence.” “But I’m not dogmatic about following the biological constraints of evolution, I want to distill the principles,” said Li.

After the question break, Marcus brought on Tversky, emeritus professor of psychology at Stanford. Her focus was encapsulate on the screen, “All living things must act in space … when motion ceases, life ends.” Tversky talked about how people make gestures, making spatial motor movements that can affect how well or how poorly people think. 

“Learning, thinking, communication,. cooperation, competing, all rely on actions, and a few words.

Daniel Kahneman was next, Nobel laureate and author of Thinking, Fast and Slow, a kind of bible for some in issues of reasoning in AI. Kahneman noted he has been identified with the book’s paradigm, “System 1 and System 2 thinking.” One is a form of intuition, the other a form of higher-level reasoning.  

Kahneman said System 1 could encompass any things that are non-symbolic, but it was not true to say System 1 is a non-symbolic system. “It’s much to rich for that,” he said. “It holds a representation of the world, the representation that allows a simulation fo the world,” which humans live with, said Kahneman. “Most of the time, we are in the valley of the normal, events that we expect.” The model accepts many events as “normal,” said Kahneman, even if unexpected. It rejects others. Picking up Pearl, Kahneman said a lot of counter-factual reasoning was in System 1, where an innate sense of what is normal would govern such reasoning. 

Following Kahneman was Doris Tsao, a professor of biology at CalTech. She focused on feedback systems, recalling early work on neurons of McCulloch and Pitts. Feedback is essential, she said, citing back-propagation in multilayer neural nets. Understanding feedback may allow one to build more robust vision systems, said Tsao. Tsao said feedback systems might help to understand phenomena such as hallucinations. Tsao said she is “very excited about the interaction between machine learning and systems neuroscience.”

Next was Marblestone of MIT, previously a research scientist at DeepMind continued the theme of neuroscience. He said making  observations of the brain, trying to abstract to a theory of functioning, was “right now at a very primitive level of doing this.” Marblestone said examples of neural nets, such as convolutional neural networks, were copying human behavior.  

Marblestone spoke about moving beyond just brain structure to human activity, using fMRIs, for example. “What if we could model not just how someone is categorizing an object, but making the system predict the full activity in the brain?” 

Next came Koch, a researcher with the Allen Institute for Brain Science in Seattle. He started with the assertion, “Don’t look to neuroscience for help with AI.” 

The Institute’s large-scale experiments reveal complexity in the brain that is far beyond anything seen in deep learning, he said. “The connector reveals the brains are not just made out of millions and billions of cells, and a thousand different neuronal cell types, that differentiate by the genes expressed, the addresses they send the information to, the differences in synaptic architecture in dendritic trees,” said Koch. “We have highly heterogeneous components,” he said. “Very different from our current VLSI hardware.”

“This really exceeds anything that science has ever studied, with these highly heterogenous components on the order of 1,000 with connectivity on the order of 10,000.” Current deep neural networks are “impoverished,” said Koch, with gain-saturation activation units. “The vast majority of them are feed forward,” said Koch, whereas the brain has “massive feedback.” 

“Understanding brains is going to take us a century or two,” he advised. “It’s mistake to look for inspiration in the mechanistic substrate of the brain to speed up AI,” said Koch, contrasting his views to those of Marblestone. “It’s just radically different from the properties of a manufactured object.” 

Following Koch, Marcus proposed some more questions. He asked about “diversity,” what could individual pieces of the cortex tell us about other parts of the cortex?

Tsao responded by pointing out “what strikes you is the similarity,” which might indicate really deep general principles of brain operation, she said. Predictive coding is a “normative model” that could “explain a lot of things.” “Seeking this general principle is going to be very powerful.” 

Koch jumpe3d on that point, saying that cell types are “quite different,” with visual neurons being very different from, for example, pre-frontal cortex neurons.

Marblestone responded “we need to get better data” to “understand empirically the computational significance that Christof is talking about.”

Marcus asked about innateness. He asked Kahneman about his concept of “object files.” “An index card in your head for each of the things that you’re tracking. Is this an innate architecture?”

Kahneman replied that object files is the idea of permanence in objects as the brain tracks them. He likened it to a police file that changes contents over time as evidence is gathered. It is “surely innate,” he said. 

Marcus posed that there is no equivalent of object files in deep learning. 

Marcus asked Tversky if she is a Kantian, meaning, Emmanuel Kant’s notion that time and space are “innate.” She replied that she has “trouble with innate,” even object files being “innate.” Object permanence, noted Tversky, “takes a while to learn.” The sharp dichotomy between innate and what is acquired made her uncomfortable, she said. She cited Li’s notion of acting in the world, as important.

Celeste Kidd echoed Tversky in “having a problem saying what it means to be innate.” However, she pointed out “we are limited in knowing what an infant knows,” and there is still progress being made to understand what infants know. 

Next was Kidd’s turn to present. Kidd is a professor at UC Berkeley whose lab studies how humans form beliefs. Kidd noted algorithmic bias is dangerous for the ways they can “drive human belief sometimes in destructive ways.” She cited AI systems for content recommendation, such as on social networks. Such systems can push people toward “stronger, inaccurate beliefs that despite our best efforts are very difficult to correct.” She cited the examples of Amazon and LinkedIn using AI for hiring, which had a tendency to bias against female job candidates.

“Biases in AI systems reinforce and strengthen bias in the people that use them,” said Kidd. “Right now is a terrifying time in AI,” said Kidd, citing the termination of Timnit Gebru from Google. “What Timnit experienced at Google is the norm, hearing about it is what’s unusual.” “You should listen to Timnit and countless others about what the environment at Google was like,” said Kidd. “Jeff Dean should be ashamed.” 

Next up was Mitchell, who was co-lead with Gebru at Google. “The typical approach to developing machine learning collects trAining data, training the model, output is filtered, and people can see the output.” But, said Mitchell, human bias is upfront in the collection of data, the annotation, and “throughout the rest of development. It further affects model training.” Things such as loss function put in a “value statement,” said Mitchel.. There are biases in post-processing,.

“People see the output, and it becomes a feedback loop,” you home in on the beliefs, she said. “We were trying to break this system, which we call bias laundering.” There is no such thing as neutrality in development, said Mitchell, “development is value laden.” Among the things that need to be operationalized, she said, is to break down what is packed into development, so that people can reflect on it, and work toward “foreseeable benefits” and away from “foreseeable harms and risks.” Tech, said Mitchell, is usually good at extolling the benefits of tech, but not the risks, nor pondering the long-term effects. 

You wouldn’t think it would be that radical to put foresight into a development process, but I think we’re seeing now just how little value people are placing on the expertise and value of their workers,” said Mitchell. She said she and Gebru continue to strive to bring the element of foresight to more areas of AI.

Next up was Rossi, an IBM fellow. She discussed the task of creating “an ecosystem of trust” for AI. “Of course we want it to be accurate, but beyond that, we want a lot of properties,” including “value alignment,” and fairness, but also explainability. “Explainability is very important especially in the context of machines that work with human beings.”

Rossi echoed Mitchel about the need for “transparency,” as she called it, the ways bias is injected in the pipeline.