Fields of learning theory, machine learning grow together at University

    Fields of learning theory, machine learning grow together at University

    By Steve Koppes
    News Office

    Approximately 100 students from across the country and around the world completed a two-week crash course in machine learning at the end of May, at the University’s International House.

    Machine learning is a branch of computer science that lends itself to applications as wide ranging as detecting credit card fraud, filtering unwanted e-mail and processing genetic data.

    The students attended the Machine Learning Summer School, which was held in North America for the first time since researchers in Europe and Australia launched the program in 2002.

    The summer school was a collaborative effort between the Toyota Technological Institute at Chicago and the University’s Computer Science Department. A branch of Japan’s Toyota Technological Institute, TTI-Chicago specializes in graduate instruction and basic research in computer science.

    “It’s unique, at least inside the United States, as far as getting experts from a lot of different areas to come to the same place and talk to the same students,” said summer school organizer John Langford, a Research Assistant Professor at TTI-Chicago.

    The Machine Learning Summer School coincided with two weeklong workshops at TTI-Chicago that brought in approximately 50 experts to exchange ideas with local specialists in the related field of learning theory, especially as it pertains to the evolution of language. Both machine learning and learning theory share many common mathematical foundations, said Partha Niyogi, Professor in Computer Science, Statistics and the Physical Sciences Collegiate Division.

    Both the Machine Learning Summer School and the workshops in learning theory culminate the three-month Program in Learning Theory and Related Areas, which researchers at TTI-Chicago and in the Computer Science Department organize.

    Machine learning is a growing segment of artificial intelligence that involves teaching a computer to learn from experience to perform tasks that a human could not do or that a human could do, but a machine could do at a much lower cost. A typical example from the business world would be teaching a computer to detect credit card fraud.

    “It’s very difficult to detect fraud as it happens because you’re seeing millions of transactions per day. You can’t really have a human sort through them very well,” Langford said. But a computer can be rapidly taught to detect fraud using far more examples than a human could ever become familiar with in a lifetime.

    The Program in Learning Theory and Related Areas, which began in March, grew out of one of the first major collaborations between the University’s Computer Science Department and TTI-Chicago. Even before TTI-Chicago formally opened its offices in the University Press building in September 2003, Niyogi and Stephen Smale, a Professor at TTI-Chicago, received a $2.2 million National Science Foundation grant to study learning theory. They organized the special program and workshops on learning theory under the auspices of that grant.

    Like machine learning, learning theory stands at the crossroads of multiple disciplines. The one-week learning theory workshop, which ended Friday, May 20, brought together statisticians, mathematicians and computer scientists to discuss mathematical aspects of learning theory. A central focus was on the relation between statistical learning and a subfield of applied mathematics and computer science called function approximation.

    The learning theory workshop that ended Friday, May 27, explored the evolutionary dynamics of learning, especially as it applies to language and language evolution. “The idea was to explore the ways in which learning and evolution manifest themselves in language, culture and biology,” Niyogi said.

    Learning theorists are attempting to understand the basic principles that describe how people or computers infer new knowledge from the facts at their disposal.

    Evolutionary theorists have produced mathematical systems that show how gene sequences of organisms change from generation to generation and how biological diversity arises according to the laws of genetics. Learning theorists, meanwhile, look for the mathematical properties of evolutionary processes, where the mode of transmission is learning rather than inheritance.

    “A classic example of that is language evolution,” Niyogi said. “Languages are transmitted from parents to children, much like genes are transmitted, except that in the process of language you don’t really inherit the language of your parents in the way you inherit their genes. You actually have to learn their language not just from data provided by them, but also data provided by many other language users in the community in which you are immersed.”

    The fields of learning theory and machine learning have different roots, but the Chicago events exemplify some of their growing common interests. “The communities are mixing to a greater and greater extent,” Langford said.