Transfer Learning using Kolmogorov Complexity: Basic Theory and Empirical Evaluations. NIPS Spotlight ID Number: W6
1. What is Transfer Learning (TL) ? Many Related Classification/Regression Tasks:
Task 1 Task 2 Task 3

M. M. Hassan Mahmud and Sylvian Ray Department of Computer Science University of Illinois At Urbana Champaign

3. Our TL Distance for Tasks: Length of Shortest Conversion Program e.g.: Hypothesis/Task space Decision Trees (DT) Amount of information DT 1 contains about DT 2 = Length of the shortest Conversion Program 1

By Transferring Information between tasks we can Learn each task using fewer examples! 2. We Address a Main Question in TL How do we measure TL Distance/amount of information tasks contain about each other ?
Leads To

  

DT 1

Output

DT 2

Input

4. In Our Work This TL distance is optimal in a formal sense.  Induces formally optimal Bayesian TL prior.  We applied approximation of the above to transfer across arbitrarily chosen databases in UCI ML repository.

How much information do we transfer ? When do we transfer Information ? What does it even mean to transfer information ?