Pascal Germain, Researcher in Machine Learning

This page is deprecated! My new webage is here.


In machine learning, Domain Adaptation (DA) arises when the distribution generating the test (target) 
data differs from the one generating the learning (source) data. It is well known that DA is an hard 
task even under strong assumptions, among which the covariate-shift where the source and target 
distributions diverge only in their marginals, i.e. they have the same labeling function. Another 
popular approach is to consider an hypothesis class that moves closer the two distributions while 
implying a low-error for both tasks. This is a VC-dim approach that restricts the complexity of an 
hypothesis class in order to get good generalization. Instead, we propose a PAC-Bayesian approach that 
seeks for suitable weights to be given to each hypothesis in order to build a majority vote. We prove 
a new DA bound in the PAC-Bayesian context. This leads us to design the first DA-PAC-Bayesian algorithm 
based on the minimization of the proposed bound. Doing so, we seek for a ρ-weighted majority vote that 
takes into account a trade-off between three quantities. The first two quantities being, as usual 
in the PAC-Bayesian approach, (a) the complexity of the majority vote (measured by a Kullback-Leibler 
divergence) and (b) its empirical risk (measured by the ρ-average errors on the source sample). The 
third quantity is (c) the capacity of the majority vote to distinguish some structural difference 
between the source and target samples. 

[ Link to text file ]

<< Go back.