My research focuses on the computational challenge
that one faces when trying to understand intelligent behavior in natural
systems, or when one attempts to build artificial intelligence. For
instance, intelligent behavior is adaptive and changes on the basis
of past seen data; it requires integration of sensory data with prior
knowledge; and it must be robust to noise. These problems are fundamental
and they occur in many intelligent tasks (cf. vision, motor control,
memory, etc.). They are shared by natural and artificial intelligence. My
general research goal is to provide theoretical insights, models and
approaches that address these issues and is at the interface of
machine learning and neuro-science.
Approximate inference
Examples of early work are Boltzmann Machines: probabilistic neural
networks with hidden structure that can learn complex pattern recognition
tasks. In order to make learning computationally feasible, efficient
novel approximate inference methods were developed, based on insights
from statistical physics (1998).
This work generalized naturally into Bayesian network research, where
inference is equally intractable and poses a fundamental problem for
large scale application. Since 1998 until present, we have made several
algorithmic contribution, such as the TAP correction, the
linear response correction, various types of bounds, the Cluster Variation
Method, and loop corrections for belief propagation. This work has been applied
by our group to
large scale applications, such as the Promedas medical diagnostic expert system and
more recently to applications in stochastic optimal control.
It is also relevant for computational neuro-science
because brains must solve very similar problems. The insights from
approximate inference methods provide guiding principles that can
constrain model design (for instance, low level vision).
Control theory
Planning of actions and behavior poses another big challenge. Biological
systems as well as AI systems must operate under high uncertainty. This
uncertainty has many sources, for instance because of the unknown behavior
of other animals or because of the noise induced by the limitations
in the sensors. Thus, control of such systems is very different from
an industrial robotic setting, where full knowledge can be assumed and
noise can be almost ignored.
Starting in 2004, we have proposed a novel class of stochastic control problems
using path integrals that can be mapped onto a Bayesian inference
problem. As a result, state-of-the-art inference methods can be applied
to obtain efficient algorithms. Recently, researchers at Computational learning and motor control lab
of the University of Southern California have shown that this approach
significantly outperforms other state-of-the-art reinforcement
methods and is being applied
to various robotic platforms.
The path integral theory makes quantitative predictions about optimal
planning under uncertainty. One such prediction is the phenomenon of
delayed choice: when uncertain about the future, it is wise to delay
a decision. Experiments are currently conducted on humans that move a
noisy cursor to one of two goal locations (with Stan Gielen). Initial
results seem to agree with the model prediction.
This approach to control is currently applied to modeling interaction
between agents. When agents cooperate or play a game, reasoning
about the optimal strategy requires an assumption or model about
the other agent(s) behavior. These probabilistic opponent models
can be incorporated into the path integral control framework. In
this way it can for instance be shown that computation of an
individual agents optimal strategy in the context of a swarm of
other agents becomes a graphical model inference task that can be
efficiently computed using belief propagation. Very recently, it
has been shown how to include an infinite recursion of nested beliefs
(I think, that you think, that I think,...) in this control framework.
This allows the development of agents that plan their future course
of action using a model of the world that includes the intelligent
minds of the other agents.
Computational neuroscience
Another line of research, that was started in 2000 and is still continued,
is on the effect that short-term synaptic plasticity has on memory
storage. The common understanding of long term memory is that it is stored
in the synaptic connections between neurons in such a way that memory
retrieval occurs as the relaxation of the neural activity to a constant
spiking pattern, that represents the memory. This idea was put forward
by Hopfield (1982) and others as the attractor neural network. Synaptic
dynamics challenges this mechanism, since persistent pre-synaptic
activity typically weakens the synaptic strength. The inclusion
of short-term synaptic plasticity in an attractor neural network
make memories metastable states that rapidly switch from one state
to the next, depending on the sensory context. This work provides
some insights on the puzzle how the brain, viewed as a dynamical
system, is able to build stable representations of the world and at
the same time is capable to effortlessly switch between them. With (J. Torres, University of Granada).
Data analysis
Modern experimental neuro-science has been revolutionarized by
sophisticated measurement equipment, such as fMRI, MEG and others. Also,
the advances in EEG measurement systems has accelerated the research
on Brain Computer Interfaces. Thirdly, research tools in genetics have
led to an explosion of DNA and expression data. These massive data sets require
advanced data analysis tools. Machine learning methods (kernel methods,
sparse dimension reduction methods, ICA, Bayesian approaches) provide
the most promising approach to analyze these data.
In 2008, we have started a collaboration with Human Genetics on the
genetic origin of psychiatric disorders. We analyze data from genome-wide
association studies using a Bayesian approach (L. Janss with Prof. J. Buitelaar
and dr. Barbara Franke). The project is funded
by STW. Since this year this initiative has been extended with
funding from the Donders Institute.
Another project is on the analysis of ECoG
data from epileptic patients, that perform working memory tasks. The
objective is to extract significant space-time structure from these data
and correlate it with the tasks (V. Gomez with Nick Ramsey, UMC Utrecht).
Brain Computer Interface
Since 2009, I have started research on
the design of an adaptive BCI system, based on the idea that subjects will
be surprised when the BCI output differs from their expectation. This
surprise is measurable as a so-called error potential. The detection of the
error potential can be used to adapt the BCI device, using the perceptron
learning rule (V. Gomez and A. Llera with O. Jensen, Donders Neuro-imaging Center).
Bayesian methods have a big potential for
immediate application in areas outside science. There is a long-standing and quite
unique tradition
in the SNN group to build such application together with her spin-off companies Smart Research and Promedas.
Here are a few examples:
Genetic inference
We have applied an advanced approximate
inference method (the Cluster Variation Method) to construct haplotypes in complex pedigrees.
The
method was shown to outperform the state-of-the-art Monte Carlo sampling
approach on a subset of problems. The software is
publicly available. Contact Kees Albers for details caa at sanger dot ac dot uk.
Aladin is a software tool for performing efficient linkage analysis of a small number of distantly-related individuals. It estimates multipoint IBD probabilities and parametric LOD scores. Contact Kees Albers for details caa at sanger dot ac dot uk.
Oil exploration
For Shell, we built a petrophysical expert system.
It estimates the type of soil and the probability that it contains oil, gas
or other valuable minerals, based on drilling measurements. The system is
based on a Bayesian network where the probability computation is done using
a Monte Carlo sampling method. See Smart Research for further details and other products.
Victim identification
For the Netherlands Forensic Institute, we are
building a victim identification system by matching of
their DNA profiles against the Pedigrees of Relatives from Missing Person's
DNA profiles in large databases, using a Bayesian network. See
Bonaparte for further details.
Promedas
We have built the world largest and most up-to-date medical
expert system for diagnostic advice in internal medicine. The system is
being commercialized by Promedas bv. The system is since end 2008 operational
at the Utrecht academic hospital. See
Promedas for further details.
Wine and food
We have built a system that selects the most appropriate wines
to combine with your food Wine wine wine.
Wim Wiegerinck is
senior researcher and is working on approximate inference, genetic
inference and various applications and is associate director of Smart Research bv
Vicenç
Gómez
is postdoc. He is working on ECoG data analysis, approximate
inference, neural networks and internet communities and CompLACS (EU
FP7) project.
Kevin Sharp is
a postdoc on a project to develop novel
methodes for GWAS
Joris Mooij was PhD
student on approximate inference.
Kees Albers was PhD student and postdoc on approximate
inference methods for genetic linkage analysis. Kees was 4 years at Sanger
Institute, Cambridge UK and is since 2012 at Human Genetics in Nijmegen.
Bram Kasteel was Bachelor student on the topic of multi-agent control
Stijn Tonk was Master student on the topic of multi-agent control
Ender Akay was programmer for Smart Research bv and Promedas bv
Gulliver de Boer was Bachelor student on the topic of multi-agent control applied to poker
Max Bakker was a Bachelor student on the topic of multi-agent systems
Ben Ruijl was a Bachelor student on the topic of multi-agent systems
Henk Griffioen was Master student on the topic of genetic association studies
Bart van den broek was PhD student on the topic of stochastic optimal control theory
Gheshlaghi Azar M., Munos R., Ghavamzadaeh M., Kappen H.J.Speedy q-learning: a computationally efficient reinforcement learning algorithm with a near optimal rate of convergenceJournal of Machine Learning Research,2013bibtex
Gheshlaghi Azar M., Munos R., Kappen H.J.Minimax pac bounds on the sample complexity of reinforcement
learning with a generative modelMachine Learning Journal,2013bibtex
Bierkens J., Chertkov, M, Kappen H.J.Linear pdes and eigenvalue problems corresponding to ergodic stochastic optimization problems on compact manifolds pp. 1-16,2013bibtex
Wiegerinck W.A.J.J., Burgers W.G., Kappen H.J.Bayesian networks, introduction and practical applicationsHandbook on Neural Information Processing, vol. 49, pp. 401-431,2013bibtex
Tramper J.J., Broek J.L. van den, Wiegerinck W.A.J.J., Kappen H.J., Gielen C.C.A.M.Stochastic optimal control predicts human motor behavior in time-constrained
sensorimotor tasksBiological Cybernetics, pp. xx,2011bibtex
Gheshlaghi Azar M., Munos R., Ghavamzadaeh M., Kappen H.J.Speedy q-learningNIPS 2011, Advances in Neural Information Processing Systems 24, vol. 25, pp. 2411--2419,2011bibtex
Broek J.L. van den, Wiegerinck W.A.J.J., Kappen H.J.Stochastic optimal control of state constrained systemsInternational Journal of Control, vol. 84, no. 3, pp. 597-615,2011bibtex
Gheshlaghi Azar M., Kappen H.J.Dynamic policy programming with kl-divergence minimizationNIPS Workshop on Probabilistic Approaches for Stochastic Optimal Control and Robotics,2009bibtex
Albers C.A., Kappen H.J.Modeling linkage disequilibrium in exact linkageBMC Proceedings of the genetic Analysis Workshop 15: Gene Expression Analysis and Approaches to Detecting Multiple Functional, vol. 1, pp. S159,2007bibtex
Torres J.J., Marro J., Cortes J.M., Kappen H.J.Attractor neural networks with activity-dependent synapses: the role
of synaptic facilitation.Neurocomputing, vol. 70, no. 10-12, pp. 2022-2025,2007bibtex
Wemmenhove B., Mooij J.M., Wiegerinck W.A.J.J., Leisink M.A.R., Kappen H.J., Neijt J.P.Inference in the promedas medical expert systemProceedings of the 11th Conference on Artificial Intelligence in
Medicine (AIME 07), vol. 4594, pp. 456-460,2007bibtex
Leisink M.A.R., Kappen H.J.Means, correlations and boundsIn: Advances in Neural Information Processing Systems, 14, vol. 14-1, pp. 455-462,2002bibtex
Cemgil A.T., Kappen H.J.Tempo tracking and rhythm quantization by sequential monte carloAdvances in Neural Information Processing Systems 14, Part VIII Applications, vol. 14-2, pp. 1361-1368,2002bibtex
Kappen H.J.A novel iteration scheme for the cluster variation methodNeural Information Processing Systems, vol. 13, pp. 415-422,2001bibtex
Kappen H.J., Gielen C.C.A.M., Wiegerinck W.A.J.J., Cemgil A.T., Nijman M.J.Approximate reasoning: real world applications of graphical modelsFoundations of Real-World Intelligence, pp. 73-121,2001bibtex
Cemgil A.T., Kappen H.J.A dynamic belief network implementation for realtime music transcriptionProceedings of the 13th Belgian-Dutch Conference on Artificial Intelligence, pp. 473-474,2001bibtex
Kappen H.J., Wiegerinck W.A.J.J.Stochastic dynamics with dominant self-couplingPresentation "Learning",,2000bibtex
Heskes T.M., Kappen H.J.An application of linear response learningProceedings of the 12th Belgium-Netherlands Artificial Intelligence Conference, pp. 117-124,2000bibtex
Kappen H.J., Nijman M.J., Moorsel T. vanLearning active visionIndustrial Applications of Neural Networks, pp. 193-202,1998bibtex
Kappen H.J., Groothof M.Just enough deliveryINMA Ideas Magazine, pp. 23,1998bibtex
1997
Nijman M.J., Kappen H.J.Symmetry breaking and training from incomplete data with radial basis boltzmann machinesInternational Journal of Neural Systems A, vol. 8, pp. 301-316,1997bibtex
Nijman M.J., Kappen H.J.Efficient learning in sparsely connected boltzmann machinesProceedings RWC'97, pp. 406-409,1997bibtex
Wiegerinck W.A.J.J., Burg W.J.P.P. ter, Dam P.S. van, Ying-Lie O., Neijt J.P., Kappen H.J.An advisory system for anaemia based on boltzmann machines5th Eurpean Congres on Intelligent Techniques and Soft Computing, vol. 1, pp. 364-368,1997bibtex
Kappen H.J.A polynomial time algorithm for boltzmann machine learningWorkshop Cambridge,1997bibtex
Kappen H.J., Mempen, Dijken A. van, Otten H.A.F.M.Voorspelling van frisdrankverkoop1997bibtex
Kappen H.J., Gielen C.C.A.M.Neural networks: best practice in europeNeural Networks: Best Practice in Europe, pp. 209,1997bibtex
Kappen H.J., Rodriquez F.B.Accelerated learning in boltzmann machines using mean field theoryArtificial Neural Networks 7, pp. 301-306,1997bibtex
Wiegerinck W.A.J.J., Burg W.J.P.P. ter, Dam P.S. van, Ying-Lie O., Neijt J.P., Kappen H.J.Lab-test selection in diagnosis of anaemiaNeural Networks: Best Practice in Europe, pp. 179-181,1997bibtex
Wiegerinck W.A.J.J., Kappen H.J.Practical confidence and prediction intervals for prediction tasksNeural Networks: Best Practice in Europe, pp. 128-135,1997bibtex
Kappen H.J., Wiegerinck W.A.J.J., Morgan T., Harris T.J., Paillet G., Kopecz K.Stimulation initiative for european neural applications (siena)Neural Networks: Best Practice in Europe, pp. 1-8,1997bibtex
Kappen H.J., Pastoors A.G.W., Gielen C.C.A.M.Confidence intervals for neural networksProceedings of the International Conference on Digital Signal Processing, vol. 1, pp. 396-401,1995bibtex
Heskes T.M., Kappen H.J.Self-organization and nonparametric regressionArtificial Neural Networks 5, vol. 1, pp. 81-86,1995bibtex
Nijman M.J., Kappen H.J.Radial basis boltzmann machines and incomplete data1995bibtex
Heskes T.M., Kappen H.J.On-line learning processes in artificial neural networks vol. 51, pp. 199-234,1993bibtex
Kappen H.J., Neijt J.P.Neural network analysis to predict treatment outcomeAnnuals of Oncology, vol. 4, pp. 31-34,1993bibtex
Kappen H.J.Using boltzmann machines as perceptronsIEEE Trans. Neural Networks,1993bibtex
Wiegerinck W.A.J.J., Kappen H.J.Optimizing the architecture of multi-layer
perceptrons for one-dimensional classificationArtificial Neural Networks 3, pp. 558-561,1993bibtex
Kappen H.J.Using boltzmann machines for probability estimationArtificial Neural Networks 3, pp. 521-526,1993bibtex
Hal R. van, Kappen H.J., Neijt J.P.Neural network analysis for prediction of treatment outcome in ovarian cancerEWOC-3,1993bibtex
Opstal A.J. van, Kappen H.J.Neural representation of saccadic eye movements in monkey superior colliculusArtificial Neural Networks 3, pp. 88-93,1993bibtex
Heskes T.M., Kappen H.J.Learning processes in neural networks1993bibtex
Kappen H.J.Error potentials for self-organizationInternational Conference on Neural Networks, San Francisco, vol. 3, pp. 1219-1223,1993bibtex
Slijpen E.T.P., Kappen H.J.Cooling schedules for learning in neural networksPhysical Review E, vol. 47, pp. 4457-4464,1993bibtex
Kappen H.J.Neurale netwerken, fuzzy rules en artificiele intelligentieKlinische Fysica, vol. 1, pp. 13-16,1993bibtex
Opstal A.J. van, Kappen H.J.A two-dimensional model for spatial-temporal transformation of saccades in monkey superior colliculusNetwork, vol. 4, pp. 19-38,1993bibtex
Gielen C.C.A.M., Kappen H.J.Proceedings of the international confidence icann'93Artificial Neural Networks 3,1993bibtex
1992
Slijpen E.T.P., Kappen H.J.Learning in neural networks with local minimaPhysical Review A, vol. 46, pp. 5221-5231,1992bibtex
Kappen H.J.Learning parameter adjustment in neural networksPhysical Review A, vol. 45, pp. 8885-8893,1992bibtex
Kappen H.J.Learning rules, stochastic processes, and local minimaArtificial Neural Networks 2, vol. 1, pp. 71-78,1992bibtex
Kappen H.J., Neijt J.P.Neural network analysis for prediction of treatment
outcome in ovarian cancerASCO,1992bibtex
Kappen H.J., Gielen C.C.A.M.Proceedings symposium on neural networksProceedings of 2nd SNN Conference on Neural Networks, vol. 2,1992bibtex
1991
Heskes T.M., Kappen H.J.Learning processes in neural networksPhysical Review A, vol. 44, pp. 2718-2726,1991bibtex
Kappen H.J., Gielen C.C.A.M.Neural networks learning in a changing environmentArtificial Neural Networks 1, vol. 1, pp. 15-20,1991bibtex
Kappen H.J.Een computersimulatie van hetbilocale correlator model1991bibtex
Opstal A.J. van, Kappen H.J.Quantitative model for spatio-temporal transformation of oculomotor signals in monkey superior colliculus.Eur. J. Neurosci. (Suppl), vol. 14, pp. 56,1991bibtex
Kappen H.J.Neural networks learning in achanging environmentInternational Joint Conference on Neural Networks, Seattle, vol. 1, pp. 823-828,1991bibtex
On the sample complexity of reinforcement learning with a generative model
Proceedings of the International Conference on Machine Learning
Learning,
vol. 29 th,
pp. 1-11,
2013
bibtex