Further Information

wordcloud

About

The contents of this website are based on the PhD thesis entitled Regression Modelling with Priors using Fisher Information Covariance Kernels (I-priors) by Haziq Jamil. It is intended as an accompaniment to the research poster of the same title.

This project was supervised by Wicher Bergsma and Irini Moustaki.

Author

Haziq Jamil is a research student in Statistics at the London School of Economics and Political Science. His interests are in statistical methodology and computation, with an emphasis on applications towards the social sciences.

Haziq graduated with first class honours from The University of Warwick in 2010, where he read Mathematics, Operational Research, Statistics and Economics. He completed an MSc in Statistics with distinction at the London School of Economics and Political Science in 2014. He is expected to complete his PhD from the same university in 2018.

Previously, Haziq worked as a research officer at the Centre of Science and Technology, Research and Development at the Ministry of Defence, Brunei, where he provided scientific decision support for strategic acquisition projects.

  • Source - GitHub
  • PhD Poster - PDF
  • R/iprior package - CRAN, GitHub
  • R/iprobit package - GitHub
  • R/ipriorBVS package - GitHub
  • HMC explainer - Shiny Apps 1, 2
  • Example of variational inference - link

References

Introduction

  • Bergsma, W. (2017, July). Regression and classification with I-priors. Manuscript in preparation. arXiv: 1707.00274.
  • Berlinet, A., & Thomas-Agnan, C. (2011). Reproducing Kernel Hilbert Spaces in Probability and Statistics. Springer Science & Business Media. DOI: 10.1007/978-1-4419-9096-9.
  • Neal, R. M. (2011). MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo2(11). Chapman & Hall/CRC Press. arXiv: 1206.1901v1.
  • Ong, C. S., Mary, X., Canu, S., & Smola, A. J. (2004, July). Learning with non-positive kernels. In Proceedings of the Twenty-first International Conference on Machine Learning (p. 81). ACM. DOI: 10.1145/1015330.1015443 .
  • Steinwart, I., & Christmann, A. (2008). Support Vector Machines. Springer. 10.1007/978-0-387-77242-4

Regression

  • Rasmussen, C. E., & Williams, C. K. (2006). Gaussian Processes for Machine Learning. The MIT press. ISBN: 0-262-18253-X.
  • Williams, C. K., & Seeger, M. (2001). Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems 13, 682-688. MIT Press. URL.

Classification

  • Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer. ISBN: 978-0-387-31073-2.
  • Blei, D. M., Kucukelbir, A., & McAuliffe, J. D. (2017). Variational Inference: A Review for Statisticians. Journal of the American Statistical Association, to appear. arXiv: 1601.00670.
  • Girolami, M., & Rogers, S. (2006). Variational Bayesian Multinomial Probit Regression with Gaussian Process Priors. Neural Computation18(8), 1790-1817. DOI: 10.1162/neco.2006.18.8.1790.

Variable Selection

  • Kuo, L., & Mallick, B. (1998). Variable selection for regression models. Sankhyā: The Indian Journal of Statistics, Series B, 65-81. URL.
  • O’Hara, R. B., & Sillanpää, M. J. (2009). A review of Bayesian variable selection methods: what, how and which. Bayesian analysis4(1), 85-117. DOI: 10.1214/09-BA403.

Examples

  • Cannings, T. I., & Samworth, R. J. (2017). Random‐projection ensemble classification. Journal of the Royal Statistical Society: Series B (Statistical Methodology)79(4), 959-1035. DOI: 10.1111/rssb.12228.
  • Diggle, P., Zheng, P., & Durr, P. (2005). Nonparametric estimation of spatial segregation in a multivariate point process: bovine tuberculosis in Cornwall, UK. Journal of the Royal Statistical Society: Series C (Applied Statistics)54(3), 645-658. DOI: 10.1111/j.1467-9876.2005.05373.x
  • Goldstein, H., Rasbash, J., Yang, M., Woodhouse, G., Pan, H., Nuttall, D., & Thomas, S. (1993). A Multilevel Analysis of School Examination Results. Oxford Review of Education19(4), 425-433. DOI: 10.1080/0305498930190401.
  • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning. Springer. DOI: 10.1007/978-0-387-84858-7.
  • Kenward, M. G. (1987). A Method for Comparing Profiles of Repeated Measurements. Journal of the Royal Statistical Society: Series C (Applied Statistics), 36(3), 296-308. DOI: 10.2307/2347788.
  • Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models. Chapman & Hall/CRC. ISBN: 978-1-58488-000-4.

Copyright © 2014-2017 by Haziq Jamil. All rights reserved. The contents of this website or any portion thereof may not be reproduced or used in any manner whatsoever without the express written permission of the author.

Updated: