The Regression Cookbook (in development)
Python and R, under two perspectives: inference and prediction.
Preface
Let the regression cooking begin!
Data science is a field in which we become aware of the fascinating overlap between machine learning and statistics. Many data science students usually come across everyday machine learning and statistics concepts or ideas that might only differ in names. For instance, simple terms such as weights in supervised learning (and their statistical counterpart as regression coefficients) might be misleading for students starting their data science formation. On the other hand, from an instructor’s perspective in a data science program that subsets its courses in machine learning in Python and statistics in R, regression courses in R also demand the inclusion of Python-related packages as alternative tools. Furthermore, in a graduate program such as the Master of Data Science (MDS) at the University of British Columbia (UBC), this is especially critical for students whose career plan leans towards the industry job market where Python is more heavily used.
That said, we can state that data science is a substantial synergy between machine learning and statistics. Nevertheless, many gaps between both disciplines still need to be addressed. Thus, closing these critical gaps is imperative in a domain with accelerated growth, such as data science. In this regard, the MDS Stat-ML dictionary has inspired us to write this textbook. It basically consists of common ground between foundational supervised learning models from machine learning and regression models commonly used in statistics. We strive to explore linear modelling approaches as a primary step while explaining different terminology found in both fields. Furthermore, this discussion is more comprehensive than a simple conceptual exploration. Hence, the second step is hands-on practice via the corresponding Python packages for machine learning and R for statistics.
Fun fact!
While thinking about possible names for this work, we were planning to name it “Machine Learning and Statistics: A Common Ground.” Nevertheless, it was quite plain and boring! That said, this whole textbook idea sounded analogous to a cookbook1, given its heavily applied focus with theoretical sparks.
Hence, the cookbook name idea!
Statement on Generative AI Use
Generative artificial intelligence (AI) tools were used selectively during the development of this textbook as part of an iterative, human-led writing and editing workflow.
Chapters typically move through multiple stages of human development and review, including drafting and content development by collaborators, substantive review by co-authors, technical verification, and subsequent editorial revision. The book’s organization, learning objectives, statistical explanations, pedagogical decisions, examples, exercises, and overall direction are determined by the human project team.
Within this workflow, generative AI tools may be used to support tasks such as brainstorming alternative explanations, refining wording and organization, assisting with code development or troubleshooting, and providing editorial suggestions. AI-generated material is not incorporated automatically. Any such suggestions are reviewed, revised, and, where appropriate, independently verified by members of the project team before being included in the book.
Particular care is taken with mathematical results, statistical claims, methodological explanations, code, and references. Citations and bibliographic information are checked against the original source whenever possible, and the project prioritizes reputable scholarly, institutional, and other authoritative sources appropriate to the material being discussed. References suggested by generative AI tools are not treated as reliable without independent verification.
Therefore, Generative AI tools are used as supporting tools within the writing and editing process rather than as autonomous authors. Intellectual, pedagogical, and editorial responsibility for the book remains with its human co-authors and collaborators, who retain responsibility for the accuracy, interpretation, and presentation of the material.
Acknowledgments
This textbook is shared as an Open Educational Resource (OER) using Quarto and is hosted on GitHub. One of its core goals is to motivate global academic communities to access, reuse, and customize its content to meet their specific needs. In the field of data science education, where openness, reproducibility, and collaboration are key, OER materials serve as a powerful means of enhancing learning and promoting equity. By making this statistical resource freely available and modifiable, we support a flexible, student-centered approach that reflects the open-source spirit of the field. Learners can engage with real code, adapt examples to their interests, and contribute back to the community.
The development of this resource is funded by the UBC Vancouver (UBCV) OER Fund, which was launched in 2019 through the UBC Academic Excellence Fund. This initiative aims to support more affordable and inclusive education by promoting the creation and integration of open resources in UBCV credit courses. With this support, we are able to design a resource that reduces textbook costs while enhancing pedagogical freedom and adaptability. Both instructors and students benefit from materials that can evolve alongside curriculum changes, disciplinary innovations, and learner diversity. This cookbook is part of a growing movement toward open, collaborative, and responsive teaching in higher education.
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Special thanks to Jonathan Graves, who mentioned the cookbook term when this textbook was conceptualized during very early stages.↩︎