The Behavioral Data Science Podcast
Conversations about behavioral data science, AI, and machine learning. We keep it rigorous, useful, and fun.
Latest Podcasts
Episode 036: Interview with Dr. Cody Morris & Dr. Bryant Silbaugh
In this episode, we have the distinct honor of being joined by Dr. Cody Morris and Dr. Bryant Silbaugh to discuss their perspective on quality measurement and client outcomes in Applied Behavior Analysis.
Episode 035: Understanding Variability in Client Profiles
This is the third of the three-part introduction to thisseason of the Behavioral Data Science Podcast. During this season, Jake and I discuss the intersection of behavior science and data science as they relate to client outcomes in Applied Behavior Analysis service delivery. In this episode, we discuss the importance and challenges of contextualizing outcome measures based on the unique clinical presentation of each client.
Episode 034: Outcomes in ABA and What Data Science Can Contribute
This is the second of the three-part introduction to this season of the Behavioral Data Science Podcast. During this season, Jake and I discuss the intersection of behavior science and data science as they relate to client outcomes in Applied Behavior Analysis service delivery. In this episode, we discuss the history of outcome measurement in Applied Behavior Analysis, broadly, and the role the data science can play moving forward.
Episode 033: The History of Quality Measurement
This is the first of the three-part introduction to this season of the Behavioral Data Science Podcast. During this season, Jake and I discuss the intersection of behavior science and data science as they relate to client outcomes in Applied Behavior Analysis service delivery. In this episode, we discuss the history of quality measurement in healthcare, broadly.
Episode 031: Reviewing and Extending the Recent Issue of JABA
In this episode, Jake and David discuss articles from the recent issue of the Journal of Applied Behavior Analysis. They also discuss how the underlying technologies and concepts might be extended via data science techniques.
Episode 030: Reviewing and Extending the Recent Issue of AI in Medicine
In this episode, Jake and David discuss articles from the recent issue of Artificial Intelligence in Medicine. They also discuss how the underlying technologies and concepts might be translated to behavioral health services.
Episode 029: ChatGPT Health & Claude Healthcare
OpenAI and Anthropic have recently released products where users can connect health-related data and receive healthcare advice and guidance. Jake and David discuss these products, the potential benefits to ease of access to personalized healthcare-related information, and the potential harms from inaccurate (or dangerous) recommendations and data privacy concerns. At the end of the episode, they each answer the question we all have to ask ourselves: "Will you be hooking your data up to these systems?"
Episode 028: Building the Dataset: From Chaos to Order
Episode 028: Building the Dataset: From Chaos to Order Realistically, you can't build any model of behavior-environment relations if you can't (a) find the data you need and (b) integrate those data into a usable database. In this episode of The Behavioral Data Science Podcast, we discuss the many considerations and decisions one needs to make. And, we do so by discussing a seven-year-long project Jake has been working on to build a usable database of all open-source articles published within five behavior-analytic journals.
Episode 027: Operationalizing Behavior in the Wild
Crucial to any behavioral data science project is identifying either (a) what behavior you want to analyze and how you'll get the data; or (b) what data you can get and what behaviors those data allow you to analyze well. In this episode, we chat about these decisions in the context of the literally wild behavior of birds at backyard feeders. For the interested, here's a link to the backyard ecology dashboard referenced during the episode: https://david-j-cox.github.io/backyard-ecology/
Episode 026: From Hot Take to Testable Question
In each episode of Season 3, we take a claim from a news story, paper, or hot topic, and walk through how a behavioral data scientist would think about it: clarify the question, identify the operant and respondent principles potentially at play, design the data pipelines, choose the models, and turn the resulting insights into data-based behavior-change tools. In this episode, we take on the claim that "late-night screen time disrupts sleep and leads teens to be more depressed".
Episode 025: Reflections on LLMs and AI with Dr. Garrison
As we close out Season 2 and our emphasis on LLMs, we had the distinct privilege of chatting with Dr. Elizabeth Garrison. She is one of the few people in the world with domain expertise spanning behavior analysis (BCBA) and artificial intelligence (PhD). In this episode, we reflect on the state of AI research and industry work pre-ChatGPT and post-ChatGPT release, the shift in academic AI research when the transformer architecture became broadly available, and the differences between academia and industry in both behavior science and AI.
Episode 024: Are we in an AI bubble?
"Bubbles" are an economic phenomenon characterized by a rapid increase in asset prices that far exceed the asset's underlying fundamental value, driven by speculative buying and herd behavior rather than intrinsic worth. In this episode, Jake and David ask, "Are we in an AI bubble?". And, if so, what might this mean for both individuals and organizations as they navigate the current AI strategic landscape?
Episode 023: Your Brain on LLMs
In this episode, Jake and David discuss the burgeoning area of research looking at how interacting with LLMs impacts our skills and abilities in good and bad ways. As with most things in life, the effects are not black-and-white. And, we discuss strategies and tactics we can all engage in to try to get the benefits without the drawbacks.
Episode 022: The Ethics of LLMs that Few Talk About
Conversations around AI ethics often focus on a suite of incredibly important topics such as data security and privacy, model bias, model transparency, and explainability. However, each time we use large AI models (e.g., diffusion models, LLMs), we reinforce a host of additional potentially unethical practices that are needed to build and maintain these systems. In this episode, Jake and David discuss some of these unsavory topics, such as human labor costs and environmental impact. Although it's a bit of a downer, it's crucial for each of us to acknowledge how our behavior impacts the larger ecosystem and recognize our role in perpetuating these practices.
Episode 021: Explainable AI and LLMs
"Explainable AI", aka XAI, refers to a suite of techniques to help AI system developers and AI system users understand why inputs to the system resulted in the observed outputs. Industries such as healthcare, education, and finance require that any system using mathematical models or algorithms to influence the lives of others is transparent and explainable. In this episode, Jake and David review what XAI is, classical techniques in XAI, and the burgeoning area of XAI techniques specific to LLM-driven systems.
Episode 020: Evidence-Based Practices for Prompt Engineering
Prompt engineering involves a lot more than simply getting smarter with how you structure the prompts you enter in an LLM browser interface. Furthermore, a growing body of peer-reviewed research provides us with best practices to improve the accuracy and reliability of LLM outputs for the specific tasks we build systems around. In this episode, Jake and David review evidence-based best practices for prompt engineering and, importantly, highlight what proper prompt engineering requires such that most of us likely cannot call ourselves prompt engineers.
Episode 019: LLM Evaluation Frameworks
Lots of people like to talk about the importance of prompts, context, and what is sent to an LLM. Few discuss the even more important aspect of an LLM-driven system in evaluating its output. In this episode, we discuss traditional and modern metrics used to evaluate LLM outputs. And, we review the common frameworks for obtaining that feedback. Though evals are a lot of work (and easy to do poorly), those building (or buying) LLM-driven systems should be transparent about their process and the current state of their eval framework.
Episode 015: Welcome to the Era of Experience
Jake and I chat about a forthcoming book chapter titled, "Welcome to the Era of Experience" by David Silver and Richard Sutton (link below). This—naturally—led other topics to surface, such as companies staffed entirely by AI agents (which turned out as well as that sounds); superintelligence (we might be legally required to reference this during the 2025 AI hype cycle); and how practical systems built on these ideas would even be architected (we both came in with different ideas here which was fun). Happy listening. Links to things mentioned: Smith, D., & Sutton, R. S. (April 26, 2025). Welcome to the Era of Experience. Preprint of a chapter to appear in Designing an Intelligence. MIT Press. Sutton, R. S., & Barto, A. G. (2015). Reinforcement Learning: An Introduction. The MIT Press. Wilkins, J. (April 27, 2025). Professors Staffed a Fake Company Entirely With AI Agents, and You'll Never Guess What Happened. Who would have thought? [Friendly write-up of the agentic company work] Xu, F. F., Song, Y., Li, B., Tang, Y., Jain, K., Bao, M., ..., & Neubig, G. (2024). TheAgentCompany: Benchmarking LLM agents on consequential real world tasks. arXiv:2412.14161. [Actual study.]
Episode 014: Conversation with Dr. Beth Garrison
In this episode, we chat with Dr. Beth Garrison about her journey in behavior analysis, what led her to pursue a PhD in artificial intelligence, and her thoughts on where this is all headed. Links to the papers Dr. Garrison references: Exploring Engagement Opportunities for Autistic Children: Using AAC as a Controller in a Wizard-of-Oz Coloring Game: https://doi.org/10.1145/3701193 Understanding the experience of neurodivergent workers in image and text data annotation: https://doi.org/10.1016/j.chbr.2023.100318
Episode 007: What does it take to go end-to-end with an AI application? Part I - The Data Lifecycle
In this episode, we talk about the many components of data engineering and parallel work data scientists get into as data moves from its data collection source to being ready for modeling.
Episode 006: Bridging the Research-to-Practice Gap in BDS
In this episode, we discuss common barriers and solutions for bridging the research-to-practice gap in behavioral data science. We also talk about many of the ways that data science or AI research differs from behavior science research in terms of practitioners' ability to integrate findings quickly into practice.
Episode 002: What the hell is behavioral data science, anyway?
In this episode, Jake and David discuss how one might even define what it means for something to exist at the intersection of behavior science and data science. We also talk about the different roles on data science teams and applied behavior analysis teams. And we talk about how long it might be until AI systems are better at the functional analysis of behavior than humans.