Falling in Love: Do the 36 Questions Actually Work?

[Kristin] (0:00 - 0:05)
And really, Regina, all of this was just a ploy for her to get a date with the hot man at the bar, really.

[Regina] (0:06 - 0:09)
Well, it worked.

[Kristin]
It did work.

[Regina]
Whatever it is, it worked for her.

[Kristin] (0:14 - 0:37)
Welcome to Normal Curves. This is a podcast for anyone who wants to learn about scientific studies and the statistics behind them. It's like a journal club, except we pick topics that are fun, relevant, and sometimes a little spicy.

We evaluate the evidence, and we also give you the tools that you need to evaluate scientific studies on your own. I'm Kristin Sainani. I'm a professor at Stanford University.

[Regina] (0:38 - 0:43)
And I'm Regina Nuzzo. I'm a professor at Gallaudet University and part-time lecturer at Stanford.

[Kristin] (0:44 - 0:49)
We are not medical doctors. We are PhDs, so nothing in this podcast should be construed as medical advice.

[Regina] (0:49 - 0:54)
Also, this podcast is separate from our day jobs at Stanford and Gallaudet University.

[Kristin] (0:54 - 1:01)
Welcome back, everyone, from the holiday break. This is the start of Season 2 of Normal Curves. Regina, how was your break?

[Regina] (1:02 - 1:09)
Oh, it was fabulous. I had a great time working on all of the fun episodes that we have in this upcoming season. How about you?

[Kristin] (1:09 - 1:19)
Yeah, the break was great, and we have some juicy episodes coming up, including the one today, which is in honor of Valentine's Day. What do you have for us, Regina?

[Regina] (1:19 - 1:30)
Kristin, have you ever heard of the viral New York Times Modern Love column from 2015 called To Fall in Love with Anyone, Do This?

[Kristin] (1:31 - 1:37)
No, I am familiar with the Modern Love column, but I don't remember that specific article, but it sounds intriguing.

[Regina] (1:37 - 2:16)
Yeah, it's one of those pieces that people still bring up years later. It's great. So I want to read to you how the writer Mandy Len Catron starts her essay, because it's really gripping.

She says, “More than 20 years ago, the psychologist Arthur Aron succeeded in making two strangers fall in love in his laboratory. Last summer, I, Mandy, applied his technique to my own life, which is how I found myself standing on a bridge at midnight, staring into a man's eyes for exactly four minutes.”

[Kristin] (2:17 - 2:20)
Wow, she had some fun doing research for that column.

[Regina] (2:20 - 2:58)
It's a fun story, actually, Kristin, how she applied this research to her own life. And she talked about it in the essay. She'd been at a bar chatting with a guy and told him about this cool study where psychologists made people fall in love.

And she described the study, which is pretty simple. She said, you got a lab, a straight man and a straight woman enter through separate doors. They sit face to face and answer a series of increasingly personal questions.

Then they stare silently into each other's eyes for four minutes.

[Kristin] (2:58 - 3:04)
Yikes, four minutes. That's a long time. I think I would get really uncomfortable after four minutes.

[Regina] (3:04 - 3:21)
It is really long, but maybe it's worth it, Kristin, because as Mandy mentioned later in the article, two of the study participants got married six months later and invited the entire lab to the wedding.

[Kristin] (3:21 - 3:25)
Oh, wow. Well, that'll do it. All it takes is one big success. So N of one.

[Regina] (3:26 - 3:40)
Right. So that is the claim, actually, that we are investigating today, that this scientifically studied protocol, which is usually just called 36 questions, can make people fall in love.

[Kristin] (3:41 - 3:45)
Sounds very intriguing, Regina, but I'm going to need details about the study.

[Regina] (3:45 - 4:01)
I promise you we will get to the study, which is linked in the Modern Love column. By the way, it was published way back in 1997 in a journal called Personality and Social Psychology Bulletin.

[Kristin] (4:01 - 4:08)
Personality and Social Psychology Bulletin. Wow, that sounds like gripping beachside reading.

[Regina] (4:09 - 4:41)
Oh, definitely. And today we'll do a little sleuthing. We'll also look at what that paper actually studied, what it found, what it didn't find, and how it turned into one of the most famous and maybe misunderstood papers on relationship science out there.

[Kristin]
Sleuthing. Yay.

[Regina]
We'll also talk through some statistical ideas along the way.

Things like what makes an appropriate control in an experiment and proper units of statistical analysis.

[Kristin] (4:42 - 4:48)
Oh, sounds exciting. But back to the column, Regina, what happened with Mandy and her guy from the bar?

[Regina] (4:49 - 5:11)
Right. So after she tells him about the study, he says, hey, let's do it right now. So she searches online for the study's questions, finds that 1997 paper, and then she and the guy spend the next couple of hours going through and answering them and then doing the eye gaze thing on a bridge at midnight.

[Kristin] (5:11 - 5:24)
Wait, OK, on a bridge. So Regina, did she know about the scary bridge study? We talked about that study in our Halloween episode last year.

So did she take him to a scary bridge to make him more attracted to her?

[Regina] (5:24 - 5:35)
That gets us to a fascinating detail, Kristin, because these two were actually in British Columbia, which is where the scary bridge study took place.

[Kristin] (5:35 - 5:35)
No way.

[Regina] (5:36 - 5:46)
Yeah. These two did their eye gazing experiment at midnight, and I'm pretty sure that they were not on the swaying Capilano Canyon rope bridge at midnight.

[Kristin] (5:46 - 5:48)
I'm just guessing. Yeah, probably not.

[Regina] (5:48 - 6:04)
But there's another weird fact. Arthur Aron, the first author of the study we're discussing today. But do you remember his name?

Because he was also a co-author on that scary bridge study from 24 years before.

[Kristin] (6:04 - 6:15)
Small world, I guess, Regina. But OK, so Mandy, our journalist, she went to a bridge, but probably not a scary bridge. They stared into each other's eyes.

And then what happened?

[Regina] (6:15 - 6:27)
According to her, the protocol worked. Because she closes her essay by saying that after they did all of this protocol, they were now in love and now were a couple.

[Kristin] (6:28 - 6:40)
So not only does she get to do really fun research for her writing job, but she met a man and fell in love through this. Can I have her job, please? Statistics, writings, not getting me dates, Regina.

[Regina] (6:41 - 6:47)
Kristin, are you saying we need even more love and sex on the podcast?

[Kristin] (6:48 - 6:48)
I think we might. Yeah.

[Regina] (6:48 - 7:05)
Be careful what you wish for. But a follow up, even better, Kristin, last year, 2025, which is about 10 years after the column was published, Mandy and this guy got married. The Times did a follow up story on it.

[Kristin] (7:05 - 7:18)
Oh, my goodness. And OK, 2025. So they haven't been married long enough to hate each other yet.

So I'm going to count this as a total success. And so now we're up to an N of two, two marriages from this protocol. Wow.

[Regina] (7:18 - 7:55)
I feel like that proves it. We're done and we can go home now because these love and marriage anecdotes, these stories are so appealing. People just loved these stories.

When the column came out, it went completely viral. And now it's often just known as the 36 questions column and still one of the most popular modern love pieces ever. Millions of people have read it.

Mandy did a TED talk. She wrote a book. The New York Times even made a phone app called 36 questions so you can try it yourself at home.

[Kristin] (7:56 - 8:05)
Wow. Now I'm really jealous. So you're telling me she got a man, a hugely popular column, a book, and a TED talk out of this?

[Regina] (8:05 - 8:35)
Absolutely right. This is good. And the questions, these 36 questions, they show up all over pop psychology websites.

You see them everywhere. I even found, Kristin, a site called never date again dot com where you could click on a stranger's photo and set up a video date to go through these 36 questions together at random. So I tried clicking, of course, but it looks like it's defunct now, sadly.

[Kristin] (8:35 - 8:45)
Well, now I want to do this, but maybe the fact that the site went defunct tells us something. Regina, have you ever tried the 36 questions protocol?

[Regina] (8:46 - 9:04)
Not with a random stranger online, but I did try it once with a guy. It was maybe our second or third date. He ended up whisking me off to a tropical vacation shortly afterwards, which is where we did the eye gaze thing.

We actually did that on the island and we dated for a while.

[Kristin] (9:04 - 9:15)
Oh, I think I know which boyfriend you're talking about. Regina, maybe it worked temporarily, but I don't think it was successful in the long run since you're not still with him.

[Regina] (9:17 - 9:32)
Well, that depends on your definition of success, right? Would we have dated and gone out on this great vacation without the questions? That's unclear.

So I'm I'm not entering this as evidence of success. I'm just making a note, let's say.

[Kristin] (9:32 - 9:37)
OK, so, Regina, I'm dying to know, what are these magic 36 questions?

[Regina] (9:38 - 10:13)
Well, before we get to them, Kristin, I want to talk about the idea, right? The science behind it. The core idea is pretty simple.

It's this structured series of questions and they are ordered so that they are increasingly self-revealing. And they do that because there is a lot of research that shows when we share personal things about ourselves and, of course, the other person is actually listening to us and sharing back, then we can feel closer to them, more intimate.

[Kristin] (10:14 - 10:16)
I mean, that makes sense. I totally believe that.

[Regina] (10:17 - 10:21)
OK, Kristin, would you like to fall in love with me and grow close to me?

[Kristin] (10:22 - 10:24)
Let's try it. Let's do it.

[Regina] (10:24 - 10:32)
OK, we're going to start with question one. Given the choice of anyone in the world, whom would you want as a dinner guest?

[Kristin] (10:33 - 10:37)
Oh, that is an interesting question. But, Regina, is that really supposed to lead to love? Really?

[Regina] (10:38 - 10:51)
You can't start too intense. You got to gradually build up to it. So the questions begin kind of innocuously, but then you build up towards this self-disclosure.

So what do you say then?

[Kristin] (10:51 - 11:02)
Oh, I feel like I need more time to think about this one. But if I had to pick on the spot, I'm going to say Jon Stewart, because he's super funny and I think would make for a really funny dinner.

[Regina] (11:02 - 11:03)
I like that one.

[Kristin] (11:03 - 11:03)
What about you?

[Regina] (11:04 - 11:11)
I actually have this as an answer to one of the prompts on the dating app profile, and I said, Otzi the Iceman.

[Kristin] (11:12 - 11:16)
Oh, well, you didn't tell me the person could be dead. Historical figures. That widens the pool.

[Regina] (11:17 - 11:23)
Right, right. He's a 5,300-year-old mummy.

[Kristin] (11:23 - 11:28)
I assume you don't want to have dinner with the mummy, but like if he could come back to life and tell you what happened to him.

[Regina] (11:28 - 11:32)
If he could come back to life. It’d be like, how'd you get that arrow in your back?

[Kristin] (11:34 - 11:38)
So you are better at answering these questions than I am, Regina. You're going to get more dates.

[Regina] (11:39 - 11:50)
Well, I don't know. I don't know if that's a particularly sexy answer. Okay.

Ready for a slightly more personal one? How about this? Would you like to be famous?

And if so, in what way?

[Kristin] (11:51 - 11:55)
Oh, well, Regina, that one is easy. I want to be famous for this podcast.

[Regina] (11:56 - 12:12)
The Sex Stats Podcast. The first and best and only sex and statistics podcast in the world. And this legit was actually my answer to this.

So we are thinking along the same lines.

[Kristin] (12:12 - 12:25)
Oh, I'm sorry. I stole your answer, but I'm glad we're on the same wavelength, right? And Regina, we clearly have our work cut out for us if we want to get famous for this podcast.

So everyone listening, please pass this on to your friends and family.

[Regina] (12:26 - 12:30)
I admire your shameless self-promotion so much.

[Kristin] (12:31 - 12:37)
No shame in promoting our teaching podcast. It's good for everyone.

[Regina] (12:37 - 12:56)
Oh, educational, very educational. Okay. So after that little bit of self-disclosure, let's talk about when things get more intense on the list. You get questions like, what is your most treasured memory?

[Kristin]
Treasured memory. Oh, that's a good one.

[Regina]
Another one.

How do you feel about your relationship with your mother?

[Kristin] (12:56 - 13:01)
Oh, that one is, wow. That feels like you need to be on a therapist's couch, not on a date.

[Regina] (13:02 - 13:13)
Oh, yeah. Oh, yeah. How about this one?

If you were to die this evening, what would you most regret not having told someone and why haven't you told them yet?

[Kristin] (13:13 - 13:15)
That one is tough, too, and really personal.

[Regina] (13:16 - 14:02)
Yeah, I think I am going to skip the whole podcast recording answers today and save those for a date or a therapy session.

[Kristin]
Agreed.

[Regina]
So that is basically the main mechanism, this escalating self-disclosure.

But there are a couple of other clever tricks that are buried in this whole list of questions. So there's a question where you're asked to say what you and your partner have in common. And there are prompts that force you to use we language, like we are both in this room feeling X.

Kristin, we are both sitting in our closets on a Sunday afternoon feeling a little cramped while we're recording this podcast.

[Kristin] (14:02 - 14:11)
That is very true. We are both taping from our closets. Although, Regina, you are a little more cramped than I am because I at least have a chair in my closet.

You're on the floor.

[Regina] (14:11 - 14:13)
Mine's a little bit more like yoga.

[Kristin] (14:15 - 14:20)
That's why we need to get famous for this podcast, so we can have an actual studio rather than our closets.

[Regina] (14:20 - 14:22)
An actual studio, or just a bigger closet, maybe.

[Kristin] (14:23 - 14:24)
Or a bigger closet, yes.

[Regina] (14:25 - 14:51)
So the idea basically here is just that we tend to feel closer to people we see as similar to ourselves. So it induces that. And the other trick is there are also questions where you're told to say what you like about the other person, to give them specific, honest compliments.

Because, of course, unsurprisingly, we tend to like people who like us, right, and see us and appreciate us.

[Kristin] (14:51 - 14:52)
Yeah, big shocker there.

[Regina] (14:52 - 15:07)
So behind all of these 36 questions, it sounds really reasonable, right, all of these principles. But I think that's why it's so important to sit down, look at the actual study, and say, what did it really look at?

[Kristin] (15:07 - 15:11)
OK, so let's actually talk about the paper itself. What did they do in this paper, Regina?

[Regina] (15:12 - 15:16)
OK, there's going to be some myth-busting going on here, Kristin.

[Kristin] (15:16 - 15:19)
Well, we love myth-busting on Normal Curves.

[Regina] (15:19 - 15:35)
We do. So here is the first myth-busting twist. And this one, honestly, it genuinely surprised me.

Despite what everyone thinks, this study was not actually designed to make people fall in love.

[Kristin] (15:36 - 15:43)
Wait, what? Didn't Mandy have that in her lead, that this was a study to make strangers fall in love? Did I get that wrong?

[Regina] (15:43 - 16:44)
No, that is what the column says. But if you actually read the published paper, that's not the goal at all. And the authors are very explicit about this.

And, Kristin, I think you will actually appreciate what they did set out to do. It wasn't to make people fall in love. They wanted to see, basically, if they could induce in the lab people to have these feelings of closeness.

So in other psychological studies, they can do things like temporarily induced different feelings, like moods, happiness or sadness. Or they found ways to temporarily make people feel socially excluded or powerful or insecure or even rich or poor. I mean, it's really cool how they figured out how to do all of that.

And these researchers wanted to see if they could do the same thing for inducing feelings of having a close personal relationship.

[Kristin] (16:45 - 16:58)
Oh, that is fascinating. So the goal was to see if they could come up with a protocol that induced at least temporary feelings of closeness between two strangers in a lab, but nothing to do with romantic love.

[Regina] (16:58 - 17:22)
No, not love, not lasting relationships. Just this feeling of closeness, something like what you might feel towards a close friend or a romantic partner, but only for the duration of the experiment. That's it.

And as you pointed out, Kristin, this is not what the column says. It says the research is about making people fall in love. It's very specific about it.

But that was wrong.

[Kristin] (17:22 - 17:31)
Well, I mean, it is the modern love column. So maybe she stretched it a bit because modern closeness column just doesn't have the same name to it.

[Regina] (17:32 - 17:52)
No, it doesn't. We will see where it came from, though. But but here is another weird myth about the column that I want to bust.

So the column said that four minutes of eye gazing was part of this whole protocol, part of the study. But, Kristin, there was absolutely no eye gazing in the paper.

[Kristin] (17:53 - 17:55)
Wait, what? So where did that come from?

[Regina] (17:56 - 18:02)
Uh-huh. I have scoured the study. It is nowhere in there.

There were indeed 36 questions, but no eye gazing.

[Kristin] (18:02 - 18:05)
Well, what happened? Where did you get the eye gazing from then?

[Regina] (18:05 - 18:07)
Yeah, this is the fun sleuthing part.

[Kristin] (18:08 - 18:52)
Oh, sleuthing. I can't wait. But let's take a short break first.

Welcome back to Normal Curves. Today we are talking about the 36 questions protocol, which was reported on in a modern love column in the New York Times in 2015. But so far we have learned that the column may not exactly represent the science beneath it.

Regina, you were about to tell us about some sleuthing.

[Regina] (18:52 - 19:23)
Absolutely. OK, this is the sleuthing part. So buried in the introduction of that 1997 paper is a brief mention of a pilot study that the authors had done about six years earlier.

And that pilot study was presented at a conference. And it's from that study that two participants met, fell in love and later got married. And the authors mentioned this pilot study almost in passing in the paper, just a parenthetical.

[Kristin] (19:23 - 19:31)
OK, so this was just a citation in the 1997 paper, and it was a reference to just a conference presentation, not even a published paper.

[Regina] (19:31 - 19:36)
It was the Conference on the International Network on Personal Relationships.

[Kristin] (19:37 - 19:41)
OK, that sounds one boring and two made up.

[Regina] (19:42 - 20:01)
It is a real conference. So I tried tracking down the conference proceedings, right? I wanted to know what was in this pilot study, but I could not find it anywhere.

There was no written summary, no paper, no proceedings online, nothing. So there was no way for me to figure out what they actually did in this original pilot study.

[Kristin] (20:02 - 20:23)
Oh, that is a problem. So we don't know anything about the methods, the data, what happened in this pilot study. And, you know, it sounds a lot like the ghost paper that we talked about back in the backfire episode.

Remember, there was a paper everyone cited, but nobody actually had ever seen the paper. I sleuthed that one down, though. Regina, did you sleuth out this one? Unghost it?

[Regina] (20:24 - 20:27)
I sleuthed out someone else who sleuthed it out.

[Kristin] (20:28 - 20:31)
Even better. Did I count? Yeah, of course.

[Regina] (20:31 - 20:35)
It was a software engineer named Ivan Vendrov.

[Kristin] (20:36 - 20:41)
Oh, Ivan Vendrov. It sounds like a name out of a Russian spy novel. So how did he get involved?

[Regina] (20:41 - 21:04)
Well, Ivan read this New York Times Modern Love column, and he got fascinated by this supposed formula for love because he likes math, but he wanted details. How long did people spend on each question? Right.

What exactly was the procedure? So he followed the link on the New York Times site to the published paper.

[Kristin] (21:05 - 21:14)
Hmm. So he's kind of like us. He wanted to know the science underneath.

And did he notice the discrepancies between the Modern Love column and the actual study that it linked to like we have?

[Regina] (21:15 - 21:33)
He absolutely did. He read it and thought, wait, this isn't about love at all. And he noticed, hmm, no eye gazing.

And then he noticed the reference to the pilot study and just like me, tried to find it, couldn't. But then finally, he just emailed Arthur Aron, the first author.

[Kristin] (21:33 - 21:34)
Oh, and what happened?

[Regina] (21:35 - 21:54)
Uh-huh. And Arthur Aron replied. He said he didn't have any of the full pilot study information available, but he did still have the original pilot study questions, which he shared with Ivan and which Ivan shared with the world.

He posted them on Twitter. Now, X.

[Kristin] (21:55 - 21:57)
Oh, very nice. We'll put those questions in the show notes, too.

[Regina] (21:58 - 22:42)
Absolutely, because once you see these pilot study questions, all of this myth busting just comes together. It makes sense because this pilot study is different from the study that was published in the 1997 paper that was linked to by the New York Times.

[Kristin]
Oh, how so?

[Regina]
Yeah, first of all, the pilot study wasn't 36 questions. It was 40. And the two lists still share some of the same questions, but a lot are different in important ways.

So they were all still progressively self-revealing, like we talked about. But the pilot study questions, some of them were much more explicitly romantic and sexual.

[Kristin] (22:43 - 22:45)
Oh, interesting. Give me an example.

[Regina] (22:46 - 23:09)
Right. So here are some examples that were in the original 40 questions, but were cut and did not make it to the published paper or the New York Times app. So, for example, share 10 things you find attractive about your partner, including how they look and how they dress.

Kristin, that is a wonderful sweatshirt you are wearing today. It looks really beautiful.

[Kristin] (23:09 - 23:15)
Yes, I'm all dressed up for taping in my closet, Regina. You know, got the hair, makeup, of course.

[Regina] (23:16 - 23:23)
Speaking of which, another question. If you wanted to look very sexy, how would you dress? Maybe nothing.

[Kristin] (23:23 - 23:27)
Well, again, nothing works. Or next to nothing also works. Yeah.

[Regina] (23:28 - 23:36)
So these next two I like very much. They are about role-playing. And there was nothing about role-playing in the 36 questions.

[Kristin] (23:36 - 23:41)
Well, role-playing, I mean, that's getting pretty intimate, right?

[Regina] (23:41 - 24:01)
Not that kind of role-playing. We are not in the bedroom yet. We are on our way to the bedroom.

OK, a little tamer, but still, you'll get the flavor. For example, role-play how you would ask your partner out on a date and have them reflect back how it feels to be asked.

[Kristin] (24:02 - 24:05)
Oh, that is definitely tamer than what I was imagining. OK, thank you.

[Regina] (24:05 - 24:12)
Yeah, I know, but still, still a little bit more explicit than we were getting in the 36 questions.

[Kristin] (24:12 - 24:14)
Yes, about love and romantic love.

[Regina] (24:14 - 24:41)
Yeah, right. OK, here's my personal favorite, though. Pretend you are in a play with your partner.

And in this particular scene, the director has asked you to tell your partner that you're interested in having more than just a casual relationship with them, that you are beginning to fall in love with them. Your partner is also in this play and will tell you how it feels to be asked to move into a more meaningful relationship.

[Kristin] (24:42 - 24:48)
Oh, that is super interesting. Although now I'm wondering, are we still in a scientific study or are we now auditioning for a play, Regina?

[Regina] (24:50 - 25:07)
We might be doing both. Kristin, I am ready to take our relationship to the next level and make it more meaningful. Let's start a podcast together.

I feel like this is a deep level of commitment that we have taken on here.

[Kristin] (25:07 - 25:08)
The podcast, oh, yes.

[Regina] (25:09 - 25:52)
OK, so this whole thing is a much more interesting protocol and listeners can go and compare the 36 questions and the 40 questions to see for themselves. But I will just tell you that there are some that are very different because there was no role playing or sexy dress or falling in love in the 36 questions. And that the New York Times has and that we have in the published article, Ivan also found that this pilot study was part of a dissertation and he got a copy of the abstract of that dissertation.

And from that, we can see that the researchers were not just measuring closeness like they were in the published paper here in the pilot study. They were also explicitly measuring attraction, which makes sense.

[Kristin] (25:52 - 26:03)
This is good. This is much more related to romantic love, much more pertinent to the modern love column. And so this is where they were actually trying to make strangers fall in love.

[Regina] (26:03 - 26:07)
Exactly. Or at least fall in love for an hour or so.

[Kristin] (26:08 - 26:12)
And the eye gazing, was this also part of the pilot study? Is that where that came from?

[Regina] (26:13 - 26:24)
Bingo. That is here too. Question 34 says, spend three minutes in complete silence with your partner, making only eye contact.

[Kristin] (26:25 - 26:34)
Oh, so the eye gazing was actually part of the 40 questions. I mean, it's not technically a question, Regina. So this is more like a 40 item exercise to do with a stranger.

[Regina] (26:35 - 27:02)
Yes, exactly. But what I love is that all the things that we that we just adore about this story, that we love about the story, psychologists making strangers fall in love, participants actually falling in love and getting married, the whole eye gaze thing. All of that comes from this earlier unpublished pilot study, not the famous 36 question paper, not the thing that is linked to in The New York Times.

[Kristin] (27:02 - 27:04)
So she got it a bit wrong.

[Regina] (27:05 - 27:15)
A little wrong. OK, a pilot study, though, where, by the way, Kristin, there was only one experimental condition, just the 40 items. There was no control group.

[Kristin] (27:16 - 27:28)
Oh, so the pilot study was a pretty weak study to begin with. Now, we don't have the methods or the results or the data from that. So we don't even know what that found.

But even if they had found something, it would not be strong evidence since there is no control group.

[Regina] (27:28 - 28:09)
There is no control group. So here's my hypothesis, my just imagining of this. Mandy, the author of the column, had heard about the pilot study years earlier.

Who knows how she heard about it? She had written a lot about love and relationships, and she remembers all of the juicy parts. And then years later, when she's sitting in the bar with this hot guy, her potential mate, she mentioned it.

He agreed to do it. And so, of course, she is looking online right there for the questions. But all she can find was the sanitized published version, not the pilot version, not the actual thing that she's remembering.

That's how I imagine it.

[Kristin] (28:10 - 28:33)
Actually, that makes total sense, Regina. So maybe she went online, she found the 36 questions from the published study. She ended up just using those questions, not realizing there were earlier questions.

And maybe the eye gaze part, she just remembered and stuck that in there, didn't read the paper carefully. And really, Regina, all of this was just a ploy for her to get a date with the hot man at the bar, really.

[Regina] (28:33 - 29:10)
Well, it worked.

[Kristin]
It did work.

[Regina]
Whatever it is, it worked for her.

But this is what I find so fascinating and also hilarious. The big kind of bombshell here for me is that ever since this Modern Love column was published, 2015, so for 10 years, people have been using these 36 questions to try to get other people to fall in love with them. The New York Times made an app.

Everyone's doing this, but they are the wrong questions. They left out the things in the original study that actually relate to love and attraction and sex.

[Kristin] (29:11 - 29:35)
There is a good lesson to be learned here about always going back to original sources and reading the study because there's all this stuff that came out of Mandy's column. But it turns out she got it wrong and we're using the wrong questions. Well, maybe that's why the Never Date Again website went defunct, Regina.

I think we need to make a new app that has the right questions, the 40 original questions, Regina.

[Regina] (29:35 - 29:44)
We absolutely do. I feel like they need to be out there because this was the original. This was the OG.

We're going to do the OG. Yes.

[Kristin] (29:44 - 29:57)
All right. So she got it kind of wrong, but we are talking about that 1997 study today with the 36 questions. They looked at closeness, not love.

There was no eye gazing. But I'm still curious as to what that study found. So can we dissect that study now?

[Regina] (29:57 - 30:07)
Absolutely. So they tested out this 36 question protocol in large intro psychology classes at University of California, Santa Cruz.

[Kristin] (30:08 - 30:16)
Oh, UC Santa Cruz. So, Regina, you and I are both banana slugs. We are graduates of their science writing program.

And that is an amazing college.

[Regina] (30:17 - 30:30)
Amazing school. We were not there when they did this 36 question protocol. So this study, everything was designed to fit into a single 45 minute class period.

[Kristin] (30:31 - 30:43)
So very short. And all of the participants are college psych students. And we've talked before on this podcast, Regina, about how a lot of psychology studies are done on undergraduates looking for extra credit.

[Regina] (30:44 - 31:18)
Right. Same thing here. So the students were randomly assigned into pairs.

Some were mixed sex pairs, men and women. But because there were more women in the classes, some were also a bunch of women-women pairs. And in this paper, there were three studies.

And this next part really matters. OK. OK.

Because in study one, they compared the 36 questions to a control condition. And in that control condition, the pairs went through a different set of questions that the author called small talk.

[Kristin] (31:18 - 31:22)
Oh, OK. Well, give me some examples of the small talk questions, Regina.

[Regina] (31:22 - 31:32)
Mm hmm. Very impersonal, kind of boring. Things like, how did you celebrate Halloween?

What's the last concert you saw? Who's your favorite actor?

[Kristin] (31:32 - 31:37)
I like those questions. You know, I think I want to be the control group for this study, Regina.

[Regina] (31:39 - 31:55)
I don't know if anyone's going to fall in love with you about your favorite actor. You're not being self-revealing enough there. But the point is, they were still talking.

They were still interacting for the same amount of time. And that's what makes a good control condition here.

[Kristin] (31:56 - 31:56)
Yeah.

[Regina] (31:56 - 32:06)
So after the pairs went through the questions, the students were separated and each asked to fill out a short questionnaire measuring how close they felt to their partner.

[Kristin] (32:06 - 32:08)
Regina, what do we even mean by closeness?

[Regina] (32:09 - 32:30)
Yeah, warning, it's kind of a gooey language here, psychology speak. Sometimes psychologists describe it as expanding your sense of self to include another person or a feeling of interconnectedness, of intimacy, of feeling understood and validated by another person.

[Kristin] (32:31 - 32:33)
OK, so how did they measure closeness?

[Regina] (32:34 - 32:42)
Right. So the researchers used three items. And just to emphasize again, none of them has anything to do with love or romance.

[Kristin] (32:42 - 32:57)
Right. So this does not involve questions like, how much do you want to engage in sexual behaviors with this woman? Which is a question we saw back in the red dress episode where they were actually measuring attraction and not just closeness.

[Regina] (32:59 - 33:26)
Definitely not that. This first one is actually kind of clever. It's visual.

So it's a set of pictures that show two circles and one circle labeled self and another circle labeled other. And there are seven versions of this. And they range from no overlap at all to almost complete overlap.

And you just pick the picture that best represents how close you feel to your study partner at that moment.

[Kristin] (33:27 - 33:31)
Oh, I like that. That's really pretty intuitive and very visual.

[Regina] (33:31 - 33:58)
Yeah. Yeah, I like it. OK, second question asks, relative to all your other relationships, how close is this one with your study partner to those?

And that's on a one to seven scale from not at all close to as close as any. And the third asks the same thing, but now it's relative to what you know about other people's close relationships. One to seven.

[Kristin] (33:59 - 34:06)
OK, so we end up with three numbers. They're all one to seven. And what do they do with these?

Like add them, average them. What?

[Regina] (34:07 - 34:23)
They just average them together to get one number representing closeness. Higher means closer. Now, that was study one.

Studies two and three look very similar to study one, but with one really important difference.

[Kristin] (34:24 - 34:25)
What was that?

[Regina] (34:25 - 34:26)
No control group.

[Kristin] (34:26 - 34:34)
Oh, we don't like this. We don't like studies with no control group. So how do you have no control group?

What exactly did they do?

[Regina] (34:35 - 35:23)
No. So in studies two and three, they're no longer testing whether this protocol, the 36 questions, works better at inducing closeness than any other ordinary conversation. No, they're doing something different.

Here, they ask the participants to fill out a pre-study questionnaire about two weeks before they get together to do the study. And from this questionnaire, they can get a lot of information about characteristics of the participants. And just one example, they were able to figure out whether participants were introverted or extroverted.

So the researchers paired introverts with introverts and extroverts with extroverts. And then they had all of them go through the 36 questions and rate their closeness to their partner, but no control group.

[Kristin] (35:25 - 35:35)
Help me out here, Regina. What were they actually trying to show? Like that the 36 questions work differently for introverts versus extroverts?

Some kind of moderator effect, maybe?

[Regina] (35:35 - 35:37)
Exactly. A moderator effect.

[Kristin] (35:37 - 36:07)
OK, my brain is short circuiting because that does not work. Here's the problem. Let's say extroverts tend to feel closer to people no matter what.

I could have the pairs blow bubbles, stand on their heads, wiggle their toes, read IKEA instructions, and then ask them how close they feel. Extroverts would probably rate higher closeness every time. But what did I learn?

Nothing. I don't know whether it's the activity or just that extroverts rate closeness higher in general.

[Regina] (36:08 - 36:14)
Absolutely. You need a control group. You cannot conclude anything without a good control group.

[Kristin] (36:14 - 36:52)
You know, Regina, this reminds me of a paper I saw recently where they made the same error. They wanted to show whether a specific drug increased fracture risk more in people with osteoporosis than people without osteoporosis. But they compared osteoporotic people on the drug to non-osteoporotic people also on the drug.

That tells you nothing. People with osteoporosis fracture more anyway. So if they fracture more while taking the drug, you have no idea whether that's because of the drug, the osteoporosis, or the combination of the two.

You need a control group here.

[Regina] (36:52 - 37:02)
You need people who are not on the drug. I think, Kristin, just to summarize all of this, in other words, what you're trying to say, studies two and three are meaningless.

[Kristin] (37:03 - 37:06)
They are useless and meaningless. Absolutely, Regina.

[Regina] (37:06 - 37:23)
You know, I once heard someone boil the entire field of statistics down to one question, compared to what? And by that measure, this is not statistics. There is no comparison group here.

No control.

[Kristin] (37:23 - 37:36)
That is well put, Regina. Very good. Yes.

All right. So studies two and three, we're just going to throw away. But we still have study one.

And we can get something meaningful out of study one because they compared the 36 questions to a control group. So what did they find there?

[Regina] (37:37 - 38:08)
Yeah, well, let's see about that. So here they had 100 students. They formed 50 pairs of students, and they assigned half to the 36 questions or the small talk.

But here's an interesting methodological point. I think you'll appreciate, Kristin, the unit of analysis here is the pair, not the individual person. So for each pair, they averaged the two people's closeness scores and treated that as a single data point.

[Kristin] (38:09 - 38:28)
Oh, that is a very interesting statistical point, because these pairs are not independent observations. The two people went through the same experience. So if one feels closeness, the other one is more likely to feel closeness.

So we can't treat them as if they are totally independent observations. They're correlated.

[Regina] (38:28 - 38:42)
Right. If we had analyzed each of these individuals separately, right, independently, which is sometimes what you see researchers do, that would be wrong because that would have artificially inflated their sample size.

[Kristin] (38:43 - 39:07)
Yeah, exactly. You'd be kind of double counting people. So actually averaging the two together so that we make the unit of analysis the pair rather than the individual, that is a perfectly reasonable way to handle the correlated observations.

There are other ways I should point out. There are ways that you can preserve the individual scores and account for the correlation. But for a 1990s paper, I think it's good that they even recognized it.

[Regina] (39:07 - 39:26)
I agree. I was very impressed with that part. Okay, so results.

The mean closeness score for the 36 questions condition was 4.06. For the small talk control condition, it was 3.25. And this is on a one to seven scale.

[Kristin] (39:26 - 39:41)
Oh, okay. So the 36 questions did produce more closeness to some extent, although both groups are kind of in the middle of the scale. So nobody felt intensely close after the end of this.

You didn't give me standard deviations, though, Regina. Those were the means. What were the standard deviations?

[Regina] (39:41 - 39:50)
You notice that that is the frustrating part. They did not report any standard deviations, no plots either, no distributions, just the means. That's it.

[Kristin] (39:50 - 39:56)
Oh, well, even though this is a paper from the 1990s, they should at least report the standard deviations.

[Regina] (39:56 - 40:09)
So good for them for the proper unit of analysis. Bad for them for no standard deviations or plots or anything else. Yep.

They do report standardized effect size, Cohen's d, which we've talked about before.

[Kristin] (40:10 - 40:25)
Here, it's 0.88. Okay, so the general rule of thumb that people like to use is if the Cohen's d is above 0.8, that's often considered a large effect. Don't put too much stock in that classification, but at least that gives us a feel for the size of the effect here.

[Regina] (40:25 - 40:31)
And they report the effect was statistically significant. And they give an F statistic here. And I can work out that the p-value is about 0.02.

[Kristin] (40:31 - 40:58)
Which tells me, Regina, that the confidence interval on that Cohen's d is going to be really wide. And so we, you know, we don't have the most reliable estimate here. All right, but it is statistically significant.

So they concluded the 36 questions condition beat this small talk. And take home message then, at least in this study, the 36 questions protocol did a little better than boring small talk.

[Regina] (40:58 - 41:13)
But that is it, because this is the only study in the paper with a control group. So this is the only evidence that we have that the 36 questions protocol actually influences closeness at all.

[Kristin] (41:13 - 41:24)
Right. As we said, we can just dump the other two studies because they're useless. So, Regina, okay, this is pretty weak.

But I'm assuming that somebody else has followed up on this since 1997 and tried to replicate it. Do we have any replication studies?

[Regina] (41:24 - 41:48)
We do. But before we talk about that and how long it took for anyone to actually try a replication study, I want to talk about just two things in the discussion section. Because this is where the authors really start to overreach and get a little creative.

And, Kristin, I think you'll love it. And, of course, by that, I mean, you'll hate it.

[Kristin] (41:50 - 42:32)
All right. I'm looking forward to hearing that. But let's take a short break first.

Welcome back to Normal Curves. Today we're talking about the famous 36 questions that supposedly would make you fall in love with a stranger. Although we've figured out now that that's not actually what the study showed.

And, Regina, you promised to give me more things to get me up on my statistical soapbox. What do you got for me?

[Regina] (42:32 - 42:53)
Oh, yes. I love getting you indignant. Okay, so I was especially disappointed by the discussion section here, I got to say, because they really do overreach, but they do it in ways that I think are kind of common in scientific papers.

So that's why I thought it was worth just slowing down and talking about it for a moment.

[Kristin] (42:54 - 42:55)
I already hate it.

[Regina] (42:56 - 43:19)
Yep, yep. Okay. It almost felt like when they sat down to write the discussion, they belatedly realized that they didn't actually have super strong evidence that the protocol did anything beyond any other 45-minute interaction.

So they tried to justify it with, let's say, some creative interpretations of the numbers.

[Kristin] (43:19 - 43:27)
And when it comes to statistics, we are not looking for creativity. That is not a bonus. This is not painting, people.

[Regina] (43:27 - 44:28)
No, this is not. This is not painting. Okay, so here is the first creative move that they make.

They take apart that composite closeness score that we talked about, and focus on just that first measure, the overlapping circles measure that goes from one to seven. And they say that averaged across all three studies, the mean score for the people who did the 36 questions was 3.82. Then the researchers compare that to a completely separate study where about 300 students from the same university rated their, quote, closest, deepest, most intimate relationship using that same circles measure. And in that completely separate study, the mean was 4.65 with a standard deviation of 1.5. And then what happens, this is important, the researchers say they treat that separate study as a reference distribution.

[Kristin] (44:29 - 45:32)
Regina, let's pause here and explain what a reference distribution is. It tells you about the distribution of a trait in a population. And good examples like SAT scores, everyone's familiar with those.

Every year, millions of students across the U.S. take the SAT under standardized conditions. From that, we get a reference distribution of SAT scores, and then we can compare individuals to that reference distribution. So, for example, nationally, a score of about 1050 is around the 50th percentile, and 1350 is around the 90th percentile.

So if you scored 1050, you did better than 50% of test takers. And if you scored 1350, you did better than 90% of test takers. Now, it sounds like, Regina, that in today's study, they are trying to create a reference distribution about feelings of closeness, but they are using a very small, unstandardized sample.

So right away, that's problematic. That is not a reliable reference distribution.

[Regina] (45:33 - 46:23)
Absolutely not a reliable reference distribution. But, Kristin, it gets worse than that. Because, let's say, even if this reference distribution was correct, even if the mean of everyone was 4.65 with a 1.5 standard deviation, they used the reference distribution incorrectly. Because what they do is say, oh, the mean score from our study was 3.82, and 3.82 falls at the 30th percentile of that reference distribution of everyone. And they conclude, therefore, this is a direct quote, after 45 minutes, this relationship with their partner is rated as close or closer than the closest relationship in the lives of 30% of similar students. And you can see the problem with that.

[Kristin] (46:24 - 46:56)
Oh, my goodness. Okay, you cannot do that. That is statistically illegal.

And the stats police will arrest you and you will do hard time for that one. Because let's think about SAT scores again. So let's say I calculate the average SAT score at my kid's high school, and it's a good high school.

So imagine the average is 1350. Do I then get to go out and announce, this high school beat 90% of students nationwide? No!

I've just taken a mean, treated it like a teenager and assigned it a percentile, and you can't do that.

[Regina] (46:57 - 48:15)
I love that analogy so much. The mean of a high school is not the same thing as a teenager's actual score. High school is not the same as a teenager's.

So the big picture here is group statistics, like a mean, they're not the same thing as individual values. You cannot treat them the same. And that's what they were doing here.

OK, so that was the first creative interpretation move that they made. Here is the second one, Kristin. So this time they look at the other part of that composite closeness measure.

That was where students rated two statements on a scale of one to seven again. So focusing just on that part, the researchers say that across all three studies, students who went through the 36 questions protocol had a mean score of about four. And then they say, well, hey, look, four is the midpoint of that scale.

The scale runs from one to seven. Four is in the middle. It's the midpoint.

Therefore, get ready. They conclude that participants rated their relationship with their study partner after the 36 questions as being about as close as the average relationship in their entire lives and other people's lives, just based on that four.

[Kristin] (48:15 - 48:57)
OK, again, you cannot do that. Also, statistically illegal because the midpoint of the scale is not the same thing as the average value in a population. The midpoint just tells you something about the scale itself.

It's basically a neutral point here. You're not very distant and you're not very close. It has nothing to do with how people actually feel in their real relationships.

Take an analogy. Imagine, Regina, you have a ruler that runs from four feet to eight feet. The midpoint is six feet.

If I measure a group of people and their average height happens to come out to six feet, I don't get to say that these people are as tall as the average person in the world.

[Regina] (48:58 - 49:21)
I love this analogy. The midpoint of the ruler has nothing to do with the average height of the actual human. And same thing here.

The midpoint of this closeness scale that we made up is just the middle of the scale. It tells you nothing about how actual humans actually fall on that scale.

[Kristin] (49:22 - 49:29)
Maybe we all hate each other and our average is really a two. Or maybe we're all very close and our average is seven.

[Regina] (49:31 - 49:33)
Glass half full, half empty.

[Kristin] (49:33 - 49:49)
Exactly. And Regina, you know, it's true that authors often speculate overreach in discussion sections. But I find these particularly egregious because they're using incorrect statistical logic to do their overreach.

Usually authors are just flowery. This is worse than that.

[Regina] (49:49 - 49:56)
Absolutely. Are you ready to talk about the next question? Did anyone ever try to replicate this?

[Kristin] (49:56 - 49:58)
That's what I want to know, Regina.

[Regina] (49:58 - 50:16)
OK, this is what really surprised me. This paper was published in 1997. And after that, most researchers basically took the effectiveness of these 36 questions as a given based on that one experiment with 50 pairs of students.

[Kristin] (50:17 - 50:26)
That is wild. It kind of reminds me of the pheromones paper that we looked at where everybody just believed it. And the thing is, this would be so easy for somebody to try to replicate.

[Regina] (50:26 - 50:53)
This seems like it would be a gimme. I thought surely someone must have done a clean replication with a control group. But Kristin, I found only three.

Now, maybe there were more, but I only found three. Now, the first was in 2005. They tried to replicate those 36 questions against, again, a small talk control.

And they found no statistically significant difference in closeness.

[Kristin] (50:54 - 50:55)
Oh, wow. So it did not replicate.

[Regina] (50:56 - 51:15)
Did not replicate. Second one, 2012. Now, this one did find higher closeness, but the setup was a little different.

First of all, they used only 15 questions, not 36. And the control group was not small talk. It was something else.

[Kristin] (51:16 - 51:16)
What was it?

[Regina] (51:17 - 51:47)
It was a little bit of a strange protocol. So the pairs had to work together to read and copy edit a scientific journal article in Aquatic Botany. That was, by the way, titled, excitingly titled, Chloroplast Evidence for the Multiple Origins of the Hybrid Potamogeton Exflutans.

[Kristin] (51:49 - 51:56)
I guess the idea was to give them a boring and administrative task so that they wouldn't bond.

[Regina] (51:57 - 52:02)
I feel like they set up this task to make the pairs hate each other. Have you ever tried editing, group editing with someone? No.

[Kristin] (52:03 - 52:13)
I kind of enjoy group editing, so I don't know.

I make my students do this exact thing in my class, so I'm hoping they're bonding.

[Regina] (52:13 - 52:24)
So, yeah, the 15 questions that they had that were condensed down from the 36 questions, they beat this very idiosyncratic scientific editing task.

[Kristin] (52:26 - 52:33)
Right. So it's maybe there's something there, but we're not really sure it's the right control group and it's not an exact replicate of the original.

[Regina] (52:33 - 53:00)
Nope, nope. OK, and then finally, finally, in 2021, nearly 25 years after the original publication, someone did a more careful, randomized, controlled replication. And they included three conditions this time, the 36 questions, the structured small talk condition like before, but now they added an unstructured just talk like you normally would condition.

[Kristin] (53:01 - 53:03)
Oh, more just like a regular first date.

[Regina] (53:03 - 53:34)
Mm hmm. Yep. Yeah, that's exactly what it's like.

So the tasks were a little bit shorter. This was only 24 minutes instead of 45 minutes for some reason. But now I feel like at least we're comparing the right things and the results were interesting.

So the 36 questions did better than the boring small talk. On closeness, they also looked at things like, how much do you like your partner? How enjoyable was this?

Do you feel like you're similar to your partner? Do you want to interact with your partner again?

[Kristin] (53:34 - 53:42)
OK, so this actually replicated the original 1997 study with some additional variables that they found differed between the two conditions. So that's good.

[Regina] (53:42 - 54:01)
Mm hmm. It's good. But this next twist is kind of interesting.

When you compare the 36 questions to the unstructured normal conversation, no difference. No difference on closeness, not on liking, not on how fun it was, not on wanting to see the person again.

[Kristin] (54:01 - 54:11)
OK, so basically, the 36 questions are better than a deliberately dull script, but not better than just having a normal conversation. Is that what you're telling me?

[Regina] (54:11 - 54:28)
That's exactly what I am saying. Yes.

[Kristin]
Wow.

[Regina]
But more importantly, for people like you and me, the takeaway for real life, what am I doing with this? I'm feeling like the answer is don't read or edit a scientific article on a date.

[Kristin] (54:28 - 54:37)
I don't know if I agree with that one. I think that might be a good date, Regina. So I'm not sure, maybe I'm unique, but I'm not sure that I want to take that off the table for dates.

[Regina] (54:38 - 54:41)
You are endlessly fascinating.

[Kristin] (54:41 - 54:47)
That sounds kind of fun. Better than having to stare into somebody's eyes for four minutes. Are you kidding me?

[Regina] (54:47 - 55:05)
OK, we're putting this on your dating profile for sure. OK, for the rest of us, do not read and edit a scientific journal article on a date. Don't work through a list of boring small talk questions either.

Again, for most of us, Kristin, you might enjoy that a little bit more.

[Kristin] (55:05 - 55:09)
No, that one, no, I hate small talk like that. No, I agree with that one.

[Regina] (55:09 - 55:22)
Instead of interrogating each other about your deepest childhood wound, you know, and your bad dreams and your relationship with your mother and your greatest fears, maybe, maybe just have a normal conversation.

[Kristin] (55:23 - 55:34)
Well, I think that's good advice, actually, Regina. And don't do aquatic botany on a date feels like a solid methodologic moral already. I will say aquatic botany, not my thing.

[Regina] (55:34 - 55:39)
Yeah, not not OK, but if it was a statistics article, then that might be an aphrodisiac.

[Kristin] (55:39 - 55:40)
It's hot in here.

[Regina] (55:41 - 55:53)
You are insane. I love it. OK, I am setting up speed dating just for you.

And it's going to involve scientific editing.

[Kristin] (55:53 - 56:05)
And I think we should set up speed dating with the original 40 questions, eye gazing and scientific editing all in one. And then that's how we're going to match people and make them fall in love.

[Regina] (56:07 - 56:09)
That's a big business venture right there.

[Kristin] (56:09 - 56:11)
Big business venture. I'm telling you, it's going to be a winner, Regina.

[Regina] (56:12 - 56:46)
Oh, I think so. OK, but one last little teaser here, Kristin. So we haven't talked again about that eye gaze part.

And not in the original published paper, but I went and looked up that topic in the literature. And I feel like it actually does have some maybe strong evidence behind that, like eye gaze might actually work. But since it wasn't part of the study, we're not doing it now.

I am saving that for a future episode.

[Kristin] (56:47 - 56:58)
Oh, excellent. Teaser and cliffhanger, Regina, a way to set up our future episodes. But Regina, I feel like we're ready for our wrap up now.

And so what was the claim for today?

[Regina] (56:58 - 57:05)
That this structured protocol, the 36 questions, makes people fall in love.

[Kristin] (57:06 - 57:20)
And how do we rate the strength of evidence for claims on this podcast? With our one to five highly scientific smooch rating scale, where one means little to no evidence for the claim and five means strong evidence for the claim. So, Regina, kiss it or diss it?

[Regina] (57:20 - 57:22)
I'm going to give it one smooch.

[Kristin] (57:23 - 57:23)
Our lowest.

[Regina] (57:24 - 57:42)
When you look at the claim, makes people fall in love. I'm sorry, but no, not not these 36 questions. Not that.

I mean, maybe the 40 questions, including the eye gaze, but based on what we've seen from the paper, no, this is not making anyone fall in love.

[Kristin] (57:42 - 58:03)
Yeah, I'm going to agree with you, Regina. The evidence for falling in love, just not here. We do have the N of two, but those are just anecdotal.

And we would need to replicate the pilot study, really, to see if we could actually induce falling in love. So I'm also going with one smooch. And I think we got to contact the New York Times and tell them they got it wrong.

Their app is all wrong.

[Regina] (58:04 - 58:22)
I will point out that it is possible to fall in love going through these 36 questions, but it's possible to fall in love doing a lot of things. You know, riding on the subway is enough. If you want to fall in love.

[Kristin]
Standing on a scary bridge.

[Regina]
Scary bridge. There are a lot of ways to fall in love.

[Kristin] (58:22 - 58:25)
Yes. All right. So, methodologic morals. What do you have for us, Regina?

[Regina] (58:26 - 58:34)
How about this one, Kristin? Before you repeat a scientific claim, follow it back to the original study and read it carefully.

[Kristin] (58:35 - 59:08)
Oh, that is great advice. And I think this is something that's going to keep coming up again and again on Normal Curves. Always go back to original sources and make sure you've checked the original source.

Otherwise, you can get things wrong.

[Regina]
What about you?

[Kristin]
So, Regina, mine is around that nonsense they did with, hey, if we divide up into subgroups, that somehow magically creates a control group.

So, here's mine. You can slice the data into subgroups all you want, but that doesn't magically give you a control group. It gives you meaningless results.

[Regina] (59:08 - 59:14)
Oh, I like it. Zinger. You need a control group, people.

You need a control group.

[Kristin] (59:14 - 59:21)
People get this wrong a lot, I've seen. They think that somehow that there's some useful comparison in there, but without the control group, there's not.

[Regina] (59:21 - 59:24)
No. I'm going to get that tattoo. Compared to what?

[Kristin] (59:24 - 59:36)
Compared to what? That's a methodologic moral right there, too, Regina.

[Regina]
There you go.

[Kristin]
Well, I think we just debunked a New York Times famous article and app. And so, we've got to send this to the New York Times, clearly.

[Regina] (59:37 - 59:38)
You heard it here.

[Kristin] (59:39 - 59:48)
All right. I think this has been a great start to season two. We have some really great episodes planned.

This one was fascinating for Valentine's Day, Regina. So, thank you.

[Regina] (59:49 - 59:50)
Thank you, Kristin. Thanks, everyone, for listening.