Screening/Participation Instruments

OMB No. 1505-0233

Moderator’s Guide—Consumer

Greet participant and thank him or her for coming. Ask participant to take a seat and if he or she had any trouble finding the site or if he or she has been there before.

Introduction of Moderator and Notetaker (10 minutes)

Welcome, and thank you for coming today. My name is _________________ and this is ________________. We are from the Kleimann Communication Group, a design and research firm in Washington, DC. Before we get started, I want you to know that I will be reading from a script. We are talking with a number of people this week, and we want to be sure we say the same things in the same way to everyone.

T he material we are testing today is part of a consumer study about the disclosure you receive when you apply for a loan to buy or refinance a home. The study is being conducted by the Consumer Financial Protection Bureau, a government agency that is also known as the CFPB.

Note to Moderator: Hand participant piece of paper with the CFPB website address:

http://www.consumerfinance.gov/knowbeforeyouowe/

Representatives from the CFPB are observing today. They are very interested in hearing your thoughts. Let me know at the end of the session if you would like to meet them.

The CFPB is working to create a disclosure that will help consumers better understand their loan terms when they apply for a loan to purchase a home or refinance an existing loan. What you say to us today will help us do that.

I will be leading today’s session and __________ will be taking notes to help us remember what you say. We will be audio- and /or video- taping this session, based on what you agreed to, to ensure that we collect complete information. The entire session will take no more than 90 minutes, and we will take a break about half way through the session. The questions we will be asking have been approved by the Office of Management and Budget and have been assigned OMB Control Number 1505-0233. Any questions so far?

Confidentiality

I have a few questions to ask you about the paperwork you filled out when you arrived. Can you please answer either yes or no to each of the questions that I will ask you? (If participant nods or shakes head, ask him or her to say the answer out loud for our tape)

• Did you fill out a questionnaire when you arrived today?

• Were you given consent forms to participate in this session and for us to audiotape and videotape your session today?

• Did you read and sign those forms?

• Did you give permission for both audiotape and videotape? (If did not give permission for video, say, “One of my colleagues in back will make sure that the video is off. “)

NOTE TO MODERATOR: Do not proceed with the session unless the participant answers “yes” to each of the first three questions. (Participant must agree to audio but video is optional). Do you have any questions?

All of the information we collect today will be kept confidential to the extent permitted by law, and we will not identify you by name when we compile our results. In addition, we will not use your full name, address, or any other personal identifying information other than your image in reports, papers, or videos based on this research. I want to remind you that you may end the interview at any time.

About the Session

Over the next 90 minutes, I am going to ask you to do a couple of things and to respond to a series of questions. As I go through these questions, and you give me your responses, please remember there are no right or wrong answers. We aren’t testing you—even if at times it may feel like we are. We’re testing the disclosure and any information you give us is good information.

We want to learn from you what works best for consumers to help them understand mortgage loan disclosures and if these disclosures give you the information you need to decide if you are interested in the loan.

Introduction to Think-Aloud Technique

One of the best ways to learn about a disclosure is by watching someone interact with it for the very first time. Once I give you the disclosure, I’m going to ask you to talk out loud about what you are thinking as you read it. You might be reading out loud and thinking, “Wow! This is great!” or “I don’t like this!”, say those things out loud. Or maybe you think “What does this word mean?” Say it out loud. We want to hear that inner voice in your head that comments on things you read. Talking out loud is the only way we can hear what you are thinking and how you are reacting. This may seem a little odd, but we really want to hear everything that you are thinking as you interact with this disclosure for the first time, so we can improve it further.

N ote to Moderator: If needed

To learn from you, it is important that you:

• Tell us out loud where you are in the disclosure.

• Tell us what you are reading, skimming, skipping in the disclosure.

• Tell us any questions, suggestions, or confusion that you might have about any part of the disclosure.

Talking out loud is very important because our notetaker will be recording what you say as you go through the disclosure. What you are thinking is more important than what you are reading.

I know this technique of thinking aloud might seem unusual. So, I’d like to let you practice. I’m going to give you a menu from a restaurant and I’d like you to “think aloud” and tell me how you would go through this menu and decide what you’d want to eat. Again, I want to hear what you are thinking, reading, skimming, skipping, how you react to the menu, and how you go about making a decision. Any questions?

Note to Moderator: Hand participant Menu.

Now, here is the menu. Remember, that you are reading to decide what to order. To get you started, what is the first thing you see or notice?

N ote to Moderator:

• Give the participant the menu and get him or her comfortable with talking about what he or she is looking at or reading, and with voicing questions, confusion, and decisions.

• Coach the participant to use the technique. Be sure that he or she has looked at and commented on the following:

1. the appearance—color, font, layout

2. symbols and graphic elements, such as pictures or symbols

3. details, such as why some items have descriptions and others don’t

4. order in which he or she reads the information

• Ask the participant what he or she is reading, where he or she is looking, and when a decision can be made. The task should take no more than 5 minutes.

• Important: This task sets the stage for how well the participant will “talk aloud” in Task One.

Great! You understand exactly what we are looking for. Do you have any questions before we get started?

Task One: Think Aloud (10 minutes)

Okay, in a moment I am going to hand you a mortgage loan disclosure.

Note to Moderator: Hand participant Scenario Card and read out loud.

Scenario-Purchase

You have been shopping for a new home and have found a house that you would like to buy. The house costs $350,000 and you are planning on putting 10% down, so you want to borrow $315,000.

Purchase Price of home: $350,000

Down payment: $35,000 (10%)

Amount you want to borrow for the purchase price of the house: $315,000

In reviewing the mortgage loan disclosure, I’d like you to imagine that you have contacted a mortgage broker or lender because you are interested in purchasing this home. After giving the broker or lender information about you and the house, you have received this disclosure which contains information about the loan. Your task is to look at this disclosure to decide if you are interested in this loan.

When I hand you the disclosure, I want to hear what you are noticing and thinking about as you look through it as well as your thoughts, questions, and ideas about the mortgage disclosure. Tell me what you like about what you see and what is confusing so we can improve the disclosure. Remember we want to hear your thoughts—both positive and negative--so we can make sure consumers have clear information about the loan terms.

Please go through the disclosure exactly as you would if you received this from a lender or broker. As you go through it, please speak aloud so I can hear what you are thinking just as we did with the menu. Remember this is a test of the disclosure, not a test of you. Do you have any questions before we get started?

Okay, here is the disclosure.

Note to Moderator: Hand participant mortgage loan disclosure.

To help get you started, what is the first thing you notice?

N ote to Moderator:

• Allow participant time to review the disclosure and comment on it.

• Ask participant to put disclosure on table.

• State aloud whenever the participant moves to another section or page.

• Ask participant what he or she is looking at.

• If participant looks confused, ask “is there something that is confusing there?”

• Remind the participant to tell you what he or she is thinking, what he or she likes and dislikes, and what he or she understands and doesn’t understand.

When the participant seems to be finished, ask the participant:

1. You have received and reviewed this disclosure. What would you do next? (If participant says they would get another disclosure, PROBE: Why? How many, etc? From a different lender or the same? If participant does not say, do not ask them if they would get information on another loan.)

2. What do you think is the purpose of this disclosure?

3. Remember that you want to borrow $315,000 to buy a house. Would you be interested in this loan? (PROBE: Why or why not?)

4. Can you explain to me what about this disclosure helped you to decide?

5. On a scale of 1-5 with 5 being very risky and 1 being a little risky, how would you rate this loan? Can you tell me why you rated it the way you did?

6. After looking at this disclosure, are there any additional questions you would have for your lender/ agent?

Task Two: Comparison with 2 similar offers (10 minutes)

[This task involves a disclosure for the same loan product with a pricing feature changed]

Thank you for working with that disclosure. Now, let’s go comparison shopping. You go to another lender or broker and ask for a loan quote. You are given this disclosure. Please look at this disclosure and then tell me if you are more interested in this loan compared to the first one. You may look back at the first disclosure as well.

Give the participant time to read and then ask:

7. Which loan would you prefer, loan 1 or loan 2? (Restate credit union choice out loud for recording)

8. Why did you choose this loan over the other loan?

9. In your own words, can you summarize how you used the two disclosures to compare?

10. In terms of fees, which of these loans is more expensive? Why?

11. Do either of these loans require you to pay for your hazard insurance? [if only find on one: how much? If find on both: why do you think the amounts differ so much?]

12. Why do you need less cash to close on [B loan] than on [A loan]?

13. Did the information on the back page influence your decision? If yes, how did it influence your decision?

Task Three: Details Probe (30 minutes)

Thank you. Let’s look back at the first disclosure I showed you. I want to ask you a series of questions to see how clear or unclear the information on the disclosure is. You may look at it as much as you like. Remember that we are trying to understand how well this disclosure is working. If you find it difficult to answer the questions, it means the disclosure isn’t working well, and that is good information for us so we can improve it.

Page 1 questions for both designs

1. What type of loan is this loan? If participant doesn’t know or can’t answer, ask, how would you describe this loan to a friend or family member? PROBE: What does that mean to you?

2. How much will you have to pay each month for your loan? Where did you get that information? (Probe for whether they respond with monthly loan amount or monthly payment.)

3. What is included in the amount you pay each month for your loan? Where did you get that information?

4. If they mention “mortgage insurance,” ask them: can you tell me what it is?

5. Can the monthly payment change? Why can it change? Where did you get that information?

6. What do you think about this section labeled “Cautions”? Why? Does that term, caution, seem like the right word? Would you use a question here?

7. Does the disclosure indicate if you have to pay anything else per month? Where did you get that information?

8. What is the interest rate for this loan? Where did you get that information?

9. Can the interest rate change? (If yes, can you explain when it can change? How much can it change? Where did you get that information? If no, skip next question.)

10. What is the maximum the interest rate can go to? Where did you get that information?

11. If you pay this loan off early, would you have to pay a fee? If yes, how much would that fee be?

12. What is included in the AT CLOSING number? Where did you get the information?

13. In terms of your cash needed to close – how firm is this number? What can make this number change? How does this number relate to the information on the back page?

14. Looking in the “Projected Payments” section, you see a monthly payment amount. Can you explain why that number is different from the monthly loan payment?

15. How often will your payments change? Can you explain why your payments change each time?

16. Why do you think no number is given for your monthly taxes and insurance after the first payment box? Can these change over time? Where did you get that information? Why might they change?

17. FOR ADJUSTABLE RATE LOANS: why are different numbers given in Year 6 and in Years 7-30? Why are there different interest rates? Why do you think the disclosure is showing those different interest rates?

18. What is the “Annual Percentage Rate”? Where did you get that information?

19. How is “Annual Percentage Rate” different from the interest rate of your loan? Why do you think the disclosure shows the Annual Percentage Rate?

20. What do these 2 numbers in “In 5 Years” tell you? Why do you think the disclosure provides the In 5 Years information? How would you use this information?

Page 2, Version 1 and 2

Let’s spend some time on page 2. There’s a lot of information on this page, so we want to explore what is useful and what is less useful. So let’s get started.

If you notice, some of this document has sections marked with letters (A, B, C, D, and so on). Let’s look at each of these sections and I am basically going to ask you the same questions about each section.

21. Which of the numbers can change before closing? Could you point those out to me?

Is that surprising? Did you expect to have costs that could change? How would that impact your decision?

22. Many of these costs say “estimated” – what does that term mean to you in this context?

23. For V2: You see a lot of costs on this page. What do the two tabs on top tell you about these various costs? How is this information helpful?

24. In Section A, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

25. What services can you shop for? What does it mean to be able to “shop for a service”?How does your ability to shop for services affect the price you will pay?

26. When you look at the area about services that you can shop for – which of these do you think you might shop for and why? How would you do that?

27. In Section B, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

28. In Section C, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

29. In Section D, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

30. In Section E, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

31. In Section F, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

32. In Section G, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

33. For version 1 only: In Section H, what is this information about? Do you think this information is “nice to know” or “critical”? Why? (Moderator restate, “nice to know” or “critical.”)

34. Can your closing costs change before the closing? If no, how do you know? If yes, where did you get that information?

35. You have to pay for many of these costs. Does the lender pay for any of these costs? If yes, where do you find that information? Why do you think the lender would pay some of the closing costs?

36. Does the seller pay for any of these costs? If yes, where do you find that information? (only if yes to seller paying: Why do you think the seller would pay some of the closing costs?)

37. What do you think “Escrow for Future Bills/Reserves for Future Costs” means? Why do you have to pay these amounts?

FOR ADJUSTABLE RATE LOANS ONLY: Let’s look at the information below the table as well.

38. In the Adjustable Interest Rate Information, what is this table trying to tell you?

39. When can the interest rate first change?

40. How much can that change be?

41. Can you tell me what the interest rate can be after that change? (want them to turn back to page 1 and look at interest rate or the projected payment table).

42. Can the interest rate continue to change? By how much?

FOR ALL LOANS:

43. Let me remind you of our scenario. (Show them the scenario card again.) So now you are looking at this back page to decide whether you are interested in this loan. Here are 5 green dots. Can you put a dot next to the five most critical pieces of information you would look at and consider?

(Moderator: Repeat what they are marking so we get them on tape.)

44. Can you explain to me, in your own words, why you selected the information you did?

We expect to see Cash Needed to Close, Estimated Closing Costs, Closing Costs to be Financed, Dates, and one of the high costs. If these come up, ask if these numbers are also anywhere on the front.

45. How would you use this information to make a decision?

46. Here are 5 red dots. Would you please place these next to the five items on the back page that you think are the least important and could be eliminated from the disclosure?

47. Why are these not important to you?

48. When you look at the back page of the disclosure generally, how would you describe the level of detail? Is it about right, too much, not enough? Why?

49. [In addition to those you have already mentioned as we’ve discussed the disclosure] Are there any [other] terms that are unfamiliar to you or that you think most people won’t understand?

50. What do you think is good or bad about this loan based on this back page? What concerns you about this loan based on the information you see on this page?

We’re more than half way finished. Let’s go ahead and take a short break for you and so I can set up for the next part. You can go get a drink, or use the restroom and I will come get you from the waiting room in just a few minutes.

Break (5 minutes)

Task Four: Comparison with 3 offers (15 minutes)

Note to Moderator: Remind participant about the Scenario Card and read out loud.

Let’s assume you have gone to yet another lender. Please take a look at this disclosure. Give them time to look it over.

1. Now that you have looked at it, please compare it with the other 2 offers. When I say compare, I want you to let me know the differences that you notice between the offers. (If participant asks what we mean say “differences between costs, features, loan terms, etc.”)

[Give a minute or two for the consumer to look/talk]

[When they are done:] Would you choose this loan over the other two? You may look back at the other two disclosures.

2. Which loan would you prefer 1, 2 or 3 (moderator restate name of credit union. Say choice out loud for recording)

Now, look at all 3 of these different loan disclosures. (Moderator sets out 1, 2, 3, in front of participant)

Please put these in order according to which you would most likely choose to the least likely to choose.

Moderator state order out loud for recording – referring to each by Credit Union Name. (Ask the participant to write the order on the top of the disclosure or do at the end of the session)

3. Why did you put them in that order?

4. Are there any of these loan products that you would not be interested in for yourself? (PROBE: Why? Which product do you think is riskiest? Why? What do you mean by “risk”?)

5. Did the information on the back page influence your decision? If yes, how did it influence your decision?

6. Now let’s look at those back pages specifically (ask the participant to turn all three over), is there any information here that would change your decision? Why or why not?

Task Five: Design Questions (15 minutes)

Note to Moderator: Place Loan 1 in front of participant.

You probably noticed that these three different loan estimates were all in roughly the same format. I’d like to ask you some specific questions about the design aspects. We’ll look at loan 1, the first disclosure you saw.

1. What is your overall impression of this form?

2. On a scale of 1 to 5, with 1 being ‘the worst form I have ever seen’ and 5 being ‘best form I have ever seen’, how would you rate this form?

Note to Moderator: Place Design 2 in front of participant.

Here is one other design. We’re almost finished.

This design is the same loan as you first worked with, but page 2 is quite different. Go ahead and spend some time looking at this other design and you can look back at Design 1 as well. (Give the participant about a minute to look at this.)

Okay, let me ask you some questions.

1. What is your overall impression of this second design?

2. What do you think about the difference in the level of detail? Is it more, the same, or less?

3. Which level of detail do you prefer? Why?

4. Here are the green dots again. Please mark the most important information. Why did you mark these? How would you use these pieces of information?

5. Pull out the one they did first: Why did you mark different (or the same) things in this one?

6. Looking at both designs, please find the section relating to title fees. How does the additional information about title fees in [Design 2] affect how you think about this loan?

Closing Remarks (5 minutes)

Thanks so much for sharing your thoughts and impressions with us today. Your feedback and suggestions have been very helpful. Do you have any questions I can try and answer? Are you interested in meeting any of the observers?

Ok. Thanks again for coming in today, you can head back to the front desk to collect your payment.

13