Final - 10_Phone Interviews Y3

Year 3 Phone Interviews: Treatment PHAs

MTW Evaluation: Landlord Incentives

Last Modified: October 5, 2022

Introduction to Protocol for Reviewers

The PHA interview protocol is a guide for the interviewer to follow a particular approach to interviewing. Interviews will be conversational; the interviewer and interviewee may speak about these topics out-of-order, or the interviewee may address multiple questions in a single response. The interviewer guides the conversation using questions listed in the protocol, following up with probes (shown in bullets) as needed to gather more detail.

In Year 3, we anticipate interviewing up to 3 key staff at each PHA, such as the Executive Director, Manager of the HCV Program, MTW Coordinator, Landlord Liaison, and Housing Specialists. These interviews will be held by phone. On average, we expect interviews to be approximately 45 minutes. Interviews will vary in length according to the information the interviewee has to share; some may be longer than 45 minutes if the interviewee has a lot of information to share, while others may be less than 45 if the interviewee has less experience with the program.

The estimated time for each section currently adds up to more than 45 minutes. Interviewers will be trained to make trade-offs and prioritize some questions over others during the interview in order to manage the interview timing. Interviewers will also have access to information about each PHA prior to the interview, so they can tailor the questions as needed to make the questions more relevant and targeted (for example, asking about a specific incentive or characteristic of the local rental market).

The following interview protocol will be piloted and adjusted accordingly to eliminate any questions which do not yield useful information. Interviewers will be trained to keep a core focus on the goals of the interview and manage the interview time accordingly.

Paperwork Reduction Act – Burden Statement

Public Reporting Burden for this information collection is estimated to average 40 minutes per respondent, including time to review the information in the Study Overview and Introduction. The MTW Evaluation: Landlord Incentives is a study of the implementation and impacts of providing selected incentives to landlords to participate in the Housing Choice Voucher (HCV) program. This information is being collected to document implementation of PHA activities that could affect landlord participation in the HCV program. PHAs in the MTW Landlord Incentives cohort are required to respond. The information requested is protected and held confidential in accordance with 5 U.S.C. § 552a (Privacy Act of 1974) and OMB Circular No. A-130.

Note to Interviewer

Note to Interviewer: Before the interview, you should carefully review the Site Summary, which collates information from the MTW Supplement, web survey responses, previous interviews, and admin data. Use this information to tailor the questions below accordingly. You should pay close attention to the challenges that they identify, the incentives that they plan to implement, and the way their incentives are targeted or tailored to their market. You should also make note of any Agency-Specific Waivers listed in the MTW Supplement that may lead to variations in implementation of incentives (for example, agencies may seek a waiver to include special programs like HUD-VASH or Mainstream Vouchers in their MTW program).

Interviews will be held with PHA staff potentially including the executive director, director of the HCV program or rental assistance programs, MTW Coordinator, housing specialists, or landlord liaisons. One-on-one interviews are strongly preferred, but group interviews (ideally with staff with similar roles) can be conducted if there are scheduling challenges that require it.

Depending on their role and their length of time with the organization, interviewees may not be able to comment on all of the questions or sections included here. For example, the executive director and program director will likely be able to answer all these questions; front-line staff may not be able to answer questions about program management. If the interviewee indicates that they are not the ‘right person’ to speak about a specific topic, move on to the next section. You should focus on the questions that are most likely to be relevant for the interviewee based on their role with the agency.

Study Overview & Introduction

Thank you very much for taking the time to speak with me today. HUD is collaborating with Abt Associates to understand the impact of landlord incentives. The purpose of the study is to understand how PHAs are implementing landlord incentives, and how these incentives affect program outcomes. We’re interviewing all the PHAs that are part of the Landlord Incentives Cohort of MTW.

These questions have been reviewed by the Office of Management and Budget [OMB#]. When your agency received MTW designation, they also agreed to participate in the evaluation – so, your participation in this interview today is required. There may be some questions that you are unable to answer or that are more appropriate for other staff. If you are unable to answer a question, just let me know and we’ll move on to the next one. We expect this interview to take about one hour.

We will be taking notes and recording this interview. The notes, recording, and transcript of this interview will not be shared outside of the study team. The information you share with us will be used for research purposes only, not for any audit or compliance purposes. The study reports will summarize the findings from these interviews and from other data collection activities. We won’t name specific individuals in our reports and publications, but we might name the PHAs who participated in the study, so it’s possible that someone who is familiar with the PHA would be able to tell who was interviewed. 

Do you have any questions before we begin? [pause for questions]

Do you consent to participate in this interview? [get verbal consent before proceeding]

Is it OK if I turn on the recorder now? [get verbal consent before proceeding]

The tape recorder is now on. This is [interviewer name] speaking with [interviewee name] from [PHA name] on [date].

Interview Questions

Program context and background

1. If interviewee was not included in baseline interviews: Can you tell me a bit more about your role with the HCV program? How long have you been in this role?

2. If interviewee was included in baseline interviews: Last time we spoke, your role was [describe]. Has that changed at all since the last time we spoke?

3. Since [year of baseline interview] have there been any changes in the community that could affect the HCV program? What were they? How did they affect the program? (e.g. changes in rental market, new rental assistance programs, legislation or policy changes)

   • Have these changes had any effect on landlords or their engagement with the program? If so, in what ways?

4. Since [year of baseline interview], have there been any changes at the PHA that may have affected the HCV program, other than the landlord incentives? What were they? (e.g., sought or received a different HUD waiver, hired a new landlord liaison, leadership changes, new programs, changes in funding or capacity)

   • Have these changes had any effect on landlords or their engagement with the program? If so, in what ways?

5. In general, how are things going with the implementation of the landlord incentives? What were the highlights of the last year? What were the challenges?

Implementation of incentives

I have a list of the landlord incentives you’re planning to implement here from your most recent MTW Supplement. I’d like to walk through each of these and ask you for a little more detail.

6. First, I see here that you are currently offering [list incentives]. Can you tell me if any of these incentives have changed since you submitted the latest Supplement? Is there anything missing from this list, or anything that I mentioned that you’re no longer offering?

[Repeat for each incentive. For new or discontinued incentives, skip to Q12/Q13.]

7. I’d like to learn a little more about how you are implementing [incentive]. Can you describe what steps you have taken to implement this activity in the past year?

8. Tell me about what has been working well in implementing this incentive.

9. Tell me more about any challenges you have had in implementing this incentive.

   • If you have been able to, how have you dealt with those challenges?

   • If you have not been able to, what is preventing you from dealing with those challenges?

10. Have you made any changes to how you are currently implementing this activity, compared to what you initially planned?

    • Have there been any changes to the design of the incentive (eligibility, dollar value, frequency, etc.) If so, what led to that change?

    • Have there been any changes to the timeline for implementation? If so, what led to that change?

    • Do you anticipate making any other programmatic changes to this activity in the future?

11. Have you needed to make any internal changes at the PHA to implement this incentive? (e.g. staffing, policies or procedures) If so, can you tell me more about these changes?

12. If the PHA has added new activities since previous supplement: Looking at your most recent supplement compared to the one before, it looks like the PHA now plans to implement [new activity]. Can you tell me more about that incentive?

    • What made you decide to add this incentive?

    • Which landlords/programs/areas will be eligible for this incentive? How did you decide how to target this incentive?

    • What is the timeframe for implementation?

    • Which staff will be involved? Will any partners be involved?

    • Do you anticipate any implications for program operations (e.g. staffing, costs, revenues)?

13. If the PHA has discontinued some activities: It looks like the PHA no longer plans to implement [incentive]. Can you tell me what factors went into that decision?

14. Are there any other changes to how your agency plans to use its MTW authority that we haven’t mentioned? If so, what are they?

Early changes from incentives

Now I’d like to ask a little more about any changes you’ve seen in the HCV program since implementing the incentives. These could be positive, negative, or neutral changes.

15. When you began offering the landlord incentives, what was the response from landlords? Have you noticed any changes in landlords’ interactions with the program so far?

    • What feedback have you heard from landlords about the incentives? How did you gather this feedback?

    • Did they have any questions or concerns about the incentives? If so, what were they? How did you address them?

16. What about tenants – how did they react to the new incentives? Have you noticed any changes in the experiences of tenants as a result of the new incentives?

    • What feedback have you heard from tenants? How did you gather this feedback?

    • Did they have any questions or concerns? If so, what were they? How did you address them?

17. Have you observed any changes in program costs after implementing these incentives? If so, can you tell me more about those changes?

18. Overall, do you think the landlord incentives are making a difference to the program so far? If so, in what ways? If not, can you say more about why you think it is not making a difference?

19. Have there been any unintended consequences of the landlord incentives so far? Can you tell me more about these consequences?

    • How have they affected the program or the experience of tenants and landlords?

    • How have you responded to these issues?

20. Of all the incentives you have implemented, which incentive(s) do you think have been the most effective? Tell me more about that.

21. How is the agency monitoring the implementation and impact of these incentives?

    • What outcome or output measures are you collecting about these incentives?

    • Have you seen any changes in those outcome or output measures since implementing the incentives?

    • Are you collecting any additional data or feedback on the incentives (e.g. from tenants or landlords)? If so, can you describe what data you are collecting and from whom?

    • How do you plan to use that data/feedback moving forward?

22. I know it’s still early, but reflecting on your experience so far, are there any ‘lessons learned’ that you would want share with other jurisdictions? Is there anything that you wish you had differently in designing and launching the incentives?

23. Do you have any other reflections you’d like to share about your experience with the landlord incentives so far?

Additional data questions – for staff who manage program data (HCV Program Manager, data specialist, or similar) only

Before we wrap up, I’d just like to confirm some of the information we gathered from your MTW Supplement and HUD administrative data. [Note that they may be unable to provide this information during the interview. If so, ask them to share information by email and follow up as needed.]

24. First, I want to confirm your agency’s success rate.

    • If success rate data is available from survey or admin data: I see that you reported [#] regular tenant-based voucher issuances between [DATE] and [DATE]. Of those, you reported that [#] successfully leased up within 180 days. Do those numbers seem correct to you? If not, do you think they are too high or low? What are the correct numbers?

    • If agency has missing or incomplete success rate data: We are looking for two key metrics: the number of regular tenant-based voucher issuances between [DATE] and [DATE], and the number of those issuances that resulted in a lease-up within 180 days. This excludes special program vouchers like HUD-VASH, Mainstream, or EHVs. Are you able to provide that information for our study?

25. For signing bonus only: I’d also like to confirm some data related to the signing bonus.

    • If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW supplement correctly. In the most recent fiscal year, you made [X] signing bonus payments in total, averaging [$X] per payment. Does that sound right?

    • If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of signing bonuses you’ve issued in the most recent fiscal year, and the total dollar value of those bonuses. Are you able to provide that information for our study?

26. For damage deposit only: I’d also like to confirm some data related to the damage reimbursement incentive.

    • If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] damage reimbursement payments in total, averaging [$X] per payment. Does that sound right?

    • If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of damage reimbursements you’ve issued in the most recent fiscal year, and the total dollar value of those payments. Are you able to provide that information for our study?

27. For front-end vacancy loss only: I’d also like to confirm some data related to the front-end vacancy loss payments. This is the vacancy loss payments that are made when the unit transitions from non-HCV to HCV.

    • If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] front-end vacancy loss payments in total, averaging [$X] per payment. Does that sound right?

    • If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of front-end vacancy loss payments you’ve issued in the most recent fiscal year, and the total dollar value of those payments. Are you able to provide that information for our study?

28. For regular vacancy loss only: I’d also like to confirm some data related to the regular vacancy loss payments. This is the vacancy loss payments that are made when the unit transitions from HCV to HCV.

    • If direct payment info is available in MTW Supplement: I’d just like to make sure we are interpreting the data from your MTW Supplement correctly. In the most recent fiscal year, you made [X] vacancy loss payments in total, averaging [$X] per payment. Does that sound right?

    • If direct payment info is missing or incomplete: We are looking for two key metrics related to direct payment incentives: the number of regular vacancy loss payments you’ve issued in the most recent fiscal year, and the total dollar value of those payments. Are you able to provide that information for our study?

29. For pre-qualifying inspections only: Since implementing your MTW program, how many pre-qualifying inspections has your agency completed to date? Of those, how many have not resulted in lease-up?

30. For initial inspection waivers only: Since implementing your MTW program, how many initial inspections have you waived to date?

31. If any other key data points from Supplement/admin data are missing or unclear: I noticed that we are also missing [information] from our information on your PHA. Could you provide that information for our study?

Thanks for your time today. Do you have any questions for me?

8