Attachment B13 - PMHCA Program Implementation Survey_Pilot Test Results 12-13-19

Attachment B13 – PMHCA Program Implementation Survey

Pilot Test Results

Health Resources and Services Administration (HRSA) Evaluation of the Maternal and Child Health Bureau (MCHB) Pediatric Mental Health Care Access (PMHCA) and Screening and Treatment for Maternal Depression and Related Behavioral Disorders (MDRBD) Programs Project

Note: The instruments included in this OMB package have been revised and incorporate

the pilot test results. The question numbers correspond with the instruments pilot tested.

Feedback	Changes Made to Instrument
Time to complete: 12 minutes 14 minutes	Projected time to complete the survey has been changed to 20 minutes.
General Comment: I think the questions are clearly written. In some cases, I think concepts may need to be defined.	Reviewed all instruments for language related to concepts and added clarifying language, as appropriate.
General Comment: Overall the questions were clear and in a logical format. The choices provided were also clear and easy to understand. Prior to the questions, on page 2 of the survey, the final paragraph discusses confidentiality. As I read it from the viewpoint of a Project Director, I wondered if HRSA would see my responses or only JBS. Perhaps that could be made clearer.	Added clarifying language.
Format: The directions about when to skip questions are very clear. The questions have been broken down into manageable pieces.	N/A—no changes required.
Format: The format was clear, logical and easy to follow. Providing responses in the charts worked well. My only concern is how answers will be marked. Will there be a fillable box that will be marked with an X when the respondent clicks an answer? Or will the format remain as it is now with the little circles in front of each response? When I completed the survey putting an X in front of the little circles, it messed up the formatting a bit and didn’t look clean.	No change made. Responses will show as filled when a response is selected.
Question 4: I think the table format works well. I think the breakdown in number and FTE is clear.	N/A—no changes required.
Question 4: This question and related chart were a little confusing to me. If I have two psychiatrists, one at 1 FTE and one at .5 FTE can I insert two lines to explain this? Perhaps providing an example in the chart would help.	An example has been added to the question for clarification.
Questions 11 and 12: For question 11 - you may want to spell out CME/CE credits. I am wondering if you may want to provide some examples for each of the options listed to help people think through the facilitators. I also wonder if it may be helpful to have an open-ended response category for folks to write down their own experience.	Added clarifying language for CME/CE to facilitate readability. Open-ended response option is already included.
Question 21: I'm not sure if brochures and briefs should be grouped together or if posters and infographics should be grouped together. Also, do you want to find out which of the promotional materials was used with which audience.	Response options requested by HRSA, as such no change made.
Question 25: You may want to define "formal plan" Are you actually getting at whether the plan has been written down or something else?	The word “formal” has been removed to avoid any confusion.
Question 29: The wording of this question was confusing to me and I had to reread it 2-3 times to understand what was being asked. I think it might be clearer if you simply eliminate the word “needs” from the question. “Will your PMHCA program require any of the following evaluation capacity-building support or technical assistance in the upcoming year?”	This change has been made to facilitate readability.

1