Stakeholder Input

Stakeholder Input for the Education and Administrative Reporting System (EARS)

1. Please provide specific comments and feedback regarding any challenges you encountered in gathering and reporting data for EARS and actions taken to resolve and or address these challenges. (Directions- Identify the section and item number from the EARS form when making comments. E.g. ”Direct Ed. # 3 Race and Ethnicity comment”)

General Comments:

1. Delaware Food Bank- It was not difficult to provide unduplicated counts; however, I think rather than asking for “FSNE contact” it would be more meaningful to ask for numbers of individual participating in single sessions and each of the series. It was also difficult to split males and females into those that participated in food stamps and those that did not.

2. Delaware (Ray Fitzgerald)- None

3. Delaware- I would like it if the EARS form could be typed onto directly…I had to send our data just as numbers on a page.

4. Virginia- We recognized several years ago the obstacles facing us when we attempted to prepare the report for you. Last year we saw the RED System at our MARO conference and started working with the computer engineer to design a data collection system that would fit our specific needs. Our GREEN System even has travel incorporated into the system. Our GREEN System is very similar to Maryland’s and we feel very confident today that we will be able to aggregate the required data for reporting purposes. Presently, our contract funder, VDSS, has us report to them on a quarterly basis much of the information being supplied through the computer system. We feel very fortunate to have been exposed to the Maryland program and had the money to purchase our own system built with us in mind.

5. West Virginia- We had particular challenges with #4 Direct Ed. Delivery Sites & #5 Direct Ed. Programming Format. We didn’t yet have our additional adult collection spreadsheet in place this year and the method of collection available to us didn’t work out as well as we hoped. These are the questions that prompted us to create a new List spreadsheet. Youth collection was in place and worked well.

• EARS Questions #1a and #1b-Partcipants

1. ASNNA- We are able to report the majority of our participants as actual unduplicated, direct contacts but contacts from older adults participating in group education at congregate meal sites are estimated. On the EARS form we must select whether we report actual or estimated contacts. Can we report both? If we can't, then we would have to report all education contacts as estimated. Correct? The other choice would be to put all older adults in as indirect contacts even though they participated in group, direct education.

2. ASNNA- Group sessions are tracked so individuals in groups are counted once, with multiple contacts (if in a series). There isn’t a way to compare individuals in indirect activities with those in direct activities, so we cannot assure an unduplicated count. Estimating even one number implies that the total is an estimate.

3. ASNNA- Grouping ages 5-17 is not very helpful in seeing what happens within school-age groups: elementary, middle, high school. Age appropriate delivery is very important.

4. New Jersey- Unable to specify if youth are on food stamps; CRS 5.03/SRS data does not indicate if youth are on food stamps; therefore applied % on food stamps from adult data was applied to youth data

5. New Jersey- Youth identified with Special Education in youth database does not provide a grade/age therefore the total number was added to the 5-17 Years and % on food stamps from adult data was applied

6. New Jersey- Adult participants that ‘age not given’: the % of adults from the actual numbers were multiplied by the ‘age not given’ and distributed/added to the actual numbers

7. New Jersey- FSNE Contacts by Age and Food Stamp Program Status: CRS 5.03/SRS does not provide Contacts by age and food stamp program status. The following calculations were used to obtain Estimated Counts of Participants: For Adults: Divided Total number of Contacts by Total Number of Series sets = average # of lessons; Multiple average # of lessons by number of participants from 1a. for the 18-59 and 60 years or more columns; For Youth: Number of Meetings divided by the Number of Groups = average # of lessons; Multiply average # of lessons by number of participants from 1a. for the <5 and 5-17 years columns

8. Pennsylvania- It was and continues to be somewhat of a challenge for our local partners to collect actual Food Stamp (SNAP) status on adults. We have provided client demographic surveys for partners to use and will continue to train and monitor this data collection element. In some cases, inability to collect SNAP status forces us to report what would otherwise be direct education contacts as indirect education

• EARS Question #2-Contacts

1. New Jersey- Number of FSNE Contacts by Gender: CRS 5.03/SRS does not provide Contacts by Gender. Total # of Adults (from 1a. – Column C & D) multiply by average # of adult lessons + Total # of Youth (from 1a. – Column A & B) multiply by average # of youth lessons divide by total number of youth and adult participants = average # of lessons for youth and adult; Multiply average number of youth and adult lessons by # of female participants; and then number of male participants.

• EARS Question #3- Race and Ethnicity:

1. ASNNA- Contractors had questions about identifying what race each Hispanic person is because information on a typical form at school might just identify “Hispanic” without a race. It is also difficult to gather data on race and ethnicity at direct education events that are point-of-sale (i.e. retail food demos) or in Food Assistance offices.

2. ASNNA- Refugees who are categorically eligible for FS do not fit any race/ethnicity category. Visual observation allows entry into race/ethnicity since asking refugees to put themselves into categories that do not fit is disrespectful.

3. New Jersey- Number of FSNE Participants by Race and Ethnicity: With the modifications imposed nearly a decade ago to separate race and ethnicity, it has become impossible to accurately collect this data for youth and adult program participants, nonetheless to report it.  We have developed a mathematical formula for generating reasonable estimates; however it requires an extremely complicated formula based on numerous data extrapolations and assumptions, to again, provide only reasonable estimates for these data.  

4. New Jersey- If our data is to communicate data from the public to our reporting authorities, a solution at the federal level would be to allow for a category of “Hispanic, Hispanics” (in addition to Hispanic, Whites; Hispanic, Asians; Hispanic Blacks, etc).  Until something of this nature is done, these data will remain speculative, as a fairly large segment of the Hispanic portion of the population we serve does not consider the current data collection categorization scheme adequate to describing “who they are”; therefore, despite a great deal of effort on our part, in both data collection and extrapolative analyses, we are left to reporting reasonable estimates based on speculation.

5. Delaware Food Bank-Feedback from clients- Many participants who check off Hispanic, consider themselves Hispanic: not black, white, etc and do not check off the race section.

6. Pennsylvania- Our reporting system does not record data on individual participants, rather on education interventions. Thus we can’t determine exact combinations of ethnicity by each racial category, nor can we determine exact numbers for all EARS multiple race categories. To collect Hispanic ethnicity with race we created combination race/ethnic indicators for racial categories most likely to include Hispanic participants, i.e. White (and) Hispanic; White (and) non-Hispanic; Black (and) Hispanic; Black (and) non-Hispanic. By doing this we’ll be able to report Hispanic numbers for the while and black race categories, but not for other race categories. Our multiple race participants will all be reported under #10 (All others reporting more than one race).

• EARS Question #4- SNAP-Ed Delivery Sites by Type of Setting:

No comments

• EARS Question #5- Programming Format:

1. New Jersey- Direct Education: # 5. Direct Education Programming Format: Obtain data from each county program on number of number of groups for each lesson series and the lesson time range for each series.

2. D.C.- Clarify what “% delivered by interactive media” means. Is this amount of time spent in a class on things like video, power point presentations or a kiosk?

• EARS Question #6- Primary Content of Direct Education:

No comments

• EARS Question #7- Description of All Social Marketing Campaigns:

No comments

• EARS Question #8-Indirect Education:

1. New Jersey- Distribution of Nutrition Education Information: Obtain data from each county program.

• EARS Question #9- Expenditures by Sources of Funding

No comments

• EARS Question #10- Expenditures by Category of Spending

No comments

2. Does FNS need to provide additional EARS training or resources? If yes, in what specific areas? What additional EARS resources would you like FNS to provide?

1. ASNNA- It would be beneficial for FNS to give guidance to state SNAP program administrators and implementing agencies regarding intent for use of the data. If we know the intent of the data, we can make better decisions regarding data collection processes.

2. New Jersey—No

3. Delaware- Tools to track and collect data would be helpful.

4. Delaware- I believe the training I received was adequate.

5. Delaware University- I did not receive any training. I would be interested in attending a general training session if one was available.

6. Delaware Food Bank- I would like to have an EARS report form available online to record my data and send it electronically to our state program manager

7. Delaware (Ray Fitzgerald)- FNS staff members were very helpful and responsive to requests for assistance.

8. D.C.- No. However, we will probably have specific questions later in the year as we begin collecting and reporting the data.

9. Pennsylvania- For question #4 “Delivery Sites by Type of Location” the instructions on the EARS form indicate that locations should be categorized according to their primary general purpose, but prior guidance suggested that they be categorized according to their primary SNAP-Ed purpose.  As an example, if SNAP-Ed is provided to clients of a food pantry that’s located in a church, the setting type would be “church” under the first definition, but “emergency food assistance site” under the second.  Additional guidance/clarification is needed.

10. Virginia- The computer programmer has adopted our GREEN System to work with your final EARS report. As long as that report stays the same we feel comfortable that we will not need further training. If changes occur training may not be necessary but we will have to re-program our data aggregation system to fit the EARS requirements.

11. West Virginia- No

3. What, if any changes did you make in your IT system or manual data collection procedures for EARS in FY 2008? What, if any costs did you incur for these changes? (Include staff, contracts, software, training, etc.).

1. ASNNA- We have been collecting this information for the past few years in preparation for EARS. The only costs incurred would be staff time to collect the information and train contractors. There may be additional staff time costs at the local level if they are required to take attendance at meal sites, etc. It depends on whether USDA will allow reasonable estimates. We could potentially lose participation (and cost-share) if we insist that names be collected as part of group education, etc.

2. ASNNA- We considered purchasing a Web-based program but the cost was prohibitive for our program. We were also concerned about adding more administrative costs to the program by requiring educators to enter data on-line. The subcontractor has the responsibility to determine the best system for managing their data collection at the local level as long as they report the required information at the end of the year.

3. New Jersey- Distribution of survey forms via email to counties

4. Delaware- It did not cost us anything, but we did make sure that we were able to estimate as closely as possible the racial and ethnic makeup of our indirect contacts

5. Delaware University- We made no changes in our IT system, but we had to collect some data by hand. This required staff time.

6. Delaware Food Bank- I created my own spread sheets to collect data for the report. In an effort to save time, it would be helpful to have these forms available since I am the only one providing, the educational classes, and collecting, tallying and reporting data.

7. Delaware (Ray Fitzgerald) - The error comments when data is not entered correctly does not give clear instructions on what is incorrect or how the entry should be corrected.

8. D.C.- We have revised a data collection tool to determine if clients are on Food Stamps

9. Pennsylvania- We implemented IT changes prior to FY2008 to collect some required EARS elements. In FY08 we modified our IT system to include the remaining components, mainly collection of demographics and SNAP status for all direct education contacts and unduplicated participants. We also added basic report compilation capabilities to compile EARS elements for annual reporting purposes. Estimated cost = $73,309

10. Virginia- We had the GREEN System built to accommodate the report. We spent approximately $20,000 to contract with Image Multimedia, Inc.

11. West Virginia- In addition to NEERS5 that we have already been using for both EFNEP & SNAP-Ed, we added an additional access database for youth data collection (created in-house), an additional excel spreadsheet for adult data collection, and refined our timesheets to collaborate the data on direct & direct contacts per week. Instructors use each of these tools to collect their data and then submit it for compilation at the state level. Training involved developing step by step instruction guidelines, one training session at Spring In-Service and another at Fall In-Service, then follow up training for new employees. One staff member is available at the state level for tech support calls throughout the year & data is submitted to the state level quarterly for review.

No cost was incurred other than time of staff to create the additional collection modules, educate the staff & each of them enter the additional data. However, staff time was significant.

4. Do you plan to make any changes in the next FY? If yes, describe type of changes planned?

1. ASNNA- Depending on our experience collecting 2009 data, we may or may not make changes. NOTE: For the second year, we are using a Web-based data collection tool to measure intensity and quality of education provided by subcontractors (only for randomly selected third grades right now). We have intended to collect that information for only two years, but may continue longer if we can afford the services

2. New Jersey- Develop SAS program to potentially obtain more specific data from SRS database.

3. Delaware- We do not plan any changes in the next FY

4. Delaware University- No

5. Delaware Food Bank- No

6. Delaware (Ray Fitzgerald)- No

7. D.C. - None we can think of at this particular time.

8. Pennsylvania- improved user input screens and data validation at point of data entry to improve quality; expansion of report compilation elements; expanded training modalities for local partners to improve accuracy of data collected.

9. Virginia- We plan on submitting some delivery initiative amendments in our 2010 Plan. The state agency has requested these initiatives be developed and implemented. We will submit amendments to address them.

10. West Virginia- We have continued to tweak our reporting for adults this year in an effort to have the best possible data by the end of the year. We do not foresee having to further modify data collection for next FY, but we might still do so if an opportunity to improve presents itself.

5. Will you report actual unduplicated data in your EARS in the upcoming fiscal year? If no, identify the barriers to doing so.

1. ASNNA- We will have to use an estimate for older adults congregate meal sites. We asked our subcontractors to do their best to provide actual, unduplicated counts. In the smaller, rural sites this may not be a big problem. But it will be very difficult, if not impossible, at the larger urban sites. Unlike a school classroom, participants are not required to be there. We will ask subcontractors to participate in a conference call this summer to get feedback on the how the data was collected. If our method(s) for estimating is not acceptable to USDA we may have to report all older adults as indirect contacts. This would create an inaccurate picture.

2. ASNNA- We are able to report the majority of our participants as actual unduplicated, direct contacts. But contacts from older adults participating in group education at congregate meal sites are estimated. On the EARS form we must select whether we report actual or estimated contacts. Can we report both? If we can't, then we would have to report all education contacts as estimated. Correct? The other choice would be to put all older adults in as indirect contacts even though they participated in group, direct education.

3. ASNNA- Using school level demographic data applied to the classroom level will always result in an estimate of age/gender and race/ethnicity, as well as the % of free lunch participants in a classroom. The effort to maintain data for an actual unduplicated count is time consuming and expensive. Many hours would be required to maintain actual counts. The hours of educators are better spent in delivering education. Community locations can be more accurately reported. There is a concern for maintaining demographic data of program participants with confidentiality, privacy, and security issues.

4. ASNNA- Social Marketing asks for participants at 130% and 185% of poverty while direct education asks for 130% and all others (no segmentation of low -income). This seems inconsistent.

5. New Jersey- Actual Unduplicated Data for 1a; 2a. No, for other tables due to limitations of CRS 5.03/SRS data filters unless the SAS program can obtain actual data.

6. Delaware- Yes, we plan to report both actual and unduplicated data in the upcoming fiscal year.

7. Delaware University- We will report unduplicated data.

8. Delaware Food Bank- Yes. The spread sheets I use for each class allows me to do this, though it is time consuming

9. Delaware (Ray Fitzgerald)- Not sure

10. D.C. - Yes, actual unduplicated data will be reported for all categories except for Food stamp participation. Food Stamp participation data will be estimated.

Barriers:

• We feel it may be uncomfortable for staff to ask and clients to acknowledge whether they participate in the Food Stamp program when working with large groups.

• Therefore, we have decided to develop a small survey with 1-5 questions and one question will be about Food Stamp participation.

• In large groups it is very time consuming to have each participant fill in a survey to ascertain if they participate in Food Stamp.

• Due to our privacy concerns we are depending on each client to fill out the survey, some clients may be at a low reading level.

11. Pennsylvania- Yes, we will report actual unduplicated data to the best of our ability. In some counties, where there is more than one service provider, assurance of accurate counts of unduplicated adult participants is very difficult.

12. Virginia- Virginia will be able to report this data to you due to the new GREEN System. We are thrilled to be able to accomplish this task!

13. West Virginia- Our counts of participants will continue to be actual. Our youth direct education contact counts will still be estimated, not actual, as we do not take attendance at every meeting and track each participant throughout the course.

We will report unduplicated direct education participant and contact counts with a small margin of error as it is possible that occasional duplication may occur. For example multiple instructors in a county could count the same person in different class settings like classroom & then again at camp that summer