Project Outcome Assesment Survey (POAS)

OMB Control No: 0970-0379

Expiration date: 07/31/2019

Project Outcome Assessment Survey

THE PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) Public reporting burden for this collection of information is estimated to average 6 hours per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information.

Identifying Information (Pre-Filled by Federal Staff)

Evaluator’s Name (First), (Last):

Other ANA Staff/Contractors Present at the Site Visit:

Evaluation Region: [DROP DOWN] East, West, Pacific, Alaska

State: [DROP DOWN] AK, AL, AR, AS, AZ, CA, CO, CT, DC, DE, FL, GA, GU, HI, IA, ID, IL, IN, KS, KY, LA, MA, MD, ME, MI, MN, MO, MP, MS, MT, NC, ND, NE, NH, NJ, NM, NV, NY, OH, OK, OR, PA, RI, SC, SD, TN, TX, UT, VA, VT, WA, WI, WV, WY

Date of Evaluation:

Grantee Information (Pre-Filled by Federal Staff)

Grantee Name:

Project Title:

Grant Number:

Updated Grantee Contact Info (if Grantee Contact Info is not Correct in GrantSolutions):

Geographic Designation:

Number of Program Specialists during Project Period:

Grant Category: [DROP DOWN MENU] Social Development, Economic Development, Governance, Strengthening Families, Environmental Regulatory Enhancement, Language Preservation, Esther Martinez, AFI (partnership), ADD (partnership)

Grant Sub-Category:

Original Project Duration:

Total Federal Funding Amount:

Grant History (Pre-Filled by Federal Staff)

Received TA: OTA o ETA o (Check all that apply)

Yes o No o

Comments?

Received Supplements:

Yes o No o

Comments?

Received Carryovers:

Yes o No o

Comments?

Received an NCE/LCE:

Yes o No o

Comments?

Pre-Visit Information (Pre-Filled by Federal Staff)

Background Information/General Comments

DPO Site Visit Information (if applicable)

1. Objective Work Plan (Pre-Filled by Federal Staff)

1.1 Objective 1

Relevant Year(s); o Yr1 o Yr2 o Yr3 o Yr4 o Yr5

Percent Complete: 0, 25, 50, 75, 100, 100+%

Objective

Results (Outputs)/Benefits (Outcomes) Expected:

Criteria for Evaluating Results and Benefits Expected:

(Asked Grantee during On-site Interview)

Deliverables (Developer note: add tags for deliverable type):

Product	Documentation Reviewed/Comments

Comments

2. Results (Outputs) and Benefits (Outcomes)

(Pre-Filled by Federal Staff from application and confirm on-site)

2.1 Community the project was designed to serve ___

2.1.1 Number of people the project was designed to serve_____

2.1.2. Number of people the project actually served_____

Indicator	Baseline	Year 1	Year 2	Year 3	3-Year End of Project	Means of Measurement

2.2 Describe how your indicator was achieved throughout the project.

2.3 Was the indicator from the application the best measurement to capture the outcomes of your project?

2.3.1 If not how would you have identified or revised the indicator?

2.4 Name up to three achievements you are most proud of and would want to share about your project?

1)

2)

3)

2.5 What project outcomes would you like to sustain after ANA funds end, and what is needed to sustain such outcomes?

2.6 Now that you are at the end of your project, is your Three-Year End of Project Target still achievable realistic?

If not, how would you revise?

2.7 As a result of this project is/will it be a(n):

• New Initiative

• Continuing Program

• Merged into an existing program

• Intended to End (e.g. Manual/Regulation Development)

2.8 What factors will contribute to the continued sustainability of this project? (evaluator instruction: use a one or two word term for the “factor”)

Factors	Comments

2.9 What obstacles are foreseen to project sustainability over the next 5 years?

2.10 Was any infrastructure created?

2.11 FOR NON-LANGUAGE PROJECTS: Were language activities incorporated into the project in any way?

If so, how?

3. Project Development

3.1 How was the project developed? Please describe how the community was involved in the development process.

3.2 Was the project’s target population involved during project planning prior to the grant award?

3.3 Is there anything you would have done differently in planning your project?

3.4 What marketing or outreach methods did you use to bring attention to your project? (Check any that apply.)

o Brochures and Pamphlets

o Email

o Festivals and Community Events

o Networking (with Other Tribes, Agencies, Organizations, Businesses)

o Newspaper Advertisements

o Newspaper Coverage

o Onsite Community Outreach (knock on doors)

o Phone Calls

o Posting Information in Public Spaces

o Radio (includes PSAs and other programming)

o Regular Mail

o Social Media (such as Facebook, Twitter, Youtube, etc.);

o Tribal/Community Meetings

o Tribal/Organizational Publications or Newsletters

o TV (includes PSAs and other programming)

o Website

o Word of Mouth

3.5 In terms of marketing/outreach, what would you have done differently?

3.6 What were the best practices in marketing to your specific population?

Target Populations	Best Practice

3.7 How was the project received by the community?

3.8 Was there opposition to the project?

Yes o No o

Comments?

3.9 How did you keep the community engaged or updated throughout the project?

3.10 How did you encourage project participants to maintain their participation in the project? Were there external factors that also helped?

3.11 How did you facilitate multigenerational activities?

3.11.1 What were the results of these activities among the participants?

3.11.2 What are some best practices in facilitating multigenerational activities?

4. Project Challenges

4.1 Did you have any challenges that affected your ability to implement the project?

Rank	Challenge	Ability to Overcome Challenge?	Comments
	Staff turnover	
	Late start due to ANA notification	
	Late start due to hiring delays	
	Scope too ambitious	
	Geographic isolation/travel issues	
	Lack of expertise	
	Challenges with ANA processes	
	Challenges with OGM processes	
	Underestimated project cost	
	Underestimated personnel needs	
	Partnership fell through	
	Lack of community support (planning)	
	Lack of community support (implementation)	
	Insufficient Community Infrastructure	
	Personal or Community Loss/Hardship	
	Natural Disaster	
	Lack of Organizational Capacity	
	Other	

5. Project Economic Benefits

5.1 Did this project support native-owned businesses?

Yes o No o N/A o

5.1.1 If so, how many?

5.1.2 Of the Native-owned businesses the project supported, what were the top 3 types of businesses by dollar amount?

Business Name	Type (eval instruction: use a one or two work descriptor)

5.2 Were any businesses created by the project?

Yes o No o N/A o

Business Name	Type (eval instruction: use a one or two work descriptor)	Ownership Type Native, Women, Vet

5.2.1 Was income generated by businesses (developer note: only show if yes is selected in 5.2)

5.2.2 List income generated: How was income generated?

5.3 Were any businesses expanded?

Yes o No o N/A o

5.3.1 If yes, how were they expanded? (developer note: only show if yes)

5.4 Complete for any person employed as a result of the project (do not include project staff).

Name of Individual	Job Title	Industry	Employment Status Prior to ANA Project	Annual Compensation Prior to ANA Project	Native American? (Yes/No)

FTE Type (Created, Obtained, or Project-Funded)	Date Employment Began	Date Employment Ended (if applicable)	Total Weeks Employed	Average # of Hours per Week	Total Hours Worked	End-of-Project Employment Status	End-of-Project Annual Compensation

 

6. Partnerships

6.1 What role did partnerships play in your project?

[DROP DOWN]Integral, Supportive, Minor, Needed more partners

Comment:

7. Program Specific (Grantees will only be asked one set of questions from this section)

ERE

7.1 What was the project type:

Forest Pollution Botany

Water Marine Wildlife

Air Land Other:

What was the main focus of your project? (Check all that apply)

Data collection to establish baseline Environmental Education

Data collection to monitor condition or trend Staff Training/Development

Develop/implement management systems Obtain additional federal funding

(GIS, enforcement) Obtain EPA IGAP Funding

Develop Regulations Other

Develop Tribal Environmental Protection Act

7.2 Did you use EPA or other federal funds to meet your non-federal share for this grant?

Yes: EPA

Yes: Other

No

Was not aware this was allowable

7.3 Does your Tribe/organization receive funding from EPA, US Forest Service, or US Fish and Wildlife?

EPA

US Forest Service

US Fish and Wildlife Service

No

Unknown

7.3.1 If yes, what is it used for? How does your ANA ERE grant fit in with this funding?

Language

7.1. Please provide the:

7.1.1 Language Family:

7.1.2 Language:

7.1.3 Dialect:

7.2 Can you rate on a scale of 1 to 5, language use in the following spaces:

7.2.1 Community

7.2.2 In the home

7.2.3 School

7.2.4 Tribal Government

7.3 Has there been any space/place that language use has increased since the grant began? Where? How has it increased?

7.4 Do you have a community or tribal language board? What is the project’s connection with the community or tribal language board?

7.5 How did you determine the means of measurement used to measure the increase in proficiency and/or fluency? (note to evaluator: see ADR #s 6-9)

7.6 Status of language prior to the project (if applicable, what means of measurement used)

7.7 Describe the current status of the language/activities (evaluator note: elaborate to note that significant changes are not going to occur within a discrete timeframe of a project, but what has improved over the course of the project)

7.8 Were there any external positive effects have been noted in the community outside of the project?

SEEDS

7.1 Do you know of (a) project participant(s) who have experienced economic hardship and were better equipped to overcome that situation as a result of this project?

Yes No N/A

7.1.1 If yes, how were they more successful?

7.2 Was training an aspect of this project, how many of the trainees are employed?

Yes No

7.2.1 Was participant employment a result of the training received from the project?

7.2.2 As a result of training, did individuals receive certifications that support their employment?

Yes No N/A

If so, how many?

7.3 Are there other supportive services provided by the project to transition participants to employment?

7.4 Business revenue/profit generated annually:

		Revenue	Profit
Year 1
Year 2
Year 3
Year 4
Year 5

7.5 Which organizations exist in your community: (developer note: list as table)

Local Chamber of Commerce Bank

Small Business Association Credit Union

Tribal Incorporation Community Development Financial Institution

7.5.1 Of the businesses created, were any a member of those organizations?

Local Chamber of Commerce Bank

Small Business Association Credit Union

Tribal Incorporation Community Development Financial Institute

7.6 Is there other government funding contributing to business creation/development? If so, which ones?

NABI

7.1 How did you adapt to community and cultural needs throughout the program?

7.2 How did you monitor for quality and effectiveness of financial education trainings? How did you monitor the relevance to participants and assess what they learned?

7.3 Has this project allowed your community to improve or expand access to financial services?

Yes No

7.4 Were there limitations on infrastructure in the local community that affected the participants’ abilities to purchase an asset?

Yes No

7.4.1 If yes, please describe limitations and how they were or were not overcome?

7.5 What are the most common reasons why individuals choose to not enroll in the program or finish the program?

7.6 What type(s) of training(s)/topic(s) and frequency was the most successful?

7.7 Did this NABI project prepare you to apply for other IDA funding?

Yes No

7.8 Do you intend to continue to offer IDAs? If yes, where will you obtain funding to continue?

Yes No

7.9 How many participants were successful in obtaining their final goal?

7.10 Participant Information

Number of men:

Number of women:

Number 16-24

Number 25+

8. Staffing

8.1 How many Principal Investigators/Project Directors served on this project?

8.1.2 If there was turn-over, did the organization have a succession plan to transfer knowledge of the project and completed activities to the next PI/PD?

Yes o No o

Comments:

8.2 Post Award Training

8.2.1 Did the Project Director/Principal Investigator receive post-award training?

Yes o No o

If no, why not?

8.2.2 Did the financial staff member receive post-award training?

Yes o No o

If no, why not?

8.3 Does your organization have an IT/Data Management Department?

Yes o No o

ANA wants to support data management among its grantees.

4. Among staff that work on this project, please indicate the highest level of knowledge in the following areas:

(Scale: Extremely Knowledgeable, Moderately knowledgeable, slightly knowledgeable, use external expertise, not part of regular project business)

1. Performance Indicators

2. Descriptive statistics

3. Program Evaluation

4. Data visualization

5. Data programming

6. Data management (and what software used)

4. How would you characterize the availability of financial resources, including personnel availability, for data usage and management:

(Scale: 1-10 from Extremely inadequate to Extremely Adequate)

8.6 Through this project, has your organization/tribe increased their ability to collect, manage, or analyze data in any way?

7. What are the primary barriers to data development and analysis that limit the ability of your project to use data to inform key policy and programmatic issues?

8.8 How many staff are employed by the tribe or organization? 1-10, 11-20, 21-50, 50-100, 100+

8.8.1 Did you have all of the staff available to carry out your individual project?

Yes o No o

9. Beneficiaries

(Evaluator: use this section to capture information collected during interaction with beneficiaries/community members/partners including indirectly. If necessary ask grantee to identify beneficiaries and discuss impact.)

Beneficiary Name	Beneficiary Type: Tribal Leader; Parent; Student; Teacher; Project Staff; Elders; Community Member (developer note: look at ADR categories)	Individual or Group
Beneficiary’s impact from project participation:
What made you want to participate in the project?
Were there any challenges you faced in participating in the project?
What was your favorite aspect of the project?
What have you seen change in your life or the community related to the project?

10. Project Specific

Question for Grantee (Evaluator: Include any project-specific questions for the grantee and community members here. These should be developed before the site visit and based upon your review of the grant file).

Question	Answer