This report contains the Module 1
Content for the following comments:
1,2,3,4,5,6,8,9,11,12,13,14,16,17,18,19,20,21,22,23,25,26,28,29,30,31,32,33,35,36,37,38,39,40,41,42,43,45,46,47,48,49,50,51,53,55,56,58,60,61,62,63,64,66,67,68,71,72,73,74,75,76,78,80,81,82,83,84,86,87,88,89,91,92,93,96,98,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,118,119,120,121,123,124,125,126,127,129,130,131,132,133,134
|
|
Comment 1 ( Arizona Department Of
Economic Security, Division Of Aging And Adult Services, Community
Action Programs And Services (DAAS/CAPS) ; State in AZ)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
The CSBG State Office
appreciates the prospect of items from this report being
auto-populated from the CSBG State Plan.
The State CSBG Office
concurs with the comments made by the Community Action Partnership
on Module 1. Regarding reporting on the percentages of local
agencies having met organizational standards.
|
|
Comment 2 ( Clarity, Impact And
Performance Project Steering Committee; RPIC in ID)
|
|
Module 1: No Comment
|
|
Comment 3 ( Hill Country Community
Action Association, Inc. ; Eligible Entity/Local Agency in TX)
|
|
Module 1: No Comment
|
|
Comment 4 ( Sacramento Employment And
Training Agency (Seta) ; Other/Unknown in CA)
|
|
Module 1: No Comment
|
|
Comment 5 (Action For Better
Community ; Eligible Entity/Local Agency in NY)
|
|
Module 1: Module 1, Section D
The agency has two concerns regarding this proposed
change. First, the section references the number of agencies that
met all of the state-adopted organizational standards. New York
State adopted the national standards and enhanced them with
additional requirements. It is unclear if this question is asking
for the number of agencies that met ONLY the national standards
that are uniform across the network, or whether this includes
additional criteria that states have added on to the national
standards as part of their own state-wide quality assurance plan.
Second, the measure provides data only on the
proportion of agencies that are fully compliant in a given year.
Given that these new reports will begin only three years into the
application of these national standards, it seems sensible to
provide more categories of compliance so that states can report
how many of their agencies are in full or close to full compliance
versus how many remain significantly below full compliance with
the standards.
|
|
Comment 6 (Action For Boston
Community Development Inc.; Other/Unknown in MA)
|
|
Module 1: No Comment
|
|
Comment 8 (Ashtabula County Community
Action Agency; Eligible Entity/Local Agency in OH)
|
|
Module 1: No Comment
|
|
Comment 9 (Association Of Nationally
Certified Roma Trainers ; Eligible Entity/Local Agency in N/A)
|
|
Module 1: • One area
currently proposed could be modified to increase utility
As
proposed, the single metric the State Agencies will report on will
be the number of eligible entities that meet 100% of the CSBG
Organizational Standards. To increase utility of the information,
State Agencies should have the option to report the number of
agencies at a variety of thresholds such as the number of local
agencies that have met 80 — 99%, 60 to 79% or less
than
60% (or similar ranges). The single standard of 100% does not show
strength or allow for demonstration of progress.
• Some areas may
need to be modified to respond to the barriers facing many local
agencies to increase clarity regarding the information to be
collected.
|
|
Comment 11 (California Community
Action Partnership Association; Other/Unknown in CA)
|
|
Module 1: No Comment
|
|
Comment 12 (Cap Service ;
Other/Unknown in WI)
|
|
Module 1: State
Administration: Module 1: Modify the options CSBG State Lead
Agencies/grantees have when reporting to OCS on CSBG
Organizational Standards to allow for a broader range of responses
as described by the Partnership. There are significant challenges
with the way this information is being reported and could expand
inappropriately beyond the Organizational Standards. Expecting
constant perfection is unrealistic and does not have practical
utility for local agencies or the State CSBG Lead Agency. As
proposed, the single metric the State Agencies will report on will
be the number of eligible entities that meet 100% of the CSBG
Organizational Standards. To increase utility of the information,
State Agencies should have the option to report the number of
agencies at a variety of thresholds that still delineate strong
performance or allow for a more accurate reflection of the
training and technical assistance needs. For instance State
Agencies should be able to report the number of local agencies
that have met 100%, 90%, 80%, 70%, 60%, 50%, and less than 50% of
the standards. This will allow for the collection of more accurate
information; better tracking of results over time; and better,
more efficient investment of training and technical assistance for
those most in need of enhancing their performance.
Module
1— Summary Analysis. I think there is a risk for
misinterpretation of data in a summary or analysis prepared by
someone not directly connected to service delivery. Time,
sophistication and deep understanding of data and its nuances are
required. For example, if a program is designed to serve people
for 18-24 months, trying to make assumptions for one year data
leaves opportunity for misrepresentation.
Module 1—
Section D. There is conflicting language regarding QIP's and TAP's
from OCS 1M 138.
|
|
Comment 13 (Cattaraugus Community
Action, Inc.; Eligible Entity/Local Agency in NY)
|
|
Module 1: No Comment
|
|
Comment 14 (Center For Community
Future ; Other/Unknown in N/A)
|
|
Module 1: No Comment
|
|
Comment 16 (Central Missouri
Community Action ; Other/Unknown in MO)
|
|
Module 1: No comment
|
|
Comment 17 (Central Nebraska
Community Action Partnership; Other/Unknown in NE)
|
|
Module 1: No Comment
|
|
Comment 18 (City Of Austin Health And
Human Services ; State in TX)
|
|
Module 1: No Comment
|
|
Comment 19 (Colorado Community Action
Association; State Association in CO)
|
|
Module 1: State
Administration: (pages 2-22)
The entirety of the
Module 1 - State Administration was not a part of this process
until the informal public comment period conducted by NASCSP/OCS
in early 2016, and has not been properly vetted and commented upon
by State Offices, and thus should be removed from this OMB FR1
process and created through its own process (including convening a
specific State Office stakeholder group review ahead of network
comments). Not only are there substantial changes to the State
Office “Final Report” represented in this module
(which again were not properly reviewed/commented upon by the
network), but the timelines involved are unrealistic in having
State Offices required to provide much of the baseline information
contained in this module at the end of August 2016 (in the Model
State Plan), for a report that has not yet been formally adopted.
Additionally, much of this module notes future Information
Memorandum’s (IM’s) and other guidance to be provided
by NASCSP/OCS, which all should first be established before the
network is asked to provide comment on the proposed changes in
this module.
Module 1, Section A: (page # 3-4)
·
A1.2 – Add “Head Start” as a potential program
administered.
· A1.2 - Add “FEMA –
Emergency Disaster Relief” as a potential program
administered.
Module 1, Section B: (page # 5-6)
·
B.7 – This high-level analysis at the State level is not
realistic given the amount of time most States have to aggregate,
review, clean and submit the data received from the local CEE’s.
In Colorado, the State Office has one month between the deadline
for submission of data from local CEE’s to the deadline for
submission of the aggregated data to NASCSP, this does not provide
enough time for in-depth analysis of trends, findings,
etc.
Module 1, Section C: (page # 7)
N/A
Module
1, Section D: (page # 8-9)
· D.2 –
Instead of an all or nothing approach to reporting on
Organizational Standards, there should be ranges of compliance
(such as 0-50%, 51%-75%, 76%-90%, 91%-99%) which would allow for a
more detailed point in time snapshot of overall compliance with
the Organizational Standards throughout the network.
·
D.3 – It is unclear at what point in time we would be
reporting Tap’s/QIP’s which could and would
drastically change from the time of monitoring to the time of
submission of this data. For instance, an agency may initially
have a TAP that is easily implemented and corrected within a
short-time frame, at which point they would come off of the TAP
and be in compliance…would we be expected to report this
TAP, or only TAP’s/QIP’s that are active at time of
report submission?
Module 1, Section E: (page # 10-14)
N/A
Module 1, Section F: (page # 15-16)
·
F.1 – It would be very time-consuming and administratively
burdensome to provide all of the actual dates of any and all
training and technical assistance activities by the State Office
and its partners. Additionally, T/TA activities locally often
cross over topics and/or cannot be easily defined according to the
list provided. It is appropriate to ask in general what T/TA needs
where addressed during the reporting period, but not be get so
detailed that the section become overly burdensome to
complete.
Module 1, Section G: (page # 17-19)
·
G.1 – Add “State Office of Information Technology”.
·
G.2 and G.3 – Administratively burdensome. This collection
of data begins to outweigh the actual provision of linkages. It
may be appropriate to ask what general linkages activities where
undertaken at the local level, or for examples of local level
linkages, but it is unrealistic to ask the State to report out on
detailed linkages activities at the local level. Additionally,
this is a duplicative data indicator as the local CEE’s are
asked to report on linkages in the Community National Performance
Indicators and Strategies/Efforts.
Module 1, Section H:
(page # 20-21)
N/A
Module 1, Section I:
(page # 22)
I.3 – This assumes
that local CEE’s have data to make management decisions
(which was the original intent of ROMA Next Generation as I
understood it). ROMA Next Generation does not accomplish
analyzation of data by local CEE’s and does not support
local CEE’s collecting and analyzing data to make management
decisions (which would imply a more long-term outlook not
continually contained within one program reporting year, which all
of this data represents).
|
|
Comment 20 (Colorado Department Of
Local Affairs ; Other/Unknown in CO)
|
|
Module 1: Module I is
duplicative of the state plan and repetitive in nature, lacking
any meaning or applicability to program development. Furthermore,
it splits the state plan application and state plan reporting into
two completely different operating systems and structures, which
is not supportive of true program analysis. The additional
reporting burden created is administratively intrusive and
unnecessary as it is or could be covered within the current state
plan structure.
The proposed reporting period for
Module I is the federal fiscal year. This does not align with
either our program year (calendar year) or state fiscal year. The
data entered in this section under this expectation would also not
align with the reporting time period from eligible entities.
States should to have the ability choose the reporting period that
aligns with the rest of the program reporting or alternatively,
Module 1 should be removed completely and accomplished within the
model state plan process and existing annual/ biannual renewal
plans.
Module
I, Section B: (page # 5-6) - In the June 21 webinar, OCS announced
that it would be releasing an Information Memorandum "soon"
on using ACS data. This 1M has not yet been released, nor has any
guidance been issued by OCS for this section. It is unreasonable
to request comment on something that has not yet been issued, and
will likely not be issued before proposed tracking would begin on
October I, 2016 (FEY 17). There is also mention of additional
support and tools to be provided but there are no proposed, plans
or tools that have been defined to evaluate. These issues further
show that this module has not been given appropriate time and
attention and should be removed until further consideration has
occurred. Module I, Section D: (page # 8-9) - D2 requires that
states report only on what entities are meeting 100% of the
organizational standards. As we have acknowledged implementation
as a process, with a desire for states to show progress towards
achieving 100% compliance, reporting should allow for a scale that
shows where eligible entities are at in working towards
compliance, rather than creating all or none criteria. This
section also appears to favor QIPs for even minor items out of
compliance. Colorado's preferred methodology is to create a
Technical Assistance Plan (TAP) first, and eligible entities may
work under a TAP as long as they are making reasonable progress to
goals on a timeline agreed upon by both the state and the eligible
entity. Colorado would not progress to a Quality Improvement Plan
(QIP) unless issues are persistent and/or pervasive or pose a high
level of risk to the program, eligible entity or state.
Module
I, Section G: (page # 17-19) Colorado's state model is one of
state oversight with local control. In this environment, linkages
are primarily determined and coordinated at the local level and
are not enforced by state offices.
In summary:
ROMA Next Generation has
moved away from outcomes based reporting and now reflects
additional information reporting/annual report measures that
triple the amount of indicators requested, and is not relevant,
necessary or useful the community action strategy and long-term
evaluation data. We request that this
Module I was added
during the process and has not been given ample consideration and
review; it is also not appropriate as part of ROMA Next
Generation. We request that this module undergo more discussion
and be integrated with the state plan process, rather than
included in ROMA NG.
The amount of reporting has been
grossly underestimated for CAAs, state association and state
agencies. We request an evaluation that prioritizes that data
points and returns to outcome-based measurement (versus data
reporting) to truly show the impact of community action and CSBG
in our nation. We also request consideration of the volume of
reporting requested of agencies and subcontractors who may be
receiving small dollar amounts but still provide for a critical
need in their communities.
We request that HHS/ACF/OCS
look at interoperability systems to allow agencies to link
multiple date reporting systems without creating yet another
unfunded mandate for funder-driven tracking that is not useful for
client-centered agencies. We further request that HHS/ACF/OCS
provides funding to support the interoperability of systems for
CSBG.
The proposed timeline is too much too fast on the
heels of the implementation of organizational standards. We
request delaying implementation by at least a year and allowing
scaled measurement to assess agencies on organizational standards
rather than an "all or nothing" approach to evaluating
their compliance.
|
|
Comment 21 (Combined Community Action
; Eligible Entity/Local Agency in TX)
|
|
Module 1: No Comment
|
|
Comment 22 (Community Action Agency ;
Other/Unknown in NE)
|
|
Module 1: No Comment
|
|
Comment 23 (Community Action Agency
Of Beaver County ; Eligible Entity/Local Agency in PA)
|
|
Module 1: No Comment
|
|
Comment 25 (Community Action Agency
Of North Alabama ; Other/Unknown in AL)
|
|
Module 1: NO Comment
|
|
Comment 26 (Community Action Agency
Of Northwest Alabama, Inc.; Eligible Entity/Local Agency in AL)
|
|
No Module 1 Content for this
comment.
|
|
Comment 28 (Community Action Agency
Of Northwest Alabama, Inc. ; Eligible Entity/Local Agency in AL)
|
|
No Module 1 Content for this
comment.
|
|
Comment 29 (Community Action Agency
Of Southern New Mexico; State Association in NM)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
Module 1, Section D:
(page # 8-9)
Response: D3 (Page 9) What is the
criteria or cause for putting a QIP or TAP in place? It is
suggesting that one unmet standard could constitute a TAP or QIP?
Will guidance or documents be provided for State use? Is this
totally at the States discretion?
|
|
Comment 30 (Community Action
Association ; State Association in NM)
|
|
Module 1: Module 1, Section D:
(page # 8-9)
Response: D3 (Page 9) What
is the criteria or cause for putting a QIP or TAP in place? It is
suggesting that one unmet standard could constitute a TAP or QIP?
Will guidance or documents be provided for State use? Is this
totally at the States discretion?
|
|
Comment 31 (Community Action
Commission Of Erie, Huron, & Richland Counties ; Eligible
Entity/Local Agency in N/A)
|
|
Module 1: No Comment
|
|
Comment 32 (Community Action
Committee Of Victoria, Texas; Eligible Entity/Local Agency in TX)
|
|
Module 1: No Comment
|
|
Comment 33 (Community Action Council
; Eligible Entity/Local Agency in KY)
|
|
Module 1: No Comment
|
|
Comment 35 (Community Action Council
Of Portage County; Other/Unknown in OH)
|
|
Module 1: No Comment
|
|
Comment 36 (Community Action
Partnership ; National Partner in DC)
|
|
Module 1: Module 1: State
Administration
· Significantly modify Question
D2 to allow for better and more accurate reporting. Question D2
does not allow for State CSBG Lead Agencies to report the number
of agencies not meeting 100% of the Organizational Standards thus
providing one, very limited and difficult to consistently achieve
metric. It is important to including reporting on the
Organizational Standards; however, the proposed format for
reporting is problematic.
· Modify Question D3
to reflect the purpose and intent of Technical Assistance Plans
and Quality Improvement Plans. Question D3 does not clarify that a
Technical Assistance Plan rather than a Quality Improvement Plan
is the preferred option for agencies not meeting Organizational
Standards as per the guidance provided in OCS IM 138.
·
Remove or significantly modify Question B7. The question places an
undue burden on State CSBG Lead Agencies and needs to be modified
to be reflective of data collected.
|
|
Comment 37 (Community Action
Partnership Of Kern ; Other/Unknown in CA)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
Module 1, Section A:
(page # 3-4)
· No Comments
Module
1, Section B: (page # 5-6)
· Question B7
Analysis: This question will require information be collected that
goes beyond what is being collected through the ROMA data included
in this draft annual report. The estimated time burden in the
federal register did not assume this additional data collection.
This is added reporting burden on State CSBG Lead Agencies because
as written, the questions require the agencies to go beyond merely
reporting the data and conduct sophisticated data analysis that
will require additional time and may exceed the capacity of agency
staff to provide meaningful information.
|
|
Comment 38 (Community Action
Partnership Of Orange County; Eligible Entity/Local Agency in CA)
|
|
Module 1: No Comment
|
|
Comment 39 (Community Action
Partnership Of Riverside County; Eligible Entity/Local Agency in
CA)
|
|
Module 1: Section D.2:
Organizational Standards Performance (page # 9)
The
information requested in the table for Number that Met All State
Standards/Number that Did Not Meet All State Standards does not
seem to have practical utility. It would seem more useful and
relevant to track the number of agencies meeting each individual
Standard, from 1.1 to 9.4. Then, instead of reporting on the
number of agencies meeting all/not meeting all of the
Organizational Standards, it would reflect the number of agencies
meeting each individual Standard in each State, and determination
could be made on which Standards each State should focus on
providing technical assistance for, or possibly having as
priorities for allocating Discretionary CSBG funding to, etc.
Also, this would more accurately reflect how each State, and
ultimately how the CSBG network is doing in regards to meeting/not
meeting each individual Standard.
|
|
Comment 40 (Community Action
Partnership Of San Luis Obispo County, Inc.; Eligible Entity/Local
Agency in CA)
|
|
Module 1: No Comment
|
|
Comment 41 (Community Action Planning
Council Of Jefferson County, Inc.; Eligible Entity/Local Agency in
NY)
|
|
Module 1: State
Administration: (pages 2-22)
Module 1, Section A:
(page # 3-4)
• No Comments
Module 1,
Section B: (page # 5-6)
• Question B7 Summary
Analysis. As written, this question exceeds the capacity of many
state CSBG Lead Agencies (grantees). This question will require
information be collected that goes beyond what is being collected
through the ROMA data included in this draft annual report. The
estimated time burden in the federal register did not assume this
additional data collection. This is added reporting burden on
State CSBG Lead Agencies because as written, the questions require
the agencies to go beyond merely reporting the data and conduct
sophisticated data analysis that will require additional time and
may exceed the capacity of agency staff to provide meaningful
information.
Module 1, Section C: (page # 7)
•
No comment
Module 1, Section D: (page # 8-9)
•
D2. This question is too narrow in its potential responses and
decreases the capacity of the state to fulfill its duties and
impacts both the quality of the data collected and the utility of
the data. The single metric sought is the number of entities that
met all state-adopted organizational standards; this is a poor
metric when used in isolation of performance at a variety of
levels. No one and no agency will score 100% of anything 100% of
the time. Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
•
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
Module
1, Section E: (page # 10-14)
• No
comments
Module 1, Section F: (page # 15-16)
•
No comments
Module 1, Section G: (page # 17-19)
•
G3b. This question would not be able to be answered with the ROMA
data being collected as proposed. This will increase the burden on
states to report.
Module 1, Section H: (page # 20-21)
• No comments
Module 1, Section I:
(page # 22)
• I4. This question seeks information
regarding ROMA Goals. As the ROMA Goals are not listed in one
module, it noted here to re-instate the current ROMA Goal:
Agencies increase their capacity to achieve results. It is
unfortunate there is not a module or section number that requests
feedback on the data collection formatted around the set of new
national ROMA goals.
|
|
Comment 42 (Community Action Program
Associations For Community Action Agencies; State Association in
WI)
|
|
Module 1: No Comment
|
|
Comment 43 (Community Renewal Team ;
Eligible Entity/Local Agency in CT)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
Module 1, Section D:
(page # 8-9)
a. D2. There are significant challenges
with the way this information is being reported and potential
creates a misuse of the Organizational Standards. As proposed, the
metric the State Agencies will report on will be the number of
eligible entities that meet 100% of the CSBG Organizational
Standards. State agencies should have the option to report the
number of agencies at a variety of thresholds (100%, 90%, 80%,
70%, 60 %...) that show strong performance or the training and
technical needs of the agencies. Presenting this type of
information in this way will reflect a more accurate picture of
agency performance will allow for better tracking over time and
assist the State agencies in more efficient investment of training
and technical assistance.
b. To manage the burden of
effort on agency this question needs to be reworked. The Quality
Improvement Plan (QIP) is the current language referenced as the
preferred option is an eligible entity has one or more unmet
organizational standards. A Technical Assistance Plan (TAP) is the
preferred option in OCS IM138. As mentioned in IM138 as long as an
eligible entity is making progress towards meeting the standards,
the state should not take action to reduce or eliminate funding,
this should be referenced.
|
|
Comment 45 (Del Norte Senior Center,
Inc.; Eligible Entity/Local Agency in CA)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
State agency staff and
administration would be in the best position to comment on
specific changes from existing state reporting requirements;
however, any increased reporting burden to the states would pass
down to the local agencies. Also, to the extent that narrative
information is being required, the burden of reporting in
narrative format is significantly more than that of reporting
statistical data. The utility and usefulness of requiring every
agency to
report this information just for the sake of
humanizing the program is questionable.
|
|
Comment 46 (Delaware Opportunities
Inc.; Eligible Entity/Local Agency in DE)
|
|
Module 1: No Comment
|
|
Comment 47 (Department Of Community
Service And Development ; State in CA)
|
|
Module 1: The State
Accountability Measures are a significant improvement to the
integrity and evaluation of CSBG. These measures would require
states to demonstrate the Results Oriented Management and
Accountability (ROMA) cycle through assessment, planning,
implementation, achievement of results, and evaluation. This is a
measurable step to hold states accountable to the strategies and
plans indicated in the State Plan. It is also a method to report
out to the CSBG network, the Office of Community Services and
the
general public the successes and efforts of Community
Action.
Considering that fact, there are still areas of
improvement to be addressed. Module 1, Section B.7 contains
statewide goals and accomplishments that require the State to
provide a summary analysis on major changes and trends within the
CSBG network. On the surface, the effort appears straightforward.
However, upon further inspection some of the questions asked would
require the State to implement several additional data collection
and analysis processes to effectively report out on the questions
asked.
Particular
efforts identified in Section B.7 will prove difficult to analyze:
O
"In what domains did eligible entities have the greatest
impact in the State based on the State's analysis of data?" o
The Department has 60 agencies servicing 58 distinct counties. The
assessment of impact is not discreet due to the vast differences
in service territory (rural and urban), population (language,
ethnic and cultural diversity) and agency operations. It would be
very difficult to assess impact statewide with the degree of
variance and effectiveness of different domains administered in a
myriad of fashions.
"How well did the State and
eligible entities do in meeting the needs of individuals,
families, and communities compared to targets? In what areas were
eligible entities most successful in meeting their targets? In
what areas did eligible entities experience challenges in meeting
their targets?" o The review of programmatic data required of
the current annual reporting structure, CSBG Information Survey
(CSBG IS), requires us to analyze current and prior year data to
assess the changes in service delivery. This effort would require
the Department to add additional data analysis tools to draw
conclusions based upon the plethora of variance in program
structure. This again is a difficult assessment to achieve and
would require a uniform structure to achieve.
The challenges put
forth in Section B.7 are directly correlated with increased
workload associated with evaluating greatest impact and trends.
This effort is very subjective, labor intensive and would require
significant and increased alterations to data collection systems
and reporting structures. It is highly encouraged that this
section be revisited or revised to remove the subjectivity and to
address the resource burden required to complete this section of
the report.
|
|
Comment 48 (Department Of Housing And
Community Development ; State in MA)
|
|
Module 1: Module 1 –
Section B we are concerned about the quality and clarity of
information to be collected for the following questions:
•B.2.
Eligible Entity Satisfaction Targets – The state’s
response to this question relies greatly on the ability of OCS,
through use of the ACSI Survey, to achieve an acceptable response
rate, administer the survey with regularity, and provide
subsequent feedback to States in a timely manner. In the first
ACSI survey administered to the CAA network in 2015, a number of
State’s learned too few of their CAAs responded for the rate
to be statistically significant. In our State, it took several
encouraging reminders to our network partners to achieve a
statistically significant response. We are concerned that if the
ACSI is administered annually (or even bi-annually), the response
rate will decline further and, as a result, feedback provided will
become less reliable for us to adjust our State Plan goals based
upon. Additionally, while OCS promised further guidance on this
particular question because the CSBG State Plan due September 1,
2016 requires us to set an Eligible Entity Satisfaction Target, no
further guidance has been provided to date.
•B.7.
Summary Analysis – The State’s ability to provide a
meaningful analysis of the trends and results this question
alludes to relies on our ability to share data from other federal
programs CAAs coordinate CSBG funding with (e.g. – Head
Start, WIC, HUD, SNAP, WIOA, etc.). Few meaningful data “bridges”
between different funding sources currently exist at the Federal
level. We are hopeful, however, that proposed changes to the CSBG
National Performance Indicators (NPIs), etc. will allow our state
to provide a more meaningful analysis of trends in demographic
shifts of those served by our agencies as well as the type and
frequency of services provided.
|
|
Comment 49 (Department Of Human
Services ; State in AR)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
As a State Administration
I am concerned that the proposed Module 1 reporting date of March
31, 2017 does not allow State CSBG Lead Agencies sufficient time
to prepare for reporting on this information, updating their
internal processes and procedures, and using OLDC.
Module
1, Section A: (page # 3-4)
The American Customer
Satisfaction Index (ACSI) was administered for the first time in
FY 2015 and States received their results in February 2016. States
were not fully informed of intended reporting requirements and
State responsibilities around the ACSI, until Module 1 was
released for review during the 60-day comment period and
additional guidance from OCS was projected to be released by
August 2016. States will need guidance and additional time to work
with their agencies and prepare reports on their use of the ACSI.
Module 1, Section C: (page # 7)
Analysis
is contextual and these questions should not be mandated as
implied in the language, and are only examples of the ways in
which data could be analyzed and understood. Language in the final
bullet that refers to States “encouraging” local
entities to change based on State analysis can be interpreted as a
potential intrusion of State authority upon statutorily mandated
local control and should be altered or eliminated. States may need
guidance and additional training to identify questions and data
analysis procedures to guide state-level analysis.
|
|
Comment 50 (Department Of Social
Services Office Of Community Services ; State in CT)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
Module 1, Section D:
(page # 8-9)
o Is there a federal threshold for when
an agency is not to be put on a TAP re: Organizational Standards?
Or is this at the discretion of the State?
|
|
Comment 51 (Dickinson Community
Action Partnership ; Other/Unknown in ND)
|
|
Module 1: OCS on CSBG
Organizational Standards to allow for a broader range of
responses. As noted below, there are significant challenges with
the way this information is being reported and creates a misuse of
the Organizational Standards. Expecting constant perfection is
unrealistic and does not have practical utility for local agencies
or the State CSBG Lead Agency. As proposed, the single metric the
State Agencies will report on will be the number of eligible
entities that meet 100% of the CSBG Organizational Standards. To
increase utility of the information, State Agencies should have
the option to report the number of agencies at a variety of
thresholds that still delineate strong performance or allow for a
more accurate reflection of the training and technical assistance
needs.
Module 1, Section B: (page # 5-6)
module
1
• Question 87 Summary Analysis. As written, this
question exceeds the capacity of many state CSBG Lead Agencies
(grantees). This question will require information be collected
that goes beyond what is being collected through the ROMA data
included in this draft annual report. The estimated time burden in
the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity Of agency staff to provide meaningful information.
Module
1, Section D: (page # 8-9)
D2. This question is too
narrow in its potential responses and decreases the capacity of
the state fulfill its duties and impacts both the quality of the
data collected and the utility of the data. The single metric
sought is the number of entities that met all state-adopted
organizational standards; this is a poor metric when used in
isolation of performance at a variety of levels. No one and no
agency will score 100% of anything 100% of the time. Ongoing
perfection is an unreasonable expectation and is inappropriate to
be used as the single reporting measure. This question should be
changed to allow states to report on how many entities met 100%,
90%, 80%, 70%, 60% and less than 50% of the Standards. This will
increase the performance of states by allowing them to focus their
limited training and technical assistance resources on those
entities not meeting multiple standards. It would also improve the
quality of data as it will provide a more robust assessment of
performance.
This question needs
to be re-worked in order to manage the burden of effort on states
and eligible entities. The current language conveys that a Quality
|
|
Comment 53 (Division Of Sate Service
; State in DE)
|
|
Module 1: No Comment
|
|
Comment 55 (Florida Department Of
Economic Opportunity ; Other/Unknown in FL)
|
|
Module 1: * Changes will take
place mostly on a state level, but we need to remember that
changes made at that level could eventually affect us at the local
level. Module 2 is information that we already gather and provide
as the total amount of
* CSBG funding expended during
the reporting period based on the various domains.
*
Allowing state agencies to only provide a 100% or fail to local
agencies for CSBG Organizational Standards doesn’t provide a
true picture of the agencies standards. States should be able
allowed to report how many agencies met 100%, 90%, 80%, etc. in
order to truly reflect a state’s performance. Also, it would
allow for focused technical assistance on agencies with the most
need.
|
|
Comment 56 (Four Square Community
Action ; Other/Unknown in NC)
|
|
Module 1: No Comment
|
|
Comment 58 (Greater Bergen Community
Action; Eligible Entity/Local Agency in NJ)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
Module 1, Section D:
(page # 8-9)
High passing grades
for almost everything is 90-100%, not just 100%.Considering the
number of standards, changing this to 95% would give a better idea
of where the agencies stand, instead of this all or nothing
approach.
|
|
Comment 60 (Hinds County Human
Resources Agency ; Other/Unknown in MS)
|
|
Module 1: Module 1, Section A:
(page # 3-4) No Comments
Module 1, Section B: (page #
5-6)
Question B7 Summary Analysis. As written, this
question exceeds the capacity of many state CSBG Lead Agencies
(grantees). This question will require information be collected
that goes beyond what is being collected through the ROMA data
included in this draft annual report. The estimated time burden in
the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful information.
Module 1, Section C: (page # 7)
No comment
Module 1, Section D: (page # 8-9)
D2. This
question is too narrow in its potential responses and decreases
the capacity of the state fulfill its duties and impacts both the
quality of the data collected and the utility of the data. The
single metric sought is the number of entities that met all
state-adopted organizational standards; this is a poor metric when
used in isolation of performance at a variety of levels. No one
and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
This
question needs to be re-worked in order to manage the burden of
effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
Module
1, Section E: (page # 10-14)
No comments
Module
1, Section F: (page # 15-16)
No comments
Module
1, Section G: (page # 17-19)
G3b. This question
would not be able to be answered with the ROMA data being
collected as proposed. This will increase the burden on states to
report.
Module 1, Section H: (page # 20-21)
No
comments
Module 1, Section I: (page # 22)
I4.
This question seeks information regarding ROMA Goals. As the ROMA
Goals are not listed in one module, it noted here to re-instate
the current ROMA Goal: Agencies increase their capacity to achieve
results. It is unfortunate there is not a module or section number
that requests feedback on the data collection formatted around the
set of new national ROMA goals.
|
|
Comment 61 (Illinois Association
Community Action Agency ; Other/Unknown in IL)
|
|
Module 1: Module 1, Section B:
(page # 5-6)
Question B7 Summary Analysis.
As written, this question exceeds the capacity of many state CSBG
Lead Agencies (grantees). This question will require information
be collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies 6 Illinois Association of Community Action Agencies
Feedback — CSBG Annual Report OMB PRA Clearance FRN #1
because as written, the questions require the agencies to go
beyond merely reporting the data and to conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful information.
Module
I, Section D: (page # 8-9)
D2. This question is
too narrow in its potential responses. As a result, it decreases
the capacity of the state to fulfill its duties and impacts both
the quality of the data collected and the utility of the data. The
single metric sought is the number of entities that met all
state-adopted organizational standards; this is a poor metric when
used in isolation of performance at a variety of levels. No one
and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
This
question needs to be re-worked in order to manage the burden of
effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS 1M 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference 1M 138's statement that as long as an eligible entity is
making progress towards meeting the standards, the state should
not take action to
reduce or eliminate funding. The
expectation that CAAs can be perfect, i.e. 100% compliant is not
realistic. On any given day, agencies/programs face uncertainty —
it is the nature of this business. Instead, consider a scaling of
compliance that will allow for improvement plans to be genuine,
thoughtful and crafted towards innovation. There should also be
resilience scaling as well, which examines past practices and the
agency's capacity to self-correct.
Module I, Section G:
(page # 17-19)
G3b. Collecting the ROMA data as is
being proposed would not provide an answer to this question. Data
collection to answer this question will further increase the
burden on states to report.
Module I, Section I: (page
# 22)
14. This question seeks information regarding
ROMA Goals. As the ROMA Goals are not listed in one module, it
noted here to re-instate the current ROMA Goal: Agencies increase
their capacity to achieve results. It is unfortunate there is not
a module or section number that requests feedback on the data
collection formatted around the set of new national ROMA goals.
|
|
Comment 62 (Inca Community Service;
Eligible Entity/Local Agency in OK)
|
|
Module 1: Section D: (page #
8-9)
· The single metric of the number of
entities that met ALL state-adopted organizational standards is
too narrow. It does not show a clear picture of the agencies. No
one and no agency will score 100% of anything 100% of the time.
Perfection to this point is an unreasonable expectation. The
purpose should be to see which agencies are thriving and which
need more training and technical assistance.
|
|
Comment 63 (Indiana Housing And
Community Development Authority ; State in ID)
|
|
Module 1: Module 1, Section A:
(page # 3-4)
Module 1, Section B: (page # 5-6)
B1
Auto populated from plan
B2 No
B3 We have
public hearing and send information out to agencies to get
feedback.
B4 Agency Level
B5 & B6
Narratives will provide information
B7 Although CSBG
has a broad interpretation of funding use, as a lead State Agency,
IHCDA need to provide more direction and oversight to ensure that
use of funds is being used to develop programs that produce
results and reduce poverty.
Module 1, Section C: (page
# 7)
C1 –Auto populated
C2 –
Auto populated
C3- No Change, Allocation remained the
same which agencies received 90%, will be populated
Module
1, Section D: (page # 8-9)
D1
Is this the CAR will be reviewed during monitoring
D2
Auto populated
D3 Monitoring (Steve)
Module
1, Section E: (page # 10-14)
E1 -No changes made to
the formula
E2 Auto populated
E3 Yes
allocation was made within 30 days.
E4 Auto Populated
E5 Auto Populated
E6 Auto Populated
E7 Auto Populated
E8
Take from State Plan
E9 Auto Populated
Module
1, Section F: (page # 15-16)
F 1 Auto Populated
F
2 Take form State Plan
Module 1, Section G: (page #
17-19)
G 1 Auto Populated
G 2 Auto
Populated
G 3 Auto Populated
G4 Take form
State Plan
G 5 Auto Populated
G 6 Auto
Populated
Module 1, Section H: (page # 20-21)
Steve’s
section- Monitoring
Module 1, Section I: (page # 22)
* Consider adding remote or desk-top monitoring as an
additional monitoring option
* Consider adding
self-assessment as an additional monitoring option
|
|
Comment 64 (Iowa Community Action
Association; Eligible Entity/Local Agency in IA)
|
|
Module 1: Module I - State
Administration: (pages 6-7)
B7 Summary Analysis. As
written, this question seems to exceed the capacity of State CSBG
Lead Agencies. This question will require information be collected
beyond what is being collected through the ROMA data included in
this draft annual report. The estimated time burden in the federal
register Agencies because as written, the questions require the
agencies to go beyond merely reporting the data and conduct
sophisticated data analysis that will require additional time and
may exceed the capacity of agency staff to provide meaningful
information.
Module I, Section D: (page # 8-9)
D2
Organizational Standards Performance. We believe this question is
too narrow in its potential responses and decreases the capacity
of the state fulfill its duties and impacts both the quality of
the data collected and the utility of the data. The single metric
sought is the number of entities that met all state-adopted
organizational standards; this is a poor metric when used in
isolation of performance at a variety of levels. No one and no
agency will score 100% of anything 100% of the time. Ongoing
perfection is an unreasonable expectation and is inappropriate to
be used as the single reporting measure. This question should be
changed to allow states to report on how many entities met 100%,
90%, 80%, 70%, 60% and less than 50% of the Standards. This will
increase the performance of states by allowing them to focus their
limited training and technical assistance resources on those
entities not meeting multiple standards. It would also improve the
quality of data as it will provide a more robust assessment of
performance.
This question needs to be re-worked in
order to manage the burden of effort on states and eligible
entities. The current language conveys that a Quality Improvement
Plan (QIP) is the preferred option if an eligible entity has one
or more unmet organizational standards. OCS 1M 138 clearly
articulates the Technical Assistance Plan (TAP) is the preferred
option. This section should also reference 1M 138's statement that
as long as an eligible entity is making progress towards meeting
the standards, the state should not take action to reduce or
eliminate funding.
|
|
Comment 66 (Lakes And Prairies
Community Action Partnership ; Other/Unknown in MN)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
Module l, Section D2:
(page # 8-9)
This question is too narrow in its
potential responses and decreases the capacity of the state
fulfill its duties and impacts both the quality of the data
collected and the utility of the data. The single metric sought is
the number of entities that met all state-adopted organizational
standards; this is a poor metric when used in isolation of
performance at a variety of levels. No one and no agency will
score 100% of anything 100% of the time. Ongoing perfection is an
unreasonable expectation and is inappropriate to be used as the
single reporting measure. This question should be changed to allow
states to report on how many entities met 100%, 90%, 80%, 70%, 60%
and less than 50% of the Standards. This will increase the
performance of states by allowing them to focus their limited
training and technical assistance resources on those entities not
meeting multiple standards. It would also improve the quality of
data as it will provide a more robust assessment of performance.
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS 1M 138 clearly articulates the Technical
Assistance
Plan (TAP) is the preferred option. This section should also
reference 1M 13 8's statement that as long as an eligible entity
is making progress towards meeting the standards, the state should
not take action to reduce or eliminate funding.
|
|
Comment 67 (Lorain County Community
Action Agency ; Other/Unknown in OH)
|
|
Module 1: No Comment
|
|
Comment 68 (Louisville Metro
Community Service ; State in KY)
|
|
Module 1: No Comment
|
|
Comment 71 (Maine Community Action
Association ; Other/Unknown in NE)
|
|
Module 1: No Comment
|
|
Comment 72 (Massachusetts Community
Action Agency Planners’ Group; State Association in MA)
|
|
Module 1: No Comment
|
|
Comment 73 (Maui Economic Opportunity
Inc. ; Other/Unknown in HI)
|
|
Module 1: No Comment
|
|
Comment 74 (Merced County Community
Action Agency ; Other/Unknown in CA)
|
|
Module 1: No Comment
|
|
Comment 75 (Michigan Community Action
; Eligible Entity/Local Agency in MI)
|
|
Module 1: Module 1: Modify the
options CSBG State Lead Agencies/grantees have when reporting to
OCS on CSBG Organizational Standards to allow for a broader range
of responses There are significant challenges with the way the
proposed changes to the CSBG Annual Report requests information
and erodes the effectiveness of the Organizational Standards. The
reporting tools set up false expectations of the data and
performance. As proposed, the single metric the State Agencies
will report on will be the number of eligible entities that
meet
100% of the CSBG Organizational Standards. State Agencies should
have the option to report the number of agencies at a variety of
thresholds that still delineate strong performance or allow for a
more accurate reflection of the training and technical assistance
needs.
Module I - State
Administration: (pages 2-22)
Module I, Section B:
(page # 5-6)
Question B7 Summary Analysis. As written,
this question exceeds the capacity of many state CSBG Lead
Agencies (grantees). This question will require information be
collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful
information.
Module I, Section D: (page # 8-9)
D2.
This question is too narrow in its potential responses and
decreases the capacity of the state fulfill its duties and impacts
both the quality of the data collected and the utility of the
data. The single metric sought is the number of entities that met
all state-adopted organizational standards; this is a poor metric
when used in isolation of performance data variety of levels. No
one and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. D3. This question needs to be re-worked in order to
manage the burden of effort on states and eligible entities. The
current language conveys that a Quality Improvement Plan (QIP) is
the preferred option if an eligible entity has one or more unmet
organizational standards. OCS 1M 138 clearly articulates the
Technical Assistance Plan (TAP) is the preferred option. This
section should also reference 1M 138's statement that as long as
an eligible entity is making progress towards meeting the
standards, the state should not take action to reduce or eliminate
funding.
Module 1, Section G: (page # 17-19)
G3b.
This question would not be able to be answered with the ROMA data
being collected as proposed. This will increase the burden on
states to report.
|
|
Comment 76 (Minnesota Community
Action Partnership; State Association in MN)
|
|
Module 1: State
Administration
Module 1, Section B: (page # 5-6)
·
Question B7 Summary Analysis. As written, this question exceeds
the capacity of many state CSBG Lead Agencies (grantees). This
question will require information be collected that goes beyond
what is being collected through the ROMA data included in this
draft annual report. The estimated time burden in the federal
register did not assume this additional data collection. This is
added reporting burden on State CSBG Lead Agencies because as
written, the questions require the agencies to go beyond merely
reporting the data and to conduct sophisticated data analysis that
will require additional time and may exceed the capacity of agency
staff to provide meaningful information. 100 Empire Drive | Suite
202 | Saint Paul, MN 55103 | www.MinnCAP.org
Module 1,
Section D: (page # 8-9)
· D2. This question is
too narrow in its potential responses. As a result, it decreases
the capacity of the state to fulfill its duties and impacts both
the quality of the data collected and the utility of the data. The
single metric sought is the number of entities that met all
state-adopted organizational standards; this is a poor metric when
used in isolation of performance at a variety of levels. No one
and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
·
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
Module
1, Section G: (page # 17-19)
· G3b. Collecting
the ROMA data as is being proposed would not provide an answer to
this question. Data collection to answer this question will
further increase the burden on states to report.
Module
1, Section I: (page # 22)
· I4. This question
seeks information regarding ROMA Goals. As the ROMA Goals are not
listed in one module, it noted here to re-instate the current ROMA
Goal: Agencies increase their capacity to achieve results. It is
unfortunate there is not a module or section number that requests
feedback on the data collection formatted around the set of new
national ROMA goals.
|
|
Comment 78 (Missouri agency Inc. ;
Other/Unknown in MO)
|
|
Module 1: No Comment
|
|
Comment 80 (Montgomery County
Community Action Development Commission; Other/Unknown in PA)
|
|
Module 1: No Comment
|
|
Comment 81 (Multi-Service Center
(Msc) ; Eligible Entity/Local Agency in WA)
|
|
Module 1: No Comment
|
|
Comment 82 (National Association For
State Community Service Program ; Other/Unknown in N/A)
|
|
Module 1: 3. Module 1
a.
B.2 and B.3 Eligible Entity Satisfaction Targets: There are issues
related to completing B.2 and B.3. These issues are related to the
need for statistically significant data, guidance from OCS, and
consistent administration and dissemination of the survey and the
results.
b. B.7 Summary Analysis: Analysis is
contextual and therefore, these questions should not be mandated.
Limited data sharing across programs creates challenges for
meaningful analysis of trends and results.
c. D.2
Organizational Standards Performance: This question sets an
unreasonable standard for CAAs to meet 100% of the Organizational
Standards in a reporting period. NASCSP acknowledges this standard
is set in the CSBG State Plan, but believes it should not be the
single reporting measure.
d.
D.3 QIPs and TAPs: Additional guidance will be necessary to ensure
consistent practices throughout the CSBG Network for creating and
implementing TAPs and QIPs.
e. H.4 and H.5 QIPs:
Question H.4 and H.5 will require additional training and
technical assistance and guidance from OCS before implementation
of Module 1.
Module 1 – State Administration
The
sections included to manage and measure State Administration align
well with the CSBG State Plan and State Accountability Measures;
however, since Module 1 was not included in the initial feedback
period for the proposed annual report, additional conversation,
training and technical assistance, and concrete guidance are
needed prior to implementation. Given its importance, Module 1
deserves thorough review and consideration by the full CSBG
Network. This need for additional vetting directly effects the
projected timeline for reporting on this module.
Overall,
NASCSP is concerned that OIS will not have sufficient time to
build Module 1 in OLDC and work with States to ensure a smooth
data entry process by March 31, 2017. Additionally, the proposed
Module 1 reporting date of March 31, 2017 does not allow State
CSBG Lead Agencies sufficient time to prepare for reporting on
this information, update their internal processes and procedures,
and use OLDC. While, NASCSP recognizes OCS’ budget request
includes additional set aside funding to support States, it is
essential—not optional— that States receive the
funding necessary to fulfill the increased administration and
oversight requirements. Given that this is a request and not an
approved budget item, NASCSP is conscientious of the potentially
increased need to dedicate staff time and resources at the State
level in order to continue the CSBG Network’s focus on
quality administration and oversight. The following sections in
Module 1 include new reporting elements that will require
intensive and targeted training and technical assistance. As such,
the proposed timeline is not realistic.
Module 1, Section B,
Statewide Goals and Accomplishments (page # 5-6):
Questions
B.2 and B.3 address the State’s target for eligible entity
Overall Satisfaction and process for considering feedback. The
State’s response to this question relies greatly on the
ability of OCS, through use of the American Customer Satisfaction
Index (ACSI) Survey, to achieve an acceptable response rate,
administer the survey with regularity, and provide subsequent
feedback to States in a timely manner. The ACSI was administered
for the first time in FY 2015 and States received their results in
February 2016. The process for gathering statistically significant
results across the States was time consuming and response rates
varied greatly across the nation. Additionally, in some states
there were not enough eligible entities to achieve a statistically
significant and anonymous result for the ACSI. States were not
fully informed of intended reporting requirements and State
responsibilities around the ACSI, until Module 1 was released for
review during the 60-day comment period and additional guidance
NASCSP Feedback – FRN #1 OMB PRA Clearance, CSBG Annual
Report from OCS was projected to be released by August 2016.
States will need guidance and additional time to work with their
agencies and prepare reports on their use of the ACSI.
The analysis
component of Question B.7 presents examples of the types of
analysis questions integral to quality performance management and
review at the State level. However, NASCSP emphasizes that
analysis is contextual and these questions should not be mandated
as implied in the language, and should instead be clearly marked
as examples of the ways in which data could be analyzed and
understood. The State’s ability to provide meaningful
analysis of trends and results relies on the States’ and
CAA’s ability to access, aggregate, and analyze data from a
wide range of federally funded programs, some of which have
limited data sharing capabilities. Furthermore, language in the
final bullet that refers to States “encouraging” local
entities to change based on state analysis can be interpreted as a
potential intrusion of State authority upon statutorily mandated
local control and should be altered or eliminated. We are hopeful
that if States receive the necessary guidance and additional
training, States will improve their ability to identify questions
and data analysis procedures to guide State-level analysis.
Module 1, Section D, Organizational Standards for
Eligible Entities (page # 8-9): NASCSP recognizes the work done by
the Community Action Partnership and the larger CSBG Network in
crafting a common set of organizational standards and the enhanced
emphasis on organizational excellence. Question D.2 sets an
unreasonable standard for CAAs to meet 100% of the Organizational
Standards in a reporting period. NASCSP acknowledges this standard
is set in the CSBG State Plan, but believes it should not be the
single reporting measure. This section should also reference IM
138’s statement that as long as an eligible entity is making
progress towards meeting the standards, the state should not take
action to reduce or eliminate funding. Question D.3 asks States to
provide the number of CSBG Eligible Entities with unmet standards
with a Technical Assistance Plan (TAP) or Quality Improvement Plan
(QIP) in place. Guidance on standardizing the development of TAPs
and QIPs must be provided to ensure consistent practices
throughout the CSBG Network.
Module 1, Section H,
Monitoring, Corrective Action, and Fiscal Controls (page # 20-21):
As stated in reference to Quality Improvement Plans in the above
paragraph, question H.4 and H.5 will require additional training
and technical assistance and guidance from OCS before
implementation of Module 1.
|
|
Comment 83 (National Community Action
Foundation ; Eligible Entity/Local Agency in N/A)
|
|
Module 1: Module #1 –
the State Reports: We strongly concur with the full Community
Action Partnership comments regarding the state reports on local
organizational standards. The proposed design is an inappropriate
“pass/fail” format and should be modified.
|
|
Comment 84 (National Community
Partnership ; Other/Unknown in ND)
|
|
Module 1:
Module
1, Section B: (page # 5-6)
· Question B7
Summary Analysis. As written, this question exceeds the capacity
of many state CSBG Lead Agencies (grantees). This question will
require information be collected that goes beyond what is being
collected through the ROMA data included in this draft annual
report. The estimated time burden in the federal register did not
assume this additional data collection. This is added reporting
burden on State CSBG Lead Agencies because as written, the
questions require the agencies to go beyond merely reporting the
data and conduct sophisticated data analysis that will require
additional time and may exceed the capacity of agency staff to
provide meaningful information.
Module 1, Section D:
(page # 8-9)
·
D2. This question is too narrow in its potential responses and
decreases the capacity of the state fulfill its duties and impacts
both the quality of the data collected and the utility of the
data. The single metric sought is the number of entities that met
all state-adopted organizational standards; this is a poor metric
when used in isolation of performance at a variety of levels. No
one and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
·
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
Module
1, Section G: (page # 17-19)
· G3b. This
question would not be able to be answered with the ROMA data being
collected as proposed. This will increase the burden on states to
report.
Module 1, Section I: (page # 22)
·
I4. This question seeks information regarding ROMA Goals. As the
ROMA Goals are not listed in one module, it noted here to
re-instate the current ROMA Goal: Agencies increase their capacity
to achieve results. It is unfortunate there is not a module or
section number that requests feedback on the data collection
formatted around the set of new national ROMA goals.
|
|
Comment 86 (New England Community
Action Partnership (NE cap) ; State Association in CT)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22)
· Module 1,
Section B: (page # 5-6) o Question B7 Summary Analysis. As
written, this question exceeds the capacity of many state CSBG
Lead Agencies (grantees). This question will require information
be collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful information.
·
Module 1, Section D: (page # 8-9) o D2. This question is too
narrow in its potential responses and decreases the capacity of
the state fulfill its duties and impacts both the quality of the
data collected and the utility of the data. The single metric
sought is the number of entities that met all state-adopted
organizational standards; this is a poor metric when used in
isolation of performance at a variety of levels. No one and no
agency will score 100% of anything 100% of the time. Ongoing
perfection is an unreasonable expectation and is inappropriate to
be used as the single reporting measure. This question should be
changed to allow states to report on how many entities met 100%,
90%, 80%, 70%, 60% and less than 50% of the Standards. This will
increase the performance of states by allowing them to focus their
limited training and technical assistance resources on those
entities not meeting multiple standards. It would also improve the
quality of data as it will provide a more robust assessment of
performance.
O
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
·
Module 1, Section G: (page # 17-19) o G3b. This question would not
be able to be answered with the ROMA data being collected as
proposed. This will increase the burden on states to report.
· Module 1,
Section I: (page # 22) o I4. This question seeks information
regarding ROMA Goals. As the ROMA Goals are not listed in one
module, it noted here to re-instate the current ROMA Goal:
Agencies increase their capacity to achieve results. It is
unfortunate there is not a module or section number that requests
feedback on the data collection formatted around the set of new
national ROMA goals.
|
|
Comment 87 (New York Department Of
Youth And Community Development ; State in NY)
|
|
Module 1: Module 1:
1.
We recommend HHS allow State agencies to report on movement
towards 100% compliance with national Organizational Standards
(e.g. 70%, 80&, 90%) as opposed to only those meeting 100% of
the Standards
|
|
Comment 88 (New York State Community
Action Association; Eligible Entity/Local Agency in NY)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22) Module 1, Section D: (page # 8-9)
·
The sole metric of the number of eligible entities that met all of
the Organizational Standards is not useful in assessing the
performance among agencies or of one agency’s improvement
over time. The 100% requirement requires perfection or suggests
that Organizational Standards reflect a low level of minimally
acceptable performance. This not only in unreasonable, it does not
inform the development of effective training and technical
assistance (T/TA) plans. States should be able to report
performance at a number of thresholds below 100% to help focus
T/TA efforts and to improve the quality of the data as it related
to performance assessment.
· Question D3
needs to be reviewed for consistency with OCS’ IM 138 which
identifies the Technical Assistance Plan (TIP) as the preferred
response if an eligible entity does not meet all of the standards.
D3 should also affirm IM 138’s recommendation that a state
should not take action to reduce funding if an agency is making
progress toward meeting the standards. Module 1, Section I: (page
# 22)
· Question I4 addresses ROMA goals and
although it is not specifically addressed here, as noted above we
strongly recommend that the Agency Capacity goal be reinstated. A
high quality Community Action network rests on a foundation of
high performing agencies.
|
|
Comment 89 (Ninth District
Opportunity, Inc.; Eligible Entity/Local Agency in GA)
|
|
Section A
No
comment, Content acceptable
Module 1, Section B:
No comment, Content acceptable
Module 1, Section C:
No comment, Content acceptable
Module 1, Section D:
No comment, Content acceptable
Module 1, Section E:
Question E3a is especially important, as some states have been
holding allocations until late in the Fiscal Year, making it
difficult for agencies to spend within appropriate
timelines.
Module 1, Section F:
No comment,
Content acceptable
Module 1, Section G:
No
comment, Content acceptable
Module 1, Section H:
No comment, Content acceptable
Module 1, Section I:
If states are expected to provide oversight and support to
agencies, they should be held accountable for having access to a
credentialed NCRI/NCRT to provide guidance (as agencies are) in
order to demonstrate their understanding of these concepts. Also,
documentation of participation of ROMA training should be required
for all state office staff in this section.
|
|
Comment 91 (North Dakota Community
Action ; State Association in ND)
|
|
Module 1: Module I, Section G:
(page # 17-19)
• G3b. This question would not be
able to be answered with the ROMA data being collected as
proposed. This will increase the burden on states to
report.
Module I, Section I: (page # 22)
14. This question
seeks information regarding ROMA Goals. As the ROMA Goals are not
listed in one module, it noted here to re-instate the current ROMA
Goal: Agencies increase their capacity to achieve results. It is
unfortunate there is not a module or section number that requests
feedback on the data collection formatted around the set of new
national ROMA goals.
|
|
Comment 92 (North Dakota Department
Of Commerce; State in ND)
|
|
Module 1: Section B: (page #
5-6)
· As written, this question exceeds the
capacity of many state CSBG Lead Agencies (grantees). This
question will require information be collected that goes beyond
what is being collected through the ROMA data included in this
draft annual report. The estimated time burden in the federal
register did not assume this additional data collection. This is
added reporting burden on State CSBG Lead Agencies because as
written, the questions require the agencies to go beyond merely
reporting the data and conduct sophisticated data analysis that
will require additional time and may exceed the capacity of agency
staff to provide meaningful information.
Module 1,
Section D: (page # 8-9)
· This question is too
narrow in its potential responses and decreases the capacity of
the state fulfill its duties and impacts both the quality of the
data collected and the utility of the data. The single metric
sought is the number of entities that met all state-adopted
organizational standards; this is a poor metric when used in
isolation of performance at a variety of levels. No one and no
agency will score 100% of anything 100% of the time. Ongoing
perfection is an unreasonable expectation and is inappropriate to
be used as the single reporting measure. This question should be
changed to allow states to report on how many entities met 100%,
90%, 80%, 70%, 60% and less than 50% of the Standards. This will
increase the performance of states by allowing them to focus their
limited training and technical assistance resources on those
entities not meeting multiple standards. It would also improve the
quality of data as it will provide a more robust assessment of
performance.
· This question needs to be
re-worked in order to manage the burden of effort on states and
eligible entities. The current language conveys that a Quality
Improvement Plan (QIP) is the preferred option if an eligible
entity has one or more unmet organizational standards. OCS IM 138
clearly articulates the Technical Assistance Plan (TAP) is the
preferred option. This section should also reference IM 138’s
statement that as long as an eligible entity is making progress
towards meeting the standards, the state should not take action to
reduce or eliminate funding.
|
|
Comment 93 (North Dakota Lead Agency;
State in ND)
|
|
Module 1: CSBG Annual Report
OMB Clearance Process- 33
1. Whether the proposed
collection of information is necessary for the proper performance
of the functions of the agency, including whether the information
shall have practical utility
· While there
are elements that may be helpful, there are a number of areas
detailed below where the collection of information does not meet
the threshold of necessity or utility. The package as proposed in
its entirety is overly burdensome, contains reporting tools that
will provide useless information, and is outside the reach of the
Community Services Block Grant (CSBG).
· The
burden on local agencies is not impacted by the implementation of
the Online Data Collection system (OLDC). To comply with the draft
CSBG Annual Report local agencies will need to modify existing
systems.
2. The accuracy of the agency’s estimate
of the burden of the proposed collection of information
· As a State grantee, we do not have the funding or
capacity for the most part, to create systems or support local
agencies to develop systems that will streamline or decrease the
burden or tripling the amount of data to be collected.
·
In regards to the changes to Module 4, many agencies may either
lack management information systems with the capacity to
disaggregate this type of data from its overall customer database,
or struggle to integrate the required data given their use of
multiple (and sometimes incompatible) reporting systems.
·
The current IS survey includes 300+ data elements to report. The
proposed CSBG Annual Report includes more than 1,000. This
increased burden of reporting comes with no new resources.
·
Many agencies lack management information systems with the
capacity to provide the required data. Data fields required for
ROMA Next Generating reporting may not be mirrored in those used
by agencies. In addition, many agencies use multiple (and
sometimes incompatible) reporting systems which adds to the time
necessary to provide the data.
3. The quality,
utility, and clarity of the information to be collected; and
·
Regarding the stability indicators selected and included in the
clearance package (Module 4), they are not helpful or practical
when reporting customers stabilized as a result of the eligible
entity’s work. Tracking clients after they have left the
program would be overly cumbersome and many individuals may be
difficult to contact. This has the potential to consume a large
amount of agency resources.
· Module 3, Section
A: Many of the indicators used here are inappropriate for
reporting by local sub-grantees to States and by States to OCS and
will produce data of limited utility. Several challenges
underscore this point. First, community-level data on indicators
is not always available at the level (e.g. regional or zip
code/census tract) or population (e.g. low-income children aged
0-5) that matches the area and population targeted by the agency.
Second, community-level indicators are influenced by a wide number
of sources that would most likely overwhelm the impact of even
successful community initiatives, a fact that significantly limits
their utility for providing meaningful information about outcomes.
Third, any meaningful data about the outcomes of community-level
initiatives would require a rigorous program evaluation and could
not be determined on the data collected through the Annual Report
alone.
4. Ways to minimize the burden of the
collection of information on respondents, including through the
use of automated collection techniques or other forms of
information technology.
· Module 4: Remove the
Characteristics for NEW Individuals and Households Report. This
report has no practical utility at the local level given the time
and expense to create it at each local agency.
·
OCS’ Dear Colleague Letter of June 17, 2016, states, “Aware
of the reporting burden the proposed Annual Report will entail,
OCS will decrease that burden by establishing an on-line automated
system for use by the states and – at the discretion of the
states – the local agencies, for reporting that allows in
many instances auto-populating data from one year to the next and
a variety of data entry processes (e.g. manual entry and automatic
upload).” It is important to note, this online automated
system may benefit State Lead Agencies (grantees), but will most
likely not benefit local Community Action Agencies (sub-grantees).
Sub-grantees (local agencies) will still need systems that track
customers, services, outcomes, and given the requirement of many
funding sources (including other federal Department of Health and
Human Services funding) requires additional and sometime separate
data collection systems. State grantees do not have the funding or
capacity for the most part, to create systems that will streamline
or decrease the burden outlined here. Furthermore, new state-level
systems will do little to address the needs of local agencies for
improved management information systems with the capacity to
collect and report the data required by ROMA Next
Generation.
Module 1 - State Administration: (pages
2-22)
Module 1, Section B: (page # 5-6)
·
As written, this question exceeds the capacity of many state CSBG
Lead Agencies (grantees). This question will require information
be collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful
information.
Module 1, Section D: (page # 8-9)
·
This question is too narrow in its potential responses and
decreases the capacity of the state fulfill its duties and impacts
both the quality of the data collected and the utility of the
data. The single metric sought is the number of entities that met
all state-adopted organizational standards; this is a poor metric
when used in isolation of performance at a variety of levels. No
one and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
·
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
|
|
Comment 96 (Northern Tier Community
Action Corp. ; Eligible Entity/Local Agency in PA)
|
|
Module 1: No Comment
|
|
Comment 98 (NORWESCAP; Eligible
Entity/Local Agency in NJ)
|
|
Module 1: No Comment
|
|
Comment 100 (Nothing Compares;
Other/Unknown in NC)
|
|
Module 1: Module 1, Section B:
(page # 5-6)
B.2 and B.3 States have not received
adequate training and guidance to address the intended reporting
requirements and use of the information gathered from the 1st time
implementation of the ACSI. OCS must provide for additional TA if
information gathered in B2 and B3 is to be beneficial.
B.7
should not be mandated but rather serve as a framework for in
which state level performance data could be analyzed to inform the
work of the state. Module I, Section D: (page # 8-9)
D.2
should not be limited to the collection of eligible entities
meeting 100% of the organizational standards but should instead
allow states to report on how many entities met various
percentages or ranges (i.e. 100%, 85-99%, 70-84%, less than 50%).
This likely will increase the performance of states by allowing
them to further analyze and focus their limited training and
technical assistance resources on entities not meeting multiple
standards.
D.3 OCS must provide for additional TA if
information gathered under D.3 is to be beneficial as guidance
standardizing the development of TAPs and QIPs has not been made
available and as such practices throughout the Network will be
inconsistent. The standardization of TAPs and QIPs is
necessary.
Module 1, Section E: (page # 10-14)
E.2,
E.3, and E.7 Currently, states are allowed to determine their
reporting period. Items E2, E3, and E7 no longer allow this and
require information reported to be based on the federal fiscal
year. This requirement will cause challenges to states with fiscal
years (reporting periods) other than the federal fiscal year. This
requirement also conflicts with Module 2, Section A which allows
reporting based on time periods other than the federal fiscal
year.
Module I, Section H: (page # 20-21)
H.4
and H.5 OCS must provide for additional TA if information gathered
under H.4 and H.5 is to be beneficial as guidance standardizing
the development of QIPs has not been made available and as such
practices throughout the Network will be inconsistent. The
standardization of QIPs is necessary.
Module I, Section
I: (page # 22)
1.3 is duplicative and should be
removed. The work states are required to complete through review
of Organizational Standard Category 9 Data and Analysis
(specifically Standard 9.3 "The organization has presented to
the governing board for review or action, at least within the past
12 months, an analysis of the agency's outcomes and any
operational or strategic program
adjustments and improvements
identified as necessary" meets the purpose of this item.
1.4 is poorly worded
and does not maximize the reporting structure. Consider the
following revision — "Has the state provided each
eligible entity written feedback regarding the data collected in
the CSBG annual report?"
|
|
Comment 101 (Ohio Association Of
Community Action Agencies; Other/Unknown in OH)
|
|
Module 1: No Comment
|
|
Comment 102 (Opportunities For
Otsego, Inc. ; Other/Unknown in NY)
|
|
Module 1: Module 1, Section B:
· Question B7 Summary Analysis. As written,
this question exceeds the capacity of many state CSBG Lead
Agencies. This question will require information be collected that
goes beyond what is being collected through the ROMA data included
in this draft annual report. The estimated time burden in the
federal register did not assume this additional data collection.
This is added reporting burden on State CSBG Lead Agencies because
as written, the questions require the agencies to go beyond merely
reporting the data and conduct sophisticated data analysis that
will require additional time and may exceed the capacity of agency
staff to provide meaningful information.
Module 1,
Section D:
· D2. This question is too narrow in
its potential responses and decreases the capacity of the state to
fulfill its duties and impacts both the quality of the data
collected and the utility of the data. The single metric sought is
the number of entities that met all state-adopted organizational
standards; this is a poor metric when used in isolation of
performance at a variety of levels. No one and no agency will
score 100% of anything 100% of the time. Ongoing perfection is an
unreasonable expectation and is inappropriate to be used as the
single reporting measure. This question should be changed to allow
states to report on how many entities met 100%, 90%, 80%, 70%, 60%
and less than 50% of the Standards. This will increase the
performance of states by allowing them to focus their limited
training and technical assistance resources on those entities not
meeting multiple standards. It would also improve the quality of
data as it will provide a more robust assessment of performance.
· This question needs to be re-worked in order
to manage the burden of effort on states and eligible entities.
The current language conveys that a Quality Improvement Plan (QIP)
is the preferred option if an eligible entity has one or more
unmet organizational standards. OCS IM 138 clearly articulates the
Technical Assistance Plan (TAP) is the preferred option. This
section should also reference IM 138’s statement that as
long as an eligible entity is making progress towards meeting the
standards, the state should not take action to reduce or eliminate
funding.
Module 1, Section G:
·
G3b. This question would not be able to be answered with the ROMA
data being collected as proposed. This will increase the burden on
states to report.
Module 1, Section I:
·
This question seeks information regarding ROMA Goals. As the ROMA
Goals are not listed in one module, it noted here to re-instate
the current ROMA Goal: Agencies increase their capacity to achieve
results. It is unfortunate there is not a module or section number
that requests feedback on the data collection formatted around the
set of new national ROMA goals.
|
|
Comment 103 (Oregon Housing And
Community Services ; Other/Unknown in OR)
|
|
Module 1: No Comment
|
|
Comment 104 (People Incorporated Of
Virginia; Other/Unknown in VA)
|
|
Module 1: Module 1, Section C:
(page # 7)
No comment.
Module 1, Section
D: (page # 8-9)
The request for States to identify the
number of agencies that met all the Organizational Performance
Standards is too narrow. This question should be changed to allow
states to report on how many entities met 100%, 90%, 80%, 70%, 60%
and less than 50% of the Standards. This will increase the
performance of states by allowing them to focus their limited
training and technical assistance resources on those entities not
meeting multiple standards, and reflect increases in positive
performance. It would also improve the quality of data as it will
provide a more robust assessment of performance.
Module
1, Section E: (page # 10-14)
No comment.
Module
1, Section F: (page # 15-16)
No comment.
Module
1, Section G: (page # 17-19)
G3b. This question would
not be able to be answered with the ROMA data being collected as
proposed. This will increase the burden on states to report.
Module 1, Section H: (page # 20-21)
No
comment.
Module 1, Section I: (page # 22)
I4.
This question seeks information regarding ROMA Goals. As the ROMA
Goals are not listed in one module, it noted here to re-instate
the current ROMA Goal: Agencies increase their capacity to achieve
results. It is unfortunate there is not a module or section number
that requests feedback on the data collection formatted around the
set of new national ROMA goals.
|
|
Comment 105 (ProAction Of Steuben And
Yates Inc.; Eligible Entity/Local Agency in NY)
|
|
Module 1: State
Administration: (pages 2-22)
Module 1, Section B:
(page # 5-6)
· Question B7 Summary Analysis. As
written, this question exceeds the capacity of many state CSBG
Lead Agencies (grantees). This question will require information
be collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful
information.
Module 1, Section D: (page # 8-9)
·
D2. This question is too narrow in its potential responses and
decreases the capacity of the state fulfill its duties and impacts
both the quality of the data collected and the utility of the
data. The single metric sought is the number of entities that met
all state-adopted organizational standards; this is a poor metric
when used in isolation of performance at a variety of levels. No
one and no agency will score 100% of anything 100% of the time.
Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
·
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
Module
1, Section G: (page # 17-19)
· G3b. This
question would not be able to be answered with the ROMA data being
collected as proposed. This will increase the burden on states to
report.
Module 1, Section I: (page # 22) I4. This
question seeks information regarding ROMA Goals. As the ROMA Goals
are not listed in one module, it noted here to re-instate the
current ROMA Goal: Agencies increase their capacity to achieve
results. It is unfortunate there is not a module or section number
that requests feedback on the data collection formatted around the
set of new national ROMA goals.
|
|
Comment 106 (Project Bravo; Eligible
Entity/Local Agency in TX)
|
|
Module 1: No Comment
|
|
Comment 107 (Pueblo County Housing
And Human Services; Other/Unknown in CO)
|
|
Module 1: Module 1: State
Administration / No Comments
5. Whether the proposed
collection of information is necessary for the proper performance
of the functions of the agency, including whether the information
shall have practical utility
6. The accuracy of the
agency’s estimate of the burden of the proposed collection
of information
7. The quality, utility, and clarity of
the information to be collected; and
8. Ways to
minimize the burden of the collection of information on
respondents, including through the use of automated collection
techniques or other forms of information technology.
|
|
Comment 108 (Redwood Community Action
Agency ; Eligible Entity/Local Agency in CA)
|
|
Module 1: Module 1, Section D.
2
In order to better reflect the number of eligible
entities meeting the newly implemented Organizational Standards,
we recommend that there be additional
columns inserted
that allow States to also report on the number of eligible
entities that have met a certain percentage of the Organizational
Standards (i.e. 50%, 80%, 90% Met) versus only capturing if an
Eligible entity has met "All". The information gathered,
especially over a period of years, will have more practical
utility if this change were made in the proposed report.
|
|
Comment 109 (Sa Howell, LLC ;
Other/Unknown in GA)
|
|
Module 1: 1.
State
offices should have the option to report multiple benchmarks
concerning the achievement of OS compliance to demonstrate
progress over time rather than insinuating failure for those not
performing at a rate of 100%.
Many state employees are
not qualified to provide ROMA oversight, technical support, and
monitoring. State offices should be held accountable for access to
a ROMA professional (NCRT, NCRI, or Master Trainer) and document
regular participation in ROMA training, along with
agencies.
Module 1, Section B
This section
requires states to demonstrate an understanding of ROMA. States
will more than likely be required to hire better qualified staff
(data analysts) or contract these activities out. This is expected
to result in a strong, more productive network along with more
accurate data on a national level.
Module 1,
Question E3a is especially
important, as some states have been holding allocations until late
in the Fiscal year, making it difficult for agencies to spend
within appropriate timelines. Some states have even gone so far as
to modify allocations without notifying agencies and the refusing
to provide an explanation of the changes. This level of
accountability is absolutely imperative to the proper operation of
the network.
|
|
Comment 110 (SEMCAC; Eligible
Entity/Local Agency in MN)
|
|
Module 1: Section A: (page #
3-4)
· No Comments
Module 1, Section
B: (page # 5-6)
· Question B7 Summary Analysis.
As written, this question exceeds the capacity of many state CSBG
Lead Agencies (grantees). This question will require information
be collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful
information.
Module 1, Section C: (page # 7)
·
No comment
Module 1, Section D: (page # 8-9)
·
D2. This question is too narrow in its potential responses and
decreases the capacities of the state to fulfill its duties and
impacts both the quality of the data collected and the utility of
the data. The single metric sought is the number of entities that
met all state-adopted organizational standards; this is a poor
metric when used in isolation of performance at a variety of
levels. No one and no agency will score 100% of anything 100% of
the time. Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the single reporting measure. This
question should be changed to allow states to report on how many
entities met 100%, 90%, 80%, 70%, 60% and less than 50% of the
Standards. This will increase the performance of states by
allowing them to focus their limited training and technical
assistance resources on those entities not meeting multiple
standards. It would also improve the quality of data as it will
provide a more robust assessment of performance.
·
This question needs to be re-worked in order to manage the burden
of effort on states and eligible entities. The current language
conveys that a Quality Improvement Plan (QIP) is the preferred
option if an eligible entity has one or more unmet organizational
standards. OCS IM 138 clearly articulates the Technical Assistance
Plan (TAP) is the preferred option. This section should also
reference IM 138’s statement that as long as an eligible
entity is making progress towards meeting the standards, the state
should not take action to reduce or eliminate funding.
Module
1, Section E: (page # 10-14)
· No
comments
Module 1, Section F: (page # 15-16)
·
No comments
Module 1, Section G: (page # 17-19)
·
G3b. This question would not be able to be answered with the ROMA
data being collected as proposed. This will increase the burden on
states to report.
Module 1, Section H: (page # 20-21)
· No comments
Module 1, Section I:
(page # 22)
· I4. This question seeks
information regarding ROMA Goals. As the ROMA Goals are not listed
in one module, it noted here to re-instate the current ROMA Goal:
Agencies increase their capacity to achieve results. It is
unfortunate there is not a module or
section number
that requests feedback on the data collection formatted around the
set of new national ROMA goals.
|
|
Comment 111 (South Plain Community
Action Association ; State Association in TX)
|
|
Module 1: No Comment
|
|
Comment 112 (Southeastern North
Dakota ; Eligible Entity/Local Agency in ND)
|
|
Module 1: Module I, Section B:
(page # 5-6)
• Question 87 Summary Analysis. As
written, this question exceeds the capacity of many state CSBG
Lead Agencies (grantees). This question will require information
be collected that goes beyond what is being collected through the
ROMA data included in this draft annual report. The estimated time
burden in the federal register did not assume this additional data
collection. This is added reporting burden on State CSBG Lead
Agencies because as written, the questions require the agencies to
go beyond merely reporting the data and conduct sophisticated data
analysis that will require additional time and may exceed the
capacity of agency staff to provide meaningful information.
Module
1, Section D: (page # 8-9)
D2. This question is too
narrow in its potential responses and decreases the capacity of
the state fulfill its duties and impacts both the quality of the
data collected and the utility of the data. The single metric
sought is the number of entities that met all state-adopted
organizational standards; this is a poor metric when used in
isolation of performance at a variety of levels. No one and no
agency will score 100% of anything 100% of the time. Ongoing
perfection is an unreasonable expectation and is inappropriate to
be used as the single reporting measure. This question should be
changed to allow states to report on how many entities met 100%,
90%, 80%, 70%, 60% and less than 50% of the Standards. This will
increase the performance of states by allowing them to focus their
limited training and technical assistance resources on those
entities not meeting multiple standards. It would also improve the
quality of data as it will provide a more robust assessment of
performance.
This question needs to be re-worked in
order to manage the burden of effort on states and eligible
entities. The current language conveys that a Quality Improvement
Plan (QIP) is the preferred option if an eligible entity has one
or more unmet organizational standards. OCS 1M 138 clearly
articulates the Technical Assistance Plan (TAP) is the preferred
option. This section should also reference 1M 138's statement that
as long as an eligible entity is making progress towards meeting
the standards, the state should not take action to reduce or
eliminate funding.
Module 1, Section G: (page # 17-19)
G3b. This question would
not be able to be answered with the ROMA data being collected as
proposed. This will increase the burden on states to
report.
Module I, Section I: (page # 22)
•
14. This question seeks information regarding ROMA Goals. As the
ROMA Goals are not listed in one module, it noted here to
re-instate the current ROMA Goal: Agencies increase their capacity
to achieve results. It is unfortunate there is not a module or
section number that requests feedback on the data collection
formatted around the set of new national ROMA goals.
|
|
Comment 113 (Southern New Hampshire
Services ; Eligible Entity/Local Agency in NH)
|
|
Module 1: No Comment
|
|
Comment 114 (Southwest Oklahoma
Community Action ; Other/Unknown in OK)
|
|
Module 1: No Comment
|
|
Comment 115 (State Of North Carolina;
State in NC)
|
|
Module 1: Module 1 State
Administration Module 1, Section B: (page # 5-6)
B.2
and B.3 States have not received adequate training and guidance to
address the intended reporting requirements and use of the
information gathered from the 1st time implementation of the
ACSI. OCS must provide for additional TA if information gathered
in B2 and B3 is to be beneficial.
B.7 should not be
mandated but rather serve as a framework for in which state level
performance data could be analyzed to inform the work of the
state.
Module I, Section D: (page # 8-9)
D.2
should not be limited to the collection of eligible entities
meeting 100% of the organizational standards but should instead
allow states to report on how many entities met various
percentages or ranges (i.e. 100%, 85-99%, 70-84%, less than 50%).
This likely will increase the performance of states by allowing
them to further analyze and focus their limited training and
technical assistance resources on entities not meeting multiple
standards.
D.3 OCS must provide for additional TA if
information gathered under D.3 is to be beneficial as guidance
standardizing the development of TAPs and QIPs has not been made
available and as such practices throughout the Network will be
inconsistent. The standardization of TAPs and QIPs is
necessary.
Module 1, Section E: (page # 10-14)
E.2,
E.3, and E.7 Currently, states are allowed to determine their
reporting period. Items E2, E3, and E7 no longer allow this and
require information reported to be based on the federal fiscal
year. This requirement will cause challenges to states with fiscal
years (reporting periods) other than the federal fiscal year. This
requirement also conflicts with Module 2, Section A which allows
reporting based on time periods other than the federal fiscal
year.
Module
I, Section H: (page # 20-21)
H.4 and H.5 OCS must
provide for additional TA if information gathered under H.4 and
H.5 is to be beneficial as guidance standardizing the development
of QIPs has not been made available and as such practices
throughout the Network will be inconsistent. The standardization
of QIPs is necessary.
Module I, Section I: (page #
22)
1.3 is duplicative and should be removed. The work
states are required to complete through review of Organizational
Standard Category 9 Data and Analysis (specifically Standard 9.3
"The organization has presented to the governing board for
review or action, at least within the past 12 months, an analysis
of the agency's outcomes and any operational or strategic
program
adjustments and improvements identified as necessary"
meets the purpose of this item.
1.4 is poorly worded and
does not maximize the reporting structure. Consider the following
revision — "Has the state provided each eligible entity
written feedback regarding the data collected in the CSBG annual
report?"
|
|
Comment 116 (Texas Association Of
Community Action Agencies, Inc. (Tacaa); Other/Unknown in TX)
|
|
Module 1: No Comment
|
|
Comment 118 (The State Of New York
Community Action Association ; State Association in NY)
|
|
Module 1: Module 1 - State
Administration: (pages 2-22) Module 1, Section D: (page # 8-9)
·
The sole metric of the number of eligible entities that met all of
the Organizational Standards is not useful in assessing the
performance among agencies or of one agency’s improvement
over time. The 100% requirement requires perfection or suggests
that Organizational Standards reflect a low level of minimally
acceptable performance. This not only in unreasonable, it does not
inform the development of effective training and technical
assistance (T/TA) plans. States should be able to report
performance at a number of thresholds below 100% to help focus
T/TA efforts and to improve the quality of the data as it related
to performance assessment.
· Question D3
needs to be reviewed for consistency with OCS’ IM 138 which
identifies the Technical Assistance Plan (TIP) as the preferred
response if an eligible entity does not meet all of the standards.
D3 should also affirm IM 138’s recommendation that a state
should not take action to reduce funding if an agency is making
progress toward meeting the standards. Module 1, Section I: (page
# 22)
· Question I4 addresses ROMA goals and
although it is not specifically addressed here, as noted above we
strongly recommend that the Agency Capacity goal be reinstated. A
high quality Community Action network rests on a foundation of
high performing agencies.
|
|
Comment 119 (TN Department Human
Services ; State in TN)
|
|
Module 1 - State
Administration: (pages 2-22)
Module
1, Section B: (page # 5-6): Minimal guidance has been provided on
exactly what is expected through the State Accountability
Measures. OCS provided a grid, which one could barely read due to
the size of the print, along with some initial feedback during the
implementation of the Organizational Standards of what exactly the
State Accountability Measures were. Little training or technical
assistance was provided on what would truly be expected of
agencies to meet each of these standards. Then OCS implemented the
Model State Plan with minimal time to train and engage eligible
entities in the process before having to submit the Model State
Plan, and again, our Department along with agencies were given
little notice of the ACSI Process, leaving little time to engage
the eligible entities. All this together has affected our overall
results. It does not seem fair to our Department to be judged upon
these results, across the board, when the results themselves are
not even considered reliable. I think it would be prudent to allow
the State Lead Agency to include additional data regarding the six
(6) drivers of satisfaction and the factors that may have led to
these responses.
Module 1, Section D:
(page # 8-9): It seems like the data requested in D.2 should be
submitted and calculated individually based upon each eligible
entities’ results. If Agency A, B, & C all met the
standards, but Agency D only met 55% of the standards it does not
seem to be a fair or accurate reflection upon those agencies who
are meeting the standards versus those who are not.
|
|
Comment 120 (Tompkins Community
Action ; Eligible Entity/Local Agency in NY)
|
|
Module 1: No Comment
|
|
Comment 121 (Total Action For
Progress ; Other/Unknown in VA)
|
|
Module 1: No Comment
|
|
Comment 123 (Tri County Community
Action ; Other/Unknown in MN)
|
|
Module 1: No Comment
|
|
Comment 124 (U.S Department Of Health
And Human Service ; Eligible Entity/Local Agency in DC)
|
|
Module 1: Please note that the
following three sections of Module I that we are commenting on
were added to the proposed CSBG Annual Report since OCS' initial
comment period. So, this is the first time anyone in the CSBG
Network has been provided with the opportunity to respond to these
new requirements.
Section B: Summary Analysis (B. 7)
These analysis
requirements would exceed our State CSBG Office's capacity to
perform; new administrative resources and training and technical
assistance regarding data analysis and procedures would be needed.
These analysis requirements would also contribute to our burden
cost estimate that we described earlier in question #2 under OCS
REQUESTED FEEDBACK.
Further, while this section offers
examples of analysis questions, these questions should not be
required, as implied by the language "should be provided"
in the first paragraph. In addition, the language in the final
bullet that refers to "changes the State will encourage and
support with local eligible entities" can be interpreted as a
potential intrusion of state authority upon statutorily mandated
local control and should be clarified or removed.
Section
l: State and Eligible Entity Analysis of Data (1.3)
This
requirement would likely exceed our State CSBG Office's capacity
to perform; new administrative resources and training and
technical assistance regarding data analysis and procedures would
be needed. This requirement would also contribute to our burden
cost estimate that we described earlier in question #2 under OCS
REQUESTED FEEDBACK.
Section I: State Feedback on Data
Collection, Analysis, and Reporting (1.4)
This
requirement for State CSBG Offices would contribute to our burden
cost estimate that we described earlier in question #2 under OCS
REQUESTED FEEDBACK. It would also be of questionable utility for
both the State CSBG Office and the eligible entities. Our eligible
entities track, collect, and report their own performance measures
and are the appropriate entity to analyze their data. This
question should be removed.
|
|
Comment 125 (Vermont Community Action
Partnership ; Other/Unknown in VT)
|
|
1. Module 1: Modify the
options State CSBG Lead Agencies and grantees have when reporting
to OCS on CSBG Organizational Standards, to allow for a broader
range of responses. As noted below, there are significant
challenges with the way this information is being reported that
misinterprets and misuses the Organizational Standards. Expecting
constant perfection is unrealistic and does not have practical
utility for either local agencies or the State CSBG Lead Agency
(LAs). As proposed, the single measure the State Agencies will
report on is the number of Eligible Entities (EEs) that meet 100%
of the CSBG Organizational Standards. To make this measure
practical and realistic, and to increase the usefulness of the
information, LAs should have the option to report the number of
agencies at a range of thresholds that still indicate strong
performance as well as allow for a more accurate reflection of the
EEs’ training and technical assistance needs. For example,
reporting the number of local agencies that have met 100%, 90%,
80%, 70%, 60%, 50%, and less than 50% of the standards will allow
for the collection of more accurate information, better tracking
of results over time, and better, more efficient investment of
training and technical assistance resources.
Module 1:
Module 1 - State Administration: (pages 2-22)
Module
1, Section D: (page # 8-9)
1. D2 – This question
is too narrow in its potential responses and decreases the
capacity of the state to fulfill its duties and impacts both the
quality of the data collected and the utility of the data. The
single measure required is the number of EEs that met all
state-adopted organizational standards; this is a poor measure
when used in isolation of performance at a variety of levels. Very
few if any agencies will likely score 100% of anything 100% of the
time. Ongoing perfection is an unreasonable expectation and is
inappropriate to be used as the only reporting measure. This
should be changed to allow states to report on how many entities
met 100%, 90%, 80%, 70%, 60% and less than 50% of the Standards.
This will increase the performance of states by allowing them to
focus their limited training and technical assistance resources on
those entities not meeting multiple standards. It would also
improve the quality of data as it will provide a more robust
assessment of performance.
2. This question needs to
be re-worked in order to manage the burden of effort on states and
eligible entities. The current language conveys that a Quality
Improvement Plan (QIP) is the preferred option if an eligible
entity has one or more unmet organizational standards. OCS IM 138
clearly articulates that the Technical Assistance Plan (TAP) is
the preferred option. This section should also reference IM 138’s
statement that as long as an eligible entity is making progress
towards meeting the standards, the state should not take action to
reduce or eliminate funding.
Module 1, Section G:
(page # 17-19)
G3b – This question would not be
able to be answered with the ROMA data being collected as
proposed. This will increase the burden on states to report.
Module 1, Section I: (page # 22)
I4 –
This question seeks information regarding ROMA Goals. Since the
ROMA Goals are not listed in a single module, we are reiterating
here our request to re-instate the current ROMA Goal of “Agencies
Increase Their Capacity to Achieve Results”. OCS should have
structured a module or section number to request feedback on the
data collection formatted around the set of new national ROMA
goals.
|
|
Comment 126 (Vernon Community Action
Council, Inc.; Eligible Entity/Local Agency in LA)
|
|
Module 1: No Comment
|
|
Comment 127 (Virginia Community
Action Partnership ; State Association in VA)
|
|
Module 1:
Regarding Module 1,
it appears that the only metric reflecting compliance with
Organizational Standards is reporting on the number of eligible
entities that meet 100% of the organizational standards. This is
of limited usefulness and we believe that state agencies should
have the option to report the number of agencies achieving various
levels of meeting the standards (100%, 90%, 80%, 70%, 60%, 50% or
less than 50% of the standards). This will assist both in the
collection of more useful information and better reflect the need
for specific training and technical assistance.
|
|
Comment 129 (Western New York
Community Action Agency; Eligible Entity/Local Agency in NY)
|
|
Module 1: No Comment
|
|
Comment 130 (Wisconsin Community
Action Program Association ; State Association in WI)
|
|
Module 1: The current draft
nearly triples the amount of required information data points.
There are over 1,100 items for local CAA reports (and even more
when counting other items), as opposed to the 374 items in the
current system. This means that CAAs will be forced to contribute
on many of the new state data items; agencies will have to
contribute because states must now investigate and report
narrative details on local agency achievement of organizational
standards
|
|
Comment 131 (Wisconsin Department Of
Children And Families; Eligible Entity/Local Agency in WI)
|
|
Section B: (page # 5-6)
p.
6 – B.7 Summary Analysis: The first six bullet points all
seem like they will be easy for states to answer and will provide
useful information.
The last two bullet points appear
to place new expectations on the State office that are not
included in the State Accountability Measures. They may be
problematic on two levels: 1) they would increase the workload for
our state’s CSBG administration, which is currently handled
by one staff person; and 2) our state’s network of eligible
entities values local control and is not looking to the State for
directives on their programming decisions.
“Based
on analysis, what services, strategies, or administrative
practices were identified to be shared across the State?”
The passive voice makes it confusing. Is the State CSBG office
charged with identifying best services, strategies, and
administrative practices to share across the state? Is this a new
requirement from OCS?
“Based on analysis of
performance data, what changes will the State encourage and
support with the local eligible entities?” Is this question
referring to 5.Sii? from the State Accountability Measures, which
requires States to provide individual written feedback on the each
eligible entity’s performance in meeting ROMA goals, or does
it refer to a new requirement?
Module 1, Section E:
(page # 10-14)
p. 14 - E.9b. Carryover for this Fiscal
Year: Wisconsin's CSBG contracts operate on the calendar year and
the agencies have 90 days after the end of December to close out
their contracts. Therefore, the State did not have the final data
to report at the time the Annual Report is due at the end of
March. Our office will be able to send the correct Carryover
amount during April, a few weeks after the Annual Report is
submitted.
Module 1, Section H: (page # 20-21)
p.
21 – H.6 Single Audit Review: Does “Audit Number”
mean “Report ID”?
Module 1, Section I:
(page # 22)
p. 22 – “State and Eligible
Analysis of Data: Describe how the State validated that the
eligible entities used data to improve service delivery.”
Would
this be addressed by monitoring the agencies for CSBG
Organizational Standard 9.3, which states, “The organization
has presented to the governing board for review and action, at
least within the 12 months, an analysis of the agency’s
outcomes and any operational or strategic program adjustments and
improvements identified as necessary.”?
What other
expectations would there be relating to how the State validates
that the eligible entities used data to improve service delivery?
|
|
Comment 132 (Work And Family Support
Bureau; Other/Unknown in NM)
|
|
Module 1: No Comment
|
|
Comment 133 (Wsos Community Action;
Eligible Entity/Local Agency in OH)
|
|
Module 1: No Comment
|
|
Comment 134 (York County Community
Action ; Eligible Entity/Local Agency in ME)
|
|
Module 1: * Allow State
Agencies to report on a scale of meeting the Organizational
Standards rather than expecting all organizations to achieve 100%.
|