ELI_Supporting Statement aprt A 5-16-16

Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The U.S. Small Business Administration’s (SBA) statutory mission is to "aid, counsel, assist and protect, insofar as is possible, the interests of small business concerns." The Agency’s Office of Entrepreneurial Development (OED) helps to carry out this mission by providing training and counseling programs and initiatives, such as the Emerging Leaders initiative, to existing and prospective small businesses. The Emerging Leaders initiative (EL) is a 7-month intensive training opportunity that builds upon business owners’ significant experiences by combining classroom sessions, out-of-class preparation work, and peer-mentoring groups to facilitate business growth. Since its pilot year in 2008, the program has been implemented in disadvantaged communities across the country by a third party nonprofit organization under contract with SBA (i.e. the program contractor).

Previously, the program contractor collected data from the small business’s enrolled in the training (i.e. the participants) for the purposes described in A.2 below; however, that collection of information was not OMB approved. SBA is now taking steps to rectify that violation, and with this submission requests approval of the following collections of information:

Intake survey to be collected prior to the first week of the course; (Appendix B-3);
Feedback survey to be collected at the end of the course (Appendix C-2); and
Annual follow-up surveys (Appendix D-3).

Purpose and Use of the Information Collected

Indicate how, by whom, how frequently, and for what purpose, the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

In response to the Office of Management and Budget (OMB) Memorandum M-13-17, “Next Steps in the Evidence and Innovation Agenda,” and OMB Memorandum M-14-06, “Guidance for Providing and Using Administrative Data for Statistical Purposes,” SBA strives to increase the use of data to improve its programming. The proposed evaluation will be used to track participants’ progress and provide guidance to the program contractor on areas for additional assistance (eg - such as the need for offering a special session on contracting for women owned businesses), and to increase SBA’s understanding of outcome trends across its program portfolio. The results are not used to evaluate the performance of the program contractor or the impact of the program, as the study design does not allow for rigorous conclusions regarding program impact.

The EL initiative is a cohort-based, executive training program. Key elements of EL program logic model include the following:

Target population: C-level executives operating a small business for at least 3 years, with at least 1 employee and between $400,000 and $10 million in annual revenue. While not required for participation in the initiative, the initiative encourages businesses located in low and moderate income areas to participate through targeted marketing efforts.
Inputs: 5-module course curriculum and out-of-class peer/cohort mentoring groups aimed at building business management skills and increased investment in the local community.
Output: 3-year strategic growth plan prepared by each participant that should guide them toward growing their business.
Short term outcomes: Increased use of management best practices (e.g. using financial analysis to make business decisions, identifying profitable customers, conducting employee evaluations, etc.) and increased involvement in the local community.
Long term outcomes: New contract awards, ability to obtain financing for growth, and increased revenue, profit and employment (particularly from the local community in which the business operates).

This packet requests approval for three survey instruments used to collect data from the small business participants in the Emerging Leaders initiative. The data from the Intake Survey will be used to establish baseline statistics on the business and the owner’s management practices and determine the extent to which the target population is reached. The Feedback Survey will be used to measure participant satisfaction or dissatisfaction with the initiative and as a basis to determine if implementation adjustments are necessary. SBA will use data from the annual Follow-up Surveys to track changes in the small business owner’s management practices and business outcomes over a period of three years.

Since 2010, the program contractor has collected Emerging Leaders’ participant data, which was primarily used to adjust and improve the course content. In 2012, SBA contracted with an additional third party (i.e. the evaluation contractor) to analyze the data and generate an annual, internal report on the initiative. The report documents customer satisfaction trends across cohorts and changes in the participants short and long term outcomes over the three years in which they implement their growth plans. Based on the results, SBA provides cross-site strategic guidance to the program contractor, and determines areas for continued assistance or service diversification in SBA’s overall program portfolio. Once approved by OMB, the results will also be reported to external stakeholders and to Congress in the aggregate, as part of the Agency’s performance plan.

Use of Information Technology and Burden Reduction

Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden. Insert any applicable electronic web address.

SBA is committed to complying with the E-government Act of 2002 and OMB Memorandum M-03-22, “OMB Guidance for Implementing the Privacy Provisions of the E-government Act of 2002,” to promote the use of technology to lessen the burden of data collection. All surveys are administered via an online survey platform. A web-based distribution method was chosen to reduce the burden placed on respondents. For instance, since automated skip patterns are integrated into web-based surveys, respondents do not need to spend time reading through instructions to follow skip patterns for questions that are not relevant to them. Furthermore, the web-based survey requires no more than hitting a submit button to transfer the information. Notices regarding the surveys, as well as the consent forms, will be sent electronically and provide for “electronic” signature for consent to help minimize the burden on respondents.

4. Efforts to Identify Duplication and Use of Similar Information

Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

SBA has worked with its program contractor for the Emerging Leaders initiative to ensure a unified data collection instrument that meets the information needs of both parties, without duplicative surveys. To the extent that the same question is asked in more than one of the three surveys, it is intended to measure that outcome at different points in time or to serve as a safeguard against gaps in the administrative records. For example, the feedback and follow-up surveys ask whether the respondent graduated from the EL program because the feedback survey is administered on the last day of class, which is unlikely to be attended by participants that dropped the program earlier. The economic data items to be collected via the survey (e.g. annual revenue and employment), though potentially collected by a federal agency, are not currently available in public administrative data that could be linked to Emerging Leaders participant data. All other data items collected are unique to the participant’s experience in the classroom and are not available from other sources.

Impacts on Small Businesses or Other Small Entities

If the collection of information impacts small business or other small entities, describe any methods used to minimize burden.

This data collection will obtain information from the small businesses that participated in the Emerging Leaders executive education initiative. To minimize the burden on respondents, efforts have been made to simplify and streamline the survey. A systematic analysis was undertaken to ensure that measuring the same variable over each year of the follow-up survey instruments is anticipated to produce different answers over time. The survey will be disseminated via an online platform, which will allow small business owners to take the survey at a time that is convenient for them and only answer questions that pertain to their experiences. For example, skip patterns are programmed into the platform so that businesses that have not had financing activity in the past year do not view the more detailed questions that pertain to financing.

Consequences of Collecting the Information Less Frequently

Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Each Emerging Leaders cohort will be asked to complete an intake survey to collect baseline information prior to the start of the course, a short feedback survey at the end of the course to measure customer satisfaction, and three years of identical follow-up surveys to measures changes from the baseline data. Although there are no technical or legal obstacles to reducing the burden, each survey serves a distinct purpose and is necessary to document the initiative with less measurement error than would be possible if data were collected less frequently.

Without both intake and follow-up surveys, SBA would not be able to reliably document change in the management practices of participants. Collecting the data prior to the program, rather than only after the program, helps reduce recall bias, which can occur with retrospective questions on behaviors (as well as economic data to a lesser extent). Second, without three successive years of follow-up data, the initiative’s program logic model cannot be fully assessed because each participant graduates with a three year business growth plan which may be enacted incrementally if the plan requires acquiring new capital or breaking into a new market. Moreover, three years of follow-up data is needed to provide adequate time to both smooth anomalies in annual business cycles and to more adequately document growth trends of participants. Small businesses often experience transitory changes, such as using temporary work contracts or winning an unusually large contract in one year.

Special Circumstances Relating to the Guideline of 5 CFR 1320.5

Explain any special circumstances that would cause an information collection to be conducted in a manner:

requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary, trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

SBA is requesting approval to provide less than 30 days for the participants to complete their intake and feedback surveys. Participants will receive the intake survey approximately two weeks prior to the start of their classes and the survey will close after the first week of class. It is important that they document their behaviors and perceptions before significant course content is delivered so the results can be compared to the post-course follow-up assessment and meaningful differences can be detected. However delivering the intake survey more than two weeks in advance of the class loses the immediacy that the upcoming start date creates in completing the request self-assessment. The feedback survey is administered on the last day of class. As a customer satisfaction survey, its reliability is improved by obtaining the information while the course content is still fresh and participants are invested in reflecting on their instructors and peer contacts. SBA requests that participants have two weeks to complete the feedback survey. There are no other special circumstances that would cause information to be collected in the manner described above.

Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency’s notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

The required public comment notice was published in the Federal Register on October 30, 2015 in 80:210 FR 27754. All comments were due on or before December 29, 2015. No comments on this collection were received.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.

Apart from the request for comment through the FR notice, SBA has also consulted with its contractors regarding the methodological approach for data collection and the survey instruments. Key questions on the instruments were tested with three former Emerging Leaders graduates. Some instructions and questions were refined for clarity as a result of this pretesting and are reflected in this proposed survey.

Explanation of Any Payment or Gift to the Respondents

Explain any decision to provide any payment or gift to respondents, other than re-enumeration of contractors or grantees.

No payments or gifts will be made to respondents.

Assurance of Confidentiality Provided to Respondents

Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

Public reports based on this data collection will only include aggregate statistics on participants and their business outcomes. The personal information collected will be protected to the extent permitted by law, including the Privacy Act of 1974 and the Freedom of Information Act (FOIA). In addition, to the extent permitted under FOIA, confidential business information will only be disclosed to contractor or Agency personnel assigned to work on the EL initiative courses and surveys. The program contractor collects and compiles data from Emerging Leaders participants and stores the data in password protected systems using industry-standard protocols such as firewalls, virus protection, and user authentication mechanisms to ensure the privacy of personal data. The data is collected through a secure online survey system that is also password protected and accessible to the program contractor and SBA. Only de-identified data are transferred to the evaluation contractor for analysis.

Justification for Sensitive Questions

Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

In this data collection, initiative participants will be asked about their businesses’ financial situation, including revenue and financing obtained. This information is essential to the mission of the initiative and to monitoring business performance. The information will be used to report on growth of participants over a three year period.

The intake survey instrument also include questions about race, ethnicity, veteran status, and disability in order to determine whether Emerging Leaders is fulfilling its mission of serving a diversity of entrepreneurs, including populations that are under-represented among business owners. The questions were written in compliance with the OMB Standards for the Classification of Federal Data on Race and Ethnicity.

Participants will also be asked for their business identification numbers (i.e. EIN, DUNS). These data are necessary to improve data validity and reduce survey burden in future years of the study or in extended follow-up analysis on current participants. Unique identifiers will allow SBA to link Emerging Leaders participants to the data it houses on loans and contracts. Given the sensitive nature of these identifiers, EIN in particular, the data will be transferred and maintained according to a data security protocol that restricts access to user with specific SBA-assigned privileges.

Participants will be provided with a consent form (Appendix B-2) that will describe how the information they provide will be handled. (The consent form is not distributed with the feedback survey, as that survey requires only customer satisfaction data).

Estimates of Hour Burden, Including Annualized Hourly Costs

Provide estimates of the hour burden of the collection of information. Indicate the affected public, number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated.

SBA estimates that 1,340 hours are needed each year to conduct this data collection. The total annualized hour and costs burdens are set out below in Exhibit 1. Given that cohorts run concurrently, 3,474 small businesses are estimated to receive a survey each year (either intake and feedback or one of the three follow-up surveys). Preliminary pre-tests were conducted for all data collection instruments to derive the hourly burden estimates (see part B for more information on pre-testing and estimated response rates). The cost burden of $33,505 is based on an estimated median annual income of $66,500 for small business owners, which equates to an hourly rate of $25.¹

Exhibit 1. Study burden estimates

Estimates	Number of small businesses	Frequency of collection	Average minutes spent	Small business burden (in dollars)	Small business burden (in hours)
Intake survey
Introduction email	960	1	2	$800	32
Completed survey	960	1	17	$6,800	272
Consent	960	1	2	$800	32
Reminders	960	1	1	$400	16
Thank you	960	1	1	$400	16
Feedback survey
Introduction email	960	1	1	$400	16
Completed survey*	838	1	9	$3,143	126
Thank you*	838	1	0.5	$175	7
Cohort 4 burden	960			$12,918	517
First follow-up survey
Invitation	838	1	2	$698	28
Introduction email	838	1	1	$349	14
Consent	838	1	2	$698	28
Reminders	838	4	1	$1,397	56
Completed survey*	541	1	25	$5,632	225
Thank you*	541	1	1	$225	9
Cohort 3 burden	838			$9,000	360
Second follow-up survey
Invitation	838	1	2	$698	28
Introduction email	838	1	1	$349	14
Consent	838	1	2	$698	28
Reminders	838	4	1	$1,397	56
Completed survey*	324	1	25	$3,379	135
Thank you*	324	1	1	$135	5
Cohort 2 burden	838			$6,657	266
Third follow-up survey
Invitation	838	1	2	$698	28
Introduction email	838	1	1	$349	14
Consent	838	1	2	$698	28
Reminders	838	4	1	$1,397	56
Completed survey*	165	1	25	$1,718	69
Thank you*	165	1	1	$69	3
Cohort 1 burden	838			$4,929	197
Total annual burden given concurrent cohorts	3,474			$33,505	1,340

* Estimates of the response rate for each survey are discussed in Part B.

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information.

There is no other annual cost burden to respondents or record keepers.

14. Annualized Costs to Federal Government

Provide estimates of annualized cost to the federal government. Provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The annualized cost to the federal government is $142,894. This includes the costs associated with the program contractor conducting the data collection and the evaluation contractor conducting the data analysis and report.

15. Explanation for Program Changes or Adjustments

Explain the reasons for any program changes or adjustments reported in items 13 or 14 of OMB form 83-I.

There are no program changes or adjustments to report; this information collection has not been reported previously.

16. Plans for Tabulation and Publication and Project Time Schedule

For collections of information whose results are planned to be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

This study will use quantitative (i.e., descriptive and inferential statistical techniques) methods for analyzing the data. The analysis is expected to include tables for the statistics and potentially graphs. A full report will be produced that includes descriptive statistics and the appropriate associated statistics, such as the p-value at the 10 percent, 5 percent, and 1 percent levels for identifying statistically significant changes over time. The final internal report will be released approximately 6 months after the data collection is complete. In addition, the first follow-up data for each cohort and select metrics from their intake (e.g. demographic distribution, business age) and feedback surveys (e.g. satisfaction) will be presented in a performance measurement report. A collection and reporting schedule for the project is in Appendix A-1.

17. Reason(s) Display of OMB Expiration Date is Inappropriate

If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

SBA plans to display the OMB expiration date.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Explain each exception to the certification statement identified in item 19, “Certification Requirement for Paperwork Reduction Act” of OMB Form 83-I. If Agency is not requesting an exception, the standard statement should be used.

SBA is not requesting any exceptions to the certification.

1 See http://www.payscale.com/research/US/Job=Small_Business_Owner_%2F_Operator/Salary.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kelsye Turner
File Modified	0000-00-00
File Created	2021-01-15

ELI_Supporting Statement aprt A 5-16-16