OMBSupportingStatementNovember2011_with_pilot_study_A[1]

TDV OMB Clearance

Supporting Statement

Information Collection Request

Supporting Statement for the

Teen Dating Relationships: Opportunities for Youth to Define what’s Healthy and Unhealthy

also known as the

Understanding Abuse in Teen Dating Relationships Through Concept Mapping Project

Project Officer:

Carrie F. Mulford, PhD

Violence and Victimization Research Division

Office of Research and Evaluation

National Institute of Justice

Office of Justice Programs

Department of Justice

810 7th Street NW
Washington, DC 20531
Phone: (202) 307-2959
Fax: (202) 616-0275
E-mail: carrie.mulford@usdoj.gov

Table of Contents

Section A: Justification

A.1. Circumstances Making the Collection of Information Necessary…………………………………….3

A.2. Purpose and Use of the Information…………………………………………………………………..5

A.3. Use of Information Technology and Burden Reduction………………………………………………8

A.4. Efforts to Identify Duplication and Use of Similar Information………………………………………8

A.5. Impact on Small Businesses or Other Small Entities………………………………………………….9

A.6. Consequences of Collecting the Information Less Frequently………………………………………..9

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5……………………………….. 10

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency……10

A.9. Explanation of Any Payment or Gift to Respondents ……………………………………………….10

A.10. Assurance of Confidentiality Provided to Respondents…………………………………………….11

A.11. Justification for Sensitive Questions………………………………………………………………..13

A.12. Estimates of Hour Burden Including Annualized Hourly Costs……………………………………14

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers………………….18

A.14. Annualized Cost to the Federal Government……………………………………………………….18

A.15. Explanation for Program Changes or Adjustments…………………………………………………18

A.16. Plans for Tabulation and Publication and Project Time Schedule………………………………….18

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate……………………………………...23

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions…………………………...23

Justification

A.1. Circumstances Making the Collection of Information Necessary

The prevalence of adolescent dating violence has surfaced as a significant issue with implications for the criminal justice, public health and education fields. It is estimated that one third of all high school and college students will have been in an abusive relationship by the time they graduate (Levy, 1991). One in five female high school students report being physically and/or sexually abused by a dating partner (Silverman, Raj, Mucci, Hathaway, 2001). Unfortunately, these estimates describe only victimization, revealing only part of the story. Information on the prevalence of perpetration is lacking, in part because the gaps in understanding of the dynamics of teen romantic relationships, the contexts in which these relationships exist, and the attitudinal perceptions that frame dating behaviors. The need to more fully identify and explicate these issues is clear, as acceptance of dating abuse among friends is one of the strongest links to future involvement in dating abuse (Arriaga and Foshee, 2004, Bergman 1992).

Existing research provides some evidence of conceptual disparities between adult and teen populations on relationship related topics. Previous studies indicate that teens may perceive certain emotions and behaviors differently than do adults and researchers. Teens, for example, tend to perceive controlling and jealous behaviors as signs of love (Levy, 1990), and “…do not perceive of dating aggression as deleterious to the relationship, nor do they view violence as a cause for ending the relationship” (O’Keefe, 2005). In other recent studies, researchers found that adolescent couples tended to work harder than adult couples to limit the intensity of negative exchanges (Galliher, Enno, & Wright, 2008), invested more in circumventing, minimizing and disowning the differences between them and their partner (Tuval-Mashiach & Shulman, 2006), and described their romantic relationships in mainly positive terms (Shulman & Kipnis, 2001).

These findings indicate that disparities between how adults and adolescents perceive certain behaviors and emotions in romantic relationships do exist; however, little research directly compares how teen and adult populations conceptualize overall healthy and unhealthy relationship behaviors and characteristics. This study intends to determine the way adolescents categorize and comprehend features of romantic and dating relationships as a whole, rather than examining a specific emotion or behavior that may occur in a relationship.

Furthermore, while existing research has examined distinctions between adult and adolescent dating behaviors, few studies have examined these dynamics from a standpoint of preventing teen dating violence. Recent research argues that “more research, particularly qualitative studies, are needed to enhance our understanding of adolescent dating violence, including the nature of relationship conflicts, as well as the meaning, content, intent and consequences of the violence.” (O’Keefe, 2005) Before we are able to address teen dating violence prevention, however, it is “critical to understand how [emotions and behaviors] at the core of adult relationships…may be played out in adolescent relationships.” (O’Keefe, 2005) The present study intends to strengthen our understanding of these characteristics in teen dating relationships.

Few studies to date have considered teens’ perceptions or definitions of dating behavior as described in their own words; rather, the majority of research on adolescent dating has been analyzed through measures and concepts developed by adult researchers and practitioners (Shulman & Kipnis, 2001, O’Keefe, 2005, Tuval-Mashiach & Shulman, 2006, Galliher, Enno, & Wright, 2008, Murphy & Smith, 2010). This field of research, therefore, lacks a definition of dating and dating behaviors that considers the perspective of adolescents, and signals the need to further identify and consider the developmental, social, and environmental components of their interactions, and mechanisms of influence that are specific to teen dating violence. While much of the prior research on teen dating violence apply adult frameworks and assumptions to characterizing relationships, “further research is needed to test these assumptions” and to determine their validity in being applied to adolescents (Murphy and & Smith, 2010). The present study seeks to address the need for an adolescent-based conceptualization and understanding of relationship behaviors. The concept mapping methodology is an appropriate tool for achieving this end, as the methodology allows for the creation of a collaboratively-authored conceptual framework of relationship features, including the range of healthy and unhealthy characteristics, which captures the viewpoints of both adults and adolescents. The utility of the concept mapping methodology relative to the purpose of the study is seen in the ability to capture the complex, interrelated, and multi-level features that constitute unhealthy and healthy dating relationships of adolescents. In particular, the deleterious effects of violent or abusive relationships in adolescence have significant implications for the health and well-being of young people and the contexts in which they reside. This study is an initial, formative step in the development of a conceptual framework, from which more quantitative methods will be employed. In particular, this foundational work will establish a set of constructs that can be used in future studies to inform a broader population-based sampling and measurement strategy in an effort to verify and confirm the presence and variability of these constructs in representative samples.

This information collection request falls under the Department of Justice, Office of Justice Programs. This information is necessary and having it will make a difference in terms of advancing the knowledge base on the issue and influencing future programming and surveillance. To our knowledge there have not been past data collections that may be applicable to the study.

A.2. Purpose and Use of the Information

The purpose of this study is to develop a better understanding of adolescent relationship features, including the range of healthy, unhealthy, and abusive characteristics, from the perspective of both adolescents and adults. More specifically, this study intends to examine the extent to which the conceptualized models of adolescents and adults (e.g., parents or other adults in the community, practitioners, and researchers) converge or diverge, and to determine the value that adolescents place on various relationship characteristics. An identified gap in the field is the intentional inclusion of adolescents in developing a research agenda in response to violence prevention and healthy relationship development. As with all emerging fields of research, definition and concept development are critical. It is important that the conceptualization that researchers and practitioners use to define relationship features, especially abuse, resonates with the population to which the definitions are being applied. It is anticipated that the data collected through this study will help advance understanding of the ways conceptualizations of dating relationship characteristics vary across and within groups. The results of the project can suggest how prevention and intervention efforts can effectively target relationship characteristics related to abusive behavior. In other words, what we learn through this study will help to inform the language and content of interventions and responses to the issue. The results may also be used to educate youth about dangerous behaviors that they may not consider negative or abusive. The findings from this study and data collection will also inform the development of new initiatives and projects or revisions of existing projects.

Additionally, the results of this study will have implications for improving the ways in which researchers and practitioners measure the frequency of teen dating violence. Definitional ambiguity and methodological variations across studies have led to a lack of consensus about interpersonal violence among teens, which has compounded challenges in accurately understanding the prevalence of and responses to dating violence (Barnett, Miller-Perrin, & Perrin, 2005). Further explication of the ways in which adolescent conceptualize positive and negative aspects of dating relationships are warranted, and methods that enable youth to directly engage in specifying and organizing concepts may yield new patterns to emerge. The emergence of these patterns is critical for the development of measurement approaches.

The goal of this research is to explicate and define, from the perspective of both youth and adults, the key elements associated with healthy and unhealthy dating relationships. Concept mapping is a participatory mixed method approach that enables diverse participant groups to develop shared conceptual frameworks that can be used in a variety of policy and practice contexts to identify and encourage complexity. Through this inductive process of conceptualization the domain can be refined, expanded, and enhanced through multiple feedback loops, including facilitated discussions with both youth and adults. Future research will be needed to test the validity of these results with the population of youth at large; the purpose of this concept mapping activity, however, is to use a convenience sample that is as diverse as possible given the study parameters, to arrive at a conceptual framework that will allow for initial comparisons between youth and adults, and provide avenues for further inquiries to confirm and enhance.

Ultimately the final product of this study will help to advance areas of policy, practice and research by enabling a collaboratively constructed framework, derived from multiple perspectives, including youth, practitioners, policy-makers, and researchers. Appropriate responses to address the promotion of healthy relationships and the reduction of teen dating violence require a common agreement of the concepts that have specific relevance to the target beneficiaries. One of the principal barriers to providing legal protections to teens has been the adult-centered nature of the criminal justice system. Similarly, treatment and prevention responses have been equally hampered by a full understanding of how teens and others conceptualize and deal with dating violence. For example, victims of teen dating violence often confide in peers, avoiding more formal sources of community support (Barnett, Miller-Perrin, & Perrin, 2005). The work proposed here is critical, as an emergent framework that emphasizes the complex, interrelated aspects of teen dating violence from multiple perspectives (particularly those from youth) will help to advance a unified practice, research, and policy agenda. The Federal Interagency Workgroup on Teen Dating Violence comprised of representatives from 18 agencies, representing the Departments of Health and Human Services, Justice, Education, and Defense will review and use the collected information to shape future efforts to better understand the issues and organize effective responses to violence prevention and healthy relationship development. Without such information, planning and development may be limited in terms of appropriately matching language and communications, intervention and responses, and measurement and data collection with targeted beneficiaries.

A.3. Use of Information Technology and Burden Reduction

The collection of information for the concept mapping portion of this study will be conducted entirely via a dedicated project website. The website will be administered by Concept Systems, Inc. (CSI), the contractor for this project. This web based collection technique will reduce the participation time burden for respondents, as it will allow them to respond virtually and remotely, during time that is convenient for them. The project website will also allow for respondents to complete the concept mapping activities over multiple visits to the website (i.e. respondents will not need to complete the activities in one sitting). The use of information technology for the concept mapping will also reduce the financial burden for respondents, as they will not incur any travel expenses in completing the tasks.

The facilitated discussions will not use information technology, as the in-person, face-to-face context of these discussions will be critical to engaging participants in a productive conversation that will elicit the necessary feedback on the conceptual framework and will confirm its validity for use in teen dating violence prevention efforts.

A.4. Efforts to Identify Duplication and Use of Similar Information

This collection of information involves no effort to identify duplication or use similar information. The Federal Interagency Workgroup on Teen Dating Violence has been meeting every 6-8 weeks since September 2006. During this time, participating agencies have collaborated on several joint efforts, including co-hosting an HHS-DOJ scientific 2 day workshop in December 2007 examining the research, both basic and applied, on teen dating violence, and determining research, evaluation and practice gaps for further study. In consulting with the organizations and research partners, including a review of the existing literature on the subject, it has been determined that the information outlined in this data collection effort does not currently exist that would serve the purpose of the project.

A.5. Impact on Small Businesses or Other Small Entities

The collection of information may impact not-for-profit youth-serving organizations or agencies which researchers request assistance in youth respondent recruitment. These organizations will receive recruitment flyers and parental permission slips to distribute to potential youth respondents. In order to minimize the burden of these requests, the research team will mail a set of instructions and recruitment materials to organizational representatives that will be prepared in advance for distribution. The research team will also arrange for a conference call with organizational representatives to answer any questions about the recruitment tasks, and will also provide ongoing support on an individual basis as needed. The research team will have reviewed the recruitment materials with a project Advisory Board in advance of distribution, to ensure that the instructions and process are as clear and as efficient as possible. The research team anticipates that the recruitment requests will take organizational representatives no more than two hours total over the course of the information collection period.

A.6. Consequences of Collecting the Information Less Frequently

This information will be collected only once, although certain respondents may be asked to participate up to three to four times during the study, depending on the tasks for which they are recruited. Responding the multiple requests to participate is voluntary. Once participants provide us with their ideas on teen dating relationships (“brainstorming”), a selected subsample of participants will be asked 4 to 8 weeks later to sort the ideas into similar themes (“sorting”). All individuals who provided ideas will also be asked approximately 4 to 8 weeks after the brainstorming to rate the ideas on perceived frequency and desirability (“ratings”). The respondent re-contact interval is necessary to allow the research team to review the originally brainstormed statement set for clarity, relevance and redundancy, and finalize a set of ideas (100 or fewer) that is manageable in number for respondents to engage in the next stage of the research (sort and rate), thereby reducing burden.

A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

Concept mapping participants will be asked to respond to the brainstorming activity within approximately six weeks. Participants that are asked to respond to the sorting and rating activities will be asked to do so within approximately four weeks after the conclusion of the brainstorming activity. Responses to all portions of the concept mapping activities are voluntary, thus any individual who is not able to provide a response within the requested time period is not obligated to do so. The information collection outlined fully complies all guidelines of 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

The agency contracted with Concept Systems, Inc. to provide project management and to facilitate the concept mapping and facilitated discussion processes. In order to ensure the clarity of instructions for all collection materials, the agency also consulted with project Planning and Advisory Groups. These Groups are comprised of researchers, practitioners and advocates in fields related to teen dating violence, as well as representatives from youth-serving organizations. These Groups advised the agency on project design, and reviewed respondent materials for their readability to make sure that all youth participants would be able to understand and take part in the different project activities. The agency and research team has met with the Planning Group on a bi-weekly bases and the Advisory Group on a monthly basis since October 2010.

A.9. Explanation of Any Payment or Gift to Respondents

Teen and young adult respondents who complete the rating activities will receive a $10 gift card. Teen and young adult respondents who complete sorting and rating activities will receive a $15 gift card for completion of both activities. The difference in compensation amount reflects the additional 30-60 minutes of time it will take respondents to complete the online sorting task in addition to the online rating task. The agency decided to provide youth sorters and raters with a gift because the sorting and rating are the most time intensive tasks of the concept mapping activities, for which youth participation is critical in developing a useful, accurate conceptual framework. It is believed that by offering a token of appreciation, teens and young adults will be more likely to complete the sorting and rating tasks than they would if there was no incentive offered. It is critical that completion of the entire task is observed, as the validity and reliability of the final product is dependent upon a full response. An expression for appreciation for data collection and research activities for youth is common, and the amount offered in this effort is consistent with similar studies. Parental permission forms indicate that youth who complete each activity will be receiving a gift card, so that parents are made fully aware that their child may be receiving such expression of appreciation. Adult respondents will not be compensated for their participation in the concept mapping activities. The adult respondents will be mainly comprised of teachers, advocates, practitioners and researchers who will be likely to complete the tasks out if interest in the project and in contributing to the field.

A.10. Assurance of Confidentiality Provided to Respondents

Phase I: Concept Mapping

Youth participation in all components of the concept mapping process will be kept confidential. Each recruitment flyer that partner organizations distribute to participants will include a unique, randomly generated username and password, which will become the username and password for the individual participant who receives the flyer. (Organizational representatives that are assisting in recruitment will be instructed to only distribute flyers to individuals 17 years old or younger if they return a signed parental permission slip.) The username and password will allow each participant to log onto the project website to complete the concept mapping activities confidentially. Participants will only be identifiable to project management by their username.

The research team will include with the mailings of recruitment flyers and parental permission slips a Master Tracking List for each organizational representative to keep a record of the individual name associated with each username. No one affiliated with this project or project management will ever receive the names of the individual participants. The database serves only as a tool for the organization representatives to keep track of which participants are affiliated with which usernames, so that the researchers can communicate to the representatives which participants have completed the concept mapping activities and are therefore eligible to receive a token of appreciation. Gift cards will be distributed to the participants by organizational representatives.

Adult participation in the brainstorming, sorting and rating activities of the concept mapping process will require that each adult participant access the different activities with a unique username and password. Due to project resourcing, the research team, rather than a third-party organization or agency, will be managing the distribution of these username and passwords to adult participants. The agency will\ maintain complete confidentiality of all input from adult participants, as the data elicited from participants will not be associated with any individual at any point in the project analysis or reporting. Adult participants will be made fully aware of these parameters and confidentiality assurance during the online consent process (See Appendices A8-A10.)

Phase II: Facilitated Discussions

Given the in-person, face-to-face context of the facilitated discussions, participant identity will not be kept anonymous. During the discussions, participants will be identifiable to one another by name tags that will display first names only. The research team will have a database of all participants’ contact information, as this information is necessary for communication, recordkeeping and travel reimbursement purposes. For youth participants (ages 14-22), participation will depend upon both individual assent and parental consent. Youth will only be allowed to participate if both assent and parental consent forms are received by the researchers in advance of the discussions (specific date to be determined). (Please reference the “Informed Consent” section of this application for the assent and consent documents that participants and/or parents will receive.)

Any content from these discussions that is incorporated into the final project report will not be specific to any participant(s); although the research team and the Planning Group will be using the content to enhance and support the final report recommendations, readers will be unable to trace statements or ideas to individual participants. The researchers will maintain a confidential database of participant names and contact information in the security of their Ithaca, New York office on a password protected server for three years after the dates of the facilitated discussions. This data will be kept in a password protected folder on the researcher organization’s server, so that only members of the research team will have access to this information. Participants will not be contacted after the conclusion of the project unless they express an interest in being contacted in the future. All participants will be provided with contact information for the research team members.

A.11. Justification for Sensitive Questions

All concept mapping participants will be asked to respond to the focus prompt, “A thought, feeling, action or behavior that teens in dating relationships might have or do is…” After extensive consultation with project Planning and Advisory Groups, the research team determined that this focus prompt was the best question to ask respondents in order to elicit the most useful content for the purposes of the study. The statements that are derived from this focus prompt will be reviewed and synthesized by the research team, to yield a set of 100 or fewer ideas that respondents will sort and rate in the subsequent concept mapping activities. These statements will become represent the range of ideas that all respondents provided on the topic of teen dating relationships, and will the basis of the resulting conceptual framework.

Teens may feel some sense of unease around the topic of dating, however we do not anticipate this discomfort to be more than what teens would likely encounter on a day-to-day basis. All participants will also have the option to abstain from any activity of the study at any time without penalty. In order to limit the discomfort that reading or listening to others’ answers and ideas may cause the research team will feature a website (www.breakthecycle.org) at the beginning and end of each concept mapping activity. Upon visiting the website, participants will have access to a list of resources, including websites and phone numbers that participants can use to learn more about healthy teen dating behaviors and what to do if they feel that they are in an unhealthy relationship. A hard copy of these resources will also be distributed to facilitated discussion participants at the meetings.

The information and instructions that will be provided to concept mapping and facilitated discussion participants, as well as consent forms for each group of respondents are included as attachments to this document. This informed consent process has been approved by the Institutional Review Board at SUNY-Cortland under Protocol # 101139.

A.12. Estimates of Hour Burden Including Annualized Hourly Costs

Four hundred participants will be invited to participate in the brainstorming task, which will take respondents no more than five to ten minutes to complete. One hundred participants will be invited to participate in the sorting task, which will take respondents approximately sixty to ninety minutes to complete. Four hundred participants will be invited to participate in the rating task, which will take respondents approximately forty to sixty minutes to complete. The total annual hour burden for a concept mapping participant will vary based on the specific combination of concept mapping activities he or she will be asked to participate in, but this hour burden will be no more than two hours and forty minutes at most.

Each facilitated discussion will be approximately three hours in duration and will require approximately one hour of total travel time for each participant. These discussions require three hours of participants’ time in order to serve their intended purpose of confirming and enhancing the results of the concept map. The three hour agenda includes time for participants to introduce themselves, for facilitators to provide an overview of the discussion, review the concept mapping process and results map, and allow for ample discussion time. The three hours also includes multiple breaks for participants to shift focus, break into smaller group discussions, ask questions, and record on paper any additional thoughts or comments. We anticipate that the topic – teen dating – will be of particular interest to youth participants, and will therefore seem less burdensome than a discussion of a less compelling topic.

There will be eight facilitated discussions comprised of twenty participants each. The agency has estimated the annual hour burden for respondents of the concept mapping and facilitated discussion phases based on the contractor’s extensive past experience administering concept mapping activities and facilitated discussions.

Prior to full implementation of the project, a pilot study will be conducted with fifty teen participants (ages 14-18) who will be recruited from two or three organizations. The fifty pilot participants will participate in all three tasks (brainstorming, sorting and rating). There will also be two facilitated discussions comprised of twenty participants each included in the pilot study. The charts below show the burden for the pilot study. There are an estimated 253 hours of public burden associated with the collection of the pilot data. This estimation was calculated using the higher end of the expected ranges of time each activity takes to complete, and also assumes 100% participation. It is highly likely that most participants will take less time that is reflected in this estimate, and that participation rates will be less than 100%. The charts below reflect burden hours for both the higher and lower estimated time frame to complete each activity.

Higher end of time estimate range for pilot study only:

Task	Estimated time (minutes)	Total Participants	Total minutes per task
Brainstorming	10	50	500
Sorting	90	50	4500
Rating	60	50	3000
Facilitated Discussions	180	40	7200
Total			15,200 minutes (~253 hours)

Lower end of time estimate range for pilot study only:

Task	Estimated time (minutes)	Total Participants	Total minutes per task
Brainstorming	5	50	250
Sorting	60	50	3000
Rating	40	50	2000
Facilitated Discussions	180	40	7200
Total			12,450 minutes (~208 hours)

There are an estimated 1097 annual total public burden hours associated with the full collection.

Higher end of time estimate range for full study:

Task	Estimated time (minutes)	Total Participants	Total minutes per task
Brainstorming	10	400	4000
Sorting	90	100	9000
Rating	60	400	24,000
Facilitated Discussions	180	160	28,800
Total			65,800 (=1097 hours)

Lower end of time estimate range:

Task	Estimated time (minutes)	Total Participants	Total minutes per task
Brainstorming	5	400	2000
Sorting	60	100	6000
Rating	40	400	16,000
Facilitated Discussions	180	160	28,800
Total			52,800 (=880 hours)

A.13. Estimate of Other Total Annual Cost Burden to Respondents or Recordkeepers

The DOJ anticipates no additional cost burden to respondents or recordkeepers beyond that which results from their customary or usually business or private practices.

A.14. Annualized Cost to the Federal Government

The contract to conduct the concept mapping project was competitively awarded to Concept Systems, Inc. The total time and materials contract estimate is based on a 30-month contract amount of $294,282.78. The pilot study will be conducted under a separate sole source contract to Concept Systems, Inc. The following are cost estimates to the Federal government based upon activities anticipated over the next three years:

1. Base Year……….$64,166.88

2. Option Year……..$183, 025.20

3. 6 Month Period….$47.090.70

4. Pilot Study………$86,818.00

5. Total Federal Government Cost: $381,100.78

A.15. Explanation for Program Changes or Adjustments

Not applicable. This is a new collection of information.

A.16. Plans for Tabulation and Publication and Project Time Schedule

Concept mapping will be the primary method for analyzing the data. The Concept System® software will be used to combine the individual participant’s sort data, and using several multivariate statistical algorithms, will organize the information and displays it in a series of easily readable concept maps. This process will begin with construction from the sort information of an NxN binary, symmetric matrix of similarities, Xij. For any two items i and j, a 1 is placed in Xij if the two items were placed in the same pile by the participant, otherwise a 0 is entered (Weller and Romney, 1988, p. 22). The total NxN similarity matrix, Tij will be obtained by summing across the individual Xij matrices. Thus, any cell in this matrix could take integer values between 0 and the number of people who sorted the statements. The value will indicates the number of people who placed the i,j pair in the same pile.

The total similarity matrix Tij will be analyzed using nonmetric multidimensional scaling (MDS) analysis with a two-dimensional solution. The solution will be limited to two dimensions because it is generally easier to work with two-dimensional configurations than with those involving more dimensions, (Kruskal & Wish, 1978). Ease of use considerations are important for decisions about dimensionality. For example, when an MDS configuration is desired primarily as the foundation on which to display clustering results, then a two-dimensional configuration is far more useful than one involving three or more dimensions (p. 58).

The analysis will yield a two-dimensional (x,y) configuration of the set of statements based on the criterion that statements piled together most often are located more proximately in two-dimensional space while those piled together less frequently are further apart. The x,y configuration will serve as the input for the hierarchical cluster analysis utilizing Ward's algorithm (Everitt, 1980) as the basis for defining a cluster. Using the MDS configuration as input to the cluster analysis in effect will force the cluster analysis to partition the MDS configuration into non-overlapping clusters in two-dimensional space. There is no simple mathematical criterion by which a final number of clusters can be selected. The procedure that is typically followed is to examine the initial cluster solution that was the maximum desirable for interpretation in this context. Then, successively lower cluster solutions will be examined, with a judgment made at each level about whether the merger seems substantively reasonable. The pattern of judgments of the suitability of different cluster solutions will be examined and the final number of clusters selected to preserve the most detail and still yield substantively interpretable clusters of statements.

The MDS configuration of the statement points will be graphed in two dimensions automatically by the Concept System program. This "point map" will display the location of all the brainstormed statements with statements closer to each other generally expected to be more similar in meaning. A "cluster map" will be also generated that displays the original statement points enclosed by polygon-shaped boundaries for the clusters. Additional reports will demonstrate similarities and differences in perception and opinion among the participants. Additional graphics, including pattern matches and bivariate scatter plots, will display stakeholders’ ratings of the ideas. The 1-to-5 frequency and desirability rating data will be averaged across persons for each item and each cluster. This rating information will be depicted graphically in a "point rating map" that will show the original point map with the average rating per item displayed as vertical columns in the third dimension, and in a "cluster rating map" that will show the cluster average rating using the third dimension. Two additional graphic and statistical analyses will be computed based upon the map results. A “pattern match” is defined as the bivariate relationship between the cluster average ratings for two groups, variables or occasions. This will be visually displayed as a ladder graph or pair-link diagram with two vertical axes that represent the two variables and horizontal lines connecting them to represent the ratings for each cluster. A standard Pearson Product-Moment Correlation (r) will be also computed to indicate the overall pattern match. In addition, standard descriptive statistics will be produced (mean, SD, N) that will enable significance tests of differences between ratings on clusters. Pattern match graphs will be used to assess consensus or differences of different participant groups on the relative frequency and desirability. A “go zone” is defined as a within-cluster bivariate plot of average statement ratings for two groups, variables or occasions. Like a pattern match, it also displays the Pearson Product-Moment Correlation (r) between the two variables. The plot is a restricted form of a standard bivariate plot in that it: (a) sets the minimum and maximum values for all plots to the same range (based on minimum and maximum statement average for that variable); and (b) the bivariate space is divided into quadrants based on the cluster average of the x and y variables. This effectively means that every go zone plot will have a quadrant that shows which statements in the cluster were rated above average on both variables, one that shows which statements were below average on both and two that show the statements that were above average on one and below on the other. This plot, like a pattern match, will be used to explore consensus, in this case, within-cluster.

The content derived from the facilitated discussions will be integrated into the final conceptual framework through an iterative process of data reduction and synthesis. It is anticipated that the facilitated discussions will yield a substantial volume of information, from which information can be extracted to further elucidate emergent concepts. Using the concept mapping framework as the analytic structure for organizing, reducing and analyzing the content captured during the facilitated discussions, we expect to identify insights, themes, patterns and constructs that will aid in the interpretation of the domain. Traditional approaches to qualitative data reduction, coding, and analysis will be used within the concept mapping structure to illuminate the connections individuals make between and among the concepts.

The conceptual framework developed out of the concept mapping exercise and facilitated discussions from the pilot study will be integrated into the plan for the full study. The pilot study framework should provide the research team with addition information to clarify research questions for the full study.

The following table indicates the project time schedule:

Information Collection Task	Month after receiving OMB Approval
Conduct the concept mapping exercise for the pilot study	1-4
Analyze the results from the pilot study	5
Create report of pilot study results	6
Conduct facilitated discussion for pilot study	7-8
Create final integrated report	9
Incorporate findings from pilot study into revised supporting statement for OMB	10
Conduct the full Concept Mapping exercise.	11-14
Analyze results, including, but not limited to developing and comparing conceptual maps for adults and youths, and comparing rating scale comments for adults and youth.	14
Present preliminary results to NIJ and TDV Workgroup at a TDV Workgroup Meeting in Washington, DC.	15
Create draft and final concept mapping report.	15-16
Identify target goals and agendas for the facilitated discussions.	16-17
Identify advocacy organizations and individuals to participate.	17-19
Invite organizations to recommend participants for the facilitated discussions.	19
Send initial notices of the meeting, invitations to participate and advance materials to facilitated discussion participants.	17 19
Prepare and Brief Planning Group and Advisory Group on roles for facilitated discussion.	20
Conduct eight facilitated discussions.	20-21
Conduct content analysis of the facilitated discussions.	21-23
Create draft and final report of the facilitated discussion activity.	23-24
Integrate confirmatory, expanded & new information from the facilitated discussions into the concept map framework.	25
Work with Planning Group to determine dissemination strategy.	25
Create report of Phase I and Phase II aggregated information.	25
Report back to NIJ and TDV Workgroup on final recommendations.	27

A summary report will be developed that will include the aggregated concept mapping and facilitated discussion results. The summary will also include a statistical report of the participant response rate for the sorting and rating activities. At the conclusion of the study, the data will be archived at the National Archive of Criminal Justice Data (NACJD).

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

Not applicable. The agency is not seeking approval to not display the expiration date for OMB approval of the information collection.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable. There are no exceptions the Certification for Paperwork Reduction Act Submissions for this information collection.

23