Ontario Problem Gambling Research Centre

Member Login / Signup

Welcome to the Ontario Problem Gambling Research Centre. If you are a member, you can use the form on the right to log in. Not a member? No problem. Just use the form on the far right to sign up for an account - free.

Please note: you MUST have cookies enabled in order to log in and use this site.

Subscriber Login

Lost your password?

Not a subscriber yet? Sign Up!

Research Award Level III & IV - Application Guidelines / Evaluation and Scoring

Level III and IV Research Funding

The purpose of the Level III and IV awards is to:

The purpose of the Research Award Level III program is to enhance our understanding of problem gambling, and to strengthen treatment and prevention practice through research.

The guiding principles of the program are to:

The Centre's Research Awards program does not fund social/economic impact studies in relation to new gambling venues (e.g. casinos, racetracks), public opinion surveys on gambling expansion, needs assessments for new treatment services, and literature reviews. When required, studies that fall within these exclusions are commissioned separately by the Government of Ontario.

Awards are for a period of up to one year with a maximum value of $175,000 per project for direct costs, and up to 20% of the requested amount for indirect costs (overhead).

Requests for funding should be appropriately aligned with the size and scope of the proposed research activity.

Letter of Intent (LOI) Guidelines and Scoring Criteria

The Letter of Intent for Level III and IV submissions must not exceed five pages (exclusive of the cover page and references), and organized under the following headings:

Significance

Briefly introduce and describe the topic of your proposed research, and identify its goal. Describe why the topic is of significance to the problem gambling research field and merits the investment of research funds. In so doing, argue the case for the importance of funding your topic. If appropriate, indicate the OPGRC funding priority with which it is aligned.

Assessment Criterion (5 points): To what extent have the proponents established the proposed topic and goal as a) of sufficient significance to warrant the investment of research funds, and/or b) aligned with a current OPGRC priority? Note: LOIs receiving low scores on the Significance criterion will not be recommended to proceed to stage 2 of the funding process regardless of their methodological merits. This policy reflects the limited availability of funds and related need to exercise stewardship over how they are invested.

Literature Review

Recognizing the limitations of space for a LOI, provide an overview of the literature relevant to your topic, and demonstrate how your topic emerges from it and/or is supported by it.

Assessment Criterion (5 points): To what extent does the literature review support/justify the proposed topic (including the research question(s) or hypothesis(es)) as a compelling extension or next step of inquiry?

Research Question/hypothesis

State the research question, hypothesis, or hypotheses that will be answered by your proposed research. Ensure that this response is consistent with the topic, goal, and literature review and is supported by them.

Assessment Criterion (5 points): To what extent is the research question(s) or hypothesis(es) clear and supported by the topic, goal, and literature review?

Design and Methodological Approach

Outline the design for the proposed study and describe the methodological approach (or approaches) you intend to use, and justify the appropriateness. Include enough detail to allow a reviewer to understand the essence of how the research question/hypothesis(es) will be addressed. In apportioning space to your submission, give a substantial allocation to this topic.

Assessment Criteria (5 points each):

To what extent is the proposed design and methodological approach clearly described?

To what extent are the design and methodology appropriate to the research question(s) or hypothesis(es)?

Team Qualifications

Describe the relevant experience and qualifications of key team members and how they will contribute to the successful implementation of the proposed study. Note whether new researchers to the field and/or students will be included, and whether ‘end-users’ of the research will be involved in the study.

Assessment Criterion (5 points): To what extent has the team achieved a desirable/appropriate balance of expertise/qualifications, capacity development, and end-user involvement?

System Support

Describe the academic/research support available from the sponsoring organization and the contribution it will make to the successful implementation of the study. Include both financial and ‘in-kind’ contributions, if applicable.

Assessment Criterion (5 points): To what extent does the research team have appropriate access to required infrastructure and system support?

Budget

Identify major expenses for the study with explanation for items that you believe will benefit from additional detail. Organize items into the following categories:

Assessment Criterion (5 points): To what extent does the budget align with reasonable expenses for the conduct of the proposed study?

Ethics Approval

Describe the principal ethical considerations for the study and how these will be addressed. Identify by name the ethical review board to which the proposed study will be subjected.

Assessment Criterion (5 points): To what extent are ethical issues identified and appropriately addressed?

In addition to the nine criteria described above, the RRC also considers a Performance Index and a Publication Index in relation to any past research funding provided to the investigator team. These reports are prepared by OPGRC staff for the RRC and are used as verification that past research investments have proceeded in a satisfactory manner. To assist with this process, applicants are asked to append a review of past funding experience:

Productivity Index (not counted in 5-page limit)

Include an appendix identifying:

Evaluation and Scoring

Letters of Intent are independently scored by members of the Research Review Committee (RRC) and combined into a consolidated report (with scores averaged). The RRC consists of OPGRC Board members and Research Consultants (research experts who do not apply for or receive OPCRC funding).

The 5-point scoring scale is specified as follows:

5 Points – Excellent
4 Points – Very good
3 Points – Good
2 Points – Acceptable
1 Point – Not acceptable

Letters of Intent are recommended for Stage 2 of the review process if they:

Applicants whose Letters of Intent meet these standards are invited to submit a detailed Research Plan for Stage 2 in the review process (link to Research Plan Guidelines here). The RRC provides feedback to applicants based on its review of the Letter of Intent, including suggestions that might be considered in writing the Research Plan.

Applicants who are not successful at the Letter of Intent stage also receive feedback from the RRC that might help in considering whether to resubmit in a later solicitation.

Research Plan Guidelines and Scoring Criteria

The Research Plan should not exceed fifteen (15) pages in total, including the title page, but excluding appendices. Appendices are limited to 2 times the length of the research plan to a maximum of 30 pages. The application should be organized under the following:

SIGNIFICANCE (25%)

1. What is the problem/issue the proposed research will address?

Criterion: To what extent is the statement of the problem/issue clear?

Description: This section should briefly (one paragraph) outline the major problem, conceptual/pragmatic question, or research/practice gap, addressed in the application. This section should capture the interest and orient the reviewer to the nature of the study proposal to follow. It should not be detailed, as a review of the literature will follow, but should set the stage for the review of literature, and can also, in broad strokes (one or two sentences each), introduce methodological design proposed to address the question, and the expected outcomes and implications.

2. State how the results of your proposed research are expected to contribute to the field of problem gambling and why that contribution is important.

Criterion: To what extent would the anticipated results make an important contribution to the problem gambling field?

Description: In alignment with Question 1, applicants should provide a clear description of how this study and its results will advance the problem/question/gap addressed in the proposal, how it aligns with current OPGRC priorities or other credible needs statements (e.g., major reviews of the literature, academic or government position papers, blue-ribbon panels, etc.), and what significant pragmatic (e.g., clinical and/or theoretical necessary “next step”) contributions the findings, if positive, will contribute. Demonstration of the innovative nature of the proposal is also encouraged.

RATIONALE & LITERATURE REVIEW (15%)

3. Provide an overview of the literature that is relevant to your research question(s) or hypothesis(es)

Criteria: Is the literature review sufficiently comprehensive? Is the proposed research convincingly supported/justified by the literature review?

Description: Applicants should provide a focused yet comprehensive review of the most relevant literature to justify all of the facets of the proposal. Topics in need of justification by the review include: the theoretical, conceptual, clinical, methodological/parametric or other issue(s) targeted in the proposal, claims for the gaps and shortcomings that the proposal is designed to address, and justification of major methodological strategies adopted. Aspects of the proposal, particularly in its design, which may be open to criticism by reviewers not familiar with the dimension of PG field addressed in the proposal (e.g., sample size, recruitment strategy, limitations inherent in the field), can be introduced and possibly countered authoritatively and preemptively with reference to the literature. Finally, applicants are invited to cite their own preliminary work and contributions to the literature to demonstrate the groundwork, continuity, proof of concept, feasibility, methodological mastery, productivity, innovation, etc. of this line of the applicants’ research.

4. State the research question(s) and/or hypothesis(es) that will be addressed by your proposed research.

Criterion: Is/are the research question(s)/hypothesis(es) consistent with the project making an important contribution to resolving the problem/issue?

Description: Applicants should ensure that the question(s)/hypothesis(es) is/are the right one(s) to speak to the problem outlined in earlier sections, and that it(they) will be answerable/operationalizable using the methodological approach introduced. Multiple questions may be addressed; however, care is needed not to ask one protocol to answer all questions, or address issues not adequately introduced and discussed in the preceding sections and literature review. Prioritization of multiple questions into primary and secondary tiers, and designation of hypotheses as primary, secondary and exploratory will aid the reviewers in focusing their appraisal of the feasibility of the study, and its likelihood of success in achieving its principal objectives.

DESIGN AND METHODOLOGICAL APPROACH (43%)

5. Describe the methodological approach and the specific design you intend to use for the proposed study, justify the appropriateness of the approach and, as appropriate, describe any preparatory work completed.

Criterion: To what extent are the methodological approach and design described and justified as appropriate to the research question(s)/hypothesis(es)? Where applicable, Include consideration of evidence that appropriate preparatory work has been done”

Description: Applicants need to provide the reviewers with adequate evidence that they are capable of carrying out the study as proposed. Simply mentioning the name of the design is insufficient. This is an opportunity for applicants to demonstrate that they truly master the methodological and parametric subtleties of the protocols. For example, in quantitative experimental studies, it is important to link the methodology to the specific hypotheses, specifying for each the design type, dependent and independent variables etc. In qualitative studies, the need for adequate detail is no less important, but often neglected. Reviewers need to see a pragmatic demonstration that the applicants fully master the subtleties of this approach by describing it authoritatively. Reference to past work, though not sufficient, helps to argue for their expertise in using the methods proposed, and addressing other methodological details (e.g., sample size) and anticipated pitfalls (e.g., attrition rates).

6. Describe and justify the target population and the sampling strategy you propose to use.

Criteria:

Are the study population and sampling well-specified and adequately justified, as appropriate to the methodological approach, in relation to:

7. Describe the limitations to the proposed design and how you propose to address them.

Criterion: Are the limitations to the proposed design identified and appropriately addressed?

Description: The ability to foresee conceptual and/or methodological challenges is a convincing argument for the mastery of the research area, even if some problems are not completely resolvable. Discussion of preemptive strategies to intercept and reduce their impact on the validity of the findings or the feasibility of the proposal is highly recommended.

8. Describe the proposed measurement protocols and attendant limitations. Include information about published evidence of the psychometric properties of measurement instruments, as available/relevant.

Criterion: As described, to what extent are the measurement protocols clear and appropriate to the design, and the related limitations identified and discussed?

Discussion: In the case of quantitative methods, the evidence for the validity and reliability of the instruments used, if available, is useful. For qualitative methods, discussion of the validity of the information is equally important. In either case, methods to appraise of the validity of information ultimately collected should be addressed.

9. Describe the proposed data collection procedures.

Criterion: As described, to what extent are the data collection procedures clear and appropriate to the design?

Description: In addition to description of how observations are made and data collected, it may be appropriate to indicate how sources of confound (e.g., halo and Hawthorne effects) will be addressed (e.g., blinding, counterbalancing etc.). In qualitative research, information about how information from focus groups will be elicited, documented, and validated etc. should be described.

DATA ANALYSIS (7%)

10. Describe the proposed data analysis procedures.

Criterion: As described, to what extent are the data analysis procedures clear and appropriate to the research question(s)/hypothesis(es)?

Discussion: In quantitative research, the statistical strategies need to be well matched to the hypotheses to be tested or questions posed. The complexity of the hypotheses and the statistical approach need to be carefully considered. Discussion related to criteria by which inferences are made (i.e., alpha), and in related fashion, analysis- or experiment-wise adjustments to error rate due to multiple analyses (i.e., Type 1 error), are appropriate. Other statistical contingencies, such as dealing with data cleansing, preparation and validity, participant attrition and missing data, should be addressed. For treatment evaluations, accounting for non-random effects that could confound interpretations of outcomes (e.g., initial group anomalies, intent-to-treat issues), should be addressed here. For qualitative research, describe the methods that will be used to draw valid conclusions.

SCIENTIFIC EXCELLENCE DESCRIPTION OF THE INVESTIGATORS (10%)

11. Briefly describe the qualifications and experience of the key members of the study team in relation to their ability to implement the proposed research.

Criterion: To what extent do the qualifications and experience of the study team align with successful implementation of the proposed research?

FEASIBILITY (PASS/FAIL - a fail on any of the items below can lead to the rejection of the proposal regardless of its scientific merit)

12. Describe how you propose to get access to the required data and/or subjects.

Criterion: To what extent is the necessary access to data/subjects secured or likely?

Description: Recruitment for studies in PG is challenging. Reviewers need to be convinced that the recruitment plan is founded on reasonable assumptions concerning participant or data accessibility, partnerships and cooperation, effort and efficiency, participant flow-through and project timelines, case mix, etc. In addition, evidence is needed that the applicants have been successful at recruitment/data access in past work, and their strategies are likely to yield the desired results.

13. Describe the proposed work plan, including the key tasks to be undertaken, the team members assigned to each key task, and the estimated time for each task.

Criterion: To what extent are the proposed work plan and team member assignments aligned with successful implementation of the research?

14. Provide a budget for the proposed study, aligning personnel costs with the work plan and team member assignments.

Criterion: To what extent is the budget an appropriate and efficient investment of funds to successfully implement the proposed research? Is it consistent with OPGRC budgetary guidelines?

15. Identify the ethical considerations for the proposed research and describe how they will be addressed.

Criterion: To what extent have salient ethical considerations been identified and addressed?

16. Describe the anticipated end-users for your results and how you propose to communicate these results to end-users and support their application.

Criterion: To what extent are the communication/application plans likely to be effective in causing the new knowledge to reach and be understood/applied by end-users?

Description: The OPGRC is increasingly concerned with the utility/application of its research investments, and the corresponding need to demonstrate and disseminate the tangible value of research outputs. Stakeholders or “end-users” related to the proposed study should be identified and aligned with specific dissemination or uptake strategies. Publication and presentation in scholarly venues is necessary in most cases, but may not be sufficient as a convincing overall knowledge translation strategy. Referring to best practices and specialized expertise regarding knowledge translation to the different stakeholders/end-users is central to a feasible and effective plan.