SEEK: Dissemination and Implementation

The Safe Environment for Every Kid (SEEK) Model: Dissemination and Implementation

Status

Completed

Phases

Study type

Interventional

Source

ClinicalTrials.gov

Registry ID

NCT03642327

Enrollment

360

Registered

2018-08-22

Start date

2019-03-15

Completion date

2024-03-31

Last updated

2025-05-29

For informational purposes only — not medical advice. Sourced from public registries and may not reflect the latest updates. Terms

Conditions

Child Maltreatment

Keywords

Child Abuse Prevention, Primary Care, Professional Education in Primary Care

Brief summary

This study aims to understand what influences the adoption and implementation of an innovative approach to pediatric primary care by medical professionals and staff providing that care. The innovation concerns addressing common psychosocial problems such as parental depression and substance abuse. Different approaches to training medical professionals will be examined as well as the use of software versus traditional 'paper and pencil' to facilitate implementation.

Detailed description

The proposed study, based on SEEK I and II findings is designed to study the effectiveness of technology driven training strategies to facilitate use of SEEK in helping prevent child maltreatment (CM) and its adoption and implementation in pediatric and family medicine settings. The implementation approach is anchored in four stages of the EPIS conceptual model: Exploration, adoption/Preparation, Implementation, and Sustainment. The strong evidence for SEEK's effectiveness has been underscored by its listing on the websites of the CDC, AHRQ, AAP, and by the California Clearinghouse for Evidence-Based Interventions in Child Welfare. Early adopters are increasingly implementing SEEK in primary care settings, including in Sweden. Four of the 5 healthcare systems in this proposal have implemented SEEK in some of their practices. However, practices within these systems operate independently of one another, and there should not be contamination among them. Each practice has its own lead physician, primary care providers (PCPs) and administrative staff and several have integrated behavioral health. Design. The investigator selected a rigorous Hybrid Type III design which is an approach to examine implementation outcomes while also examining SEEK's effectiveness in preventing CM, measured by electron health record (EHR) indicators. Type III designs are facilitated by good EHR systems due to the low cost of data routinely gathered for clinical, services, and financial purposes. Five healthcare systems have committed to participating in the project, without committing individual practices and primary care providers (PCPs). Professionals, office staff and parents are nested within practices which are nested within the 5 participating healthcare systems. Practices will choose whether to participate, as will medical professionals within those practices. Participating practices will be randomized to one of the two training strategies for PCPs - Independent online training vs. a Maintenance of Certification (MOC) activity approved by the Boards of Pediatrics and of Family Physicians. Practices will be able to select the facilitation strategy (SEEKonline software or Traditional 'paper and pencil'). Participating professionals and staff within practices will need to adhere to the approach adopted by the practice. The design accounts for heterogeneity in geography, size of healthcare systems, type of primary care (pediatric and family medicine), and presence of integrated behavioral health. A small random sample of parents will be recruited from each practice. For Aim 1, practices and PCPs will be randomized to one of the two training approaches. Using baseline and follow-up surveys, the investigators will assess the impact of each approach on PCPs' attitudes, knowledge, sense of competence, level of comfort and practice behavior with regard to addressing the targeted problems (e.g., parental depression). Aim 2 focuses on the implementation of the SEEK model in pediatric and family medicine primary care practices. Practices will have the option of implementing the model using the SEEKonline software or the traditional paper and pencil approach. The evaluation involves measuring aspects of implementation such rates of adoption, fidelity to the model, and sustainment beyond the training. In addition, qualitative interviews will provide data regarding what facilitates or impedes optimal implementation. For Aim 3, The investigators will assess the impact of the interventions on rates of child maltreatment (CM) based on ICD 10 diagnoses in EHRs. The investigators will include a cost-effectiveness analysis by estimating the costs associated with the different approaches to implementing SEEK.

Interventions

BEHAVIORALExperimental: Maintenance of Certification (MOC)

MOC training will lead to more positive attitudes, comfort level and competence in addressing risk factors for Child Maltreatment.

OTHERIndependent online training (IND)

IND refers to practitioners Independently doing the online training.

Study design

Allocation

RANDOMIZED

Intervention model

PARALLEL

Primary purpose

PREVENTION

Masking

NONE

Intervention model description

RCT

Eligibility

Sex/Gender

ALL

Age

18 Years to No maximum

Healthy volunteers

Yes

Inclusion criteria

* Practice not already implementing the SEEK model. * The practice provides primary care to children. * The practice agrees to participate in the study. * PCPs, office staff and parents of children 0-5 years agree to participate. * Able to comprehend basic English * Access to a computer to complete online surveys.

Exclusion criteria

* None

Design outcomes

Primary

Measure	Time frame	Description
Stages of SEEK Implementation Completion (SIC) - Startup (at a Practice Level)	up to 30 months	The SIC is an assessment tool with 8 stages extending from Engagement to achievement of Competency. Each stage maps onto 3 phases of implementation (Pre-Implementation, Implementation and Sustainment). Startup reflects whether the practice implemented SEEK - Yes/No. Yes = practice started implemented SEEK. Practices that did start are reported.
Stages of Implementation Completion (SIC) - as Assessed by the Safe Environment for Every Kid (SEEK) Fidelity Score (at a Practice Level)	up to 30 months	The SIC is an assessment tool with 8 stages extending from Engagement to achievement of Fidelity. Each stage maps onto 3 phases of implementation (Pre-Implementation, Implementation and Sustainment). Competence/Fidelity score reflects the extent to which a practice implemented the intervention as intended. Score range: 0-100; higher is better fidelity to the intervention as planned.
Stages of SEEK Implementation Completion (SIC) - Sustainability	up to 30 months	The SIC is an assessment tool with 8 stages extending from Engagement to achievement of Competency. Each stage maps onto 3 phases of implementation (Pre-Implementation, Implementation and Sustainability). Sustainability was determined if the practice was implementing SEEK 24 months after initiation and doing so with fidelity. Reporting the number of practices that achieved sustainability.

Secondary

Measure	Time frame	Description
Rates of Under Immunized Children	2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.	ICD-10 codes related to possible maltreatment (child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.
Rates of Children With Possible Abuse.	2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.	ICD-10 codes related to possible maltreatment (child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.
The Safe Environment for Every Kid (SEEK) PCP Questionnaire (PCPQ) - Addressing Depression Subscale	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Addressing Food Insecurity	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Addressing Major Stress	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Addressing Intimate Partner Violence	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Addressing Harsh Punishment	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Addressing Substance Abuse	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
Overall Rates of Children With Possible Abuse and Neglect.	2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.	International Classification of Diseases - 10 (ICD-10) codes related to possible child maltreatment (CM or child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.
SEEK PCP Questionnaire (PCPQ) - Comfort Level Regarding Addressing Health-related Social Needs	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Perceived Competence Regarding Addressing Health-related Social Needs	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Knowledge Regarding Addressing Health-related Social Needs	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK PCP Questionnaire (PCPQ) - Practice Behavior Regarding Addressing Health-related Social Needs	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
SEEK Parent View of Relationship With Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	24 months	Parents rate their child's PCP on this measure adapted for pediatric practice from the Patient-Doctor Interaction Scale. The Likert scale included 8 questions; the response set was from 1 (no) to 4 (always). Subscale mean scores range from 1 - 4; higher scores indicate greater satisfaction. The mean of the Likert subscale response is reported.
SEEK Parent View of the Attentiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	24 months	Parents rate their child's PCP on this measure adapted for pediatric practice from the Patient-Doctor Interaction Scale. The Likert scale included 8 questions; the response set was from 1 (no) to 4 (always). Subscale mean scores range from 1 - 4; higher scores indicate greater satisfaction. The mean of the Likert subscale response is reported.
SEEK Parent View of the Comprehensiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	24 months	Parents rate their child's PCP on this measure adapted for pediatric practice from the Patient-Doctor Interaction Scale. The Likert scale included 8 questions; the response set was from 1 (no) to 4 (always). Subscale mean scores range from 1 - 4; higher scores indicate greater satisfaction. The mean of the Likert subscale response is reported.
SEEK Parent View of the Communication Skills of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	24 months	Parents rate their child's PCP on this measure adapted for pediatric practice from the Patient-Doctor Interaction Scale. The Likert scale included 8 questions; the response set was from 1 (no) to 4 (always). Subscale mean scores range from 1 - 4; higher scores indicate greater satisfaction. The mean of the Likert subscale response is reported.
SEEK PCP Questionnaire (PCPQ) - Attitudes Regarding Addressing Health-related Social Needs	30 months	The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.
Rates of Children With Possible Neglect.	2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.	ICD-10 codes related to possible maltreatment (child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.
Rates of Children With Non-adherence to Medical Care	2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.	ICD-10 codes related to possible maltreatment (child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.

Countries

United States

Participant flow

Recruitment details

Pediatric and family medicine practices across the U.S. were contacted from March 2019 to June 2022 through several avenues: American Academy of Pediatrics state chapters, collegial networks, healthcare system liaisons, and primary care networks. To be eligible, practices had to agree to randomization to either training modality and to introducing SEEK into their practice.

Pre-assignment details

Of the 44 practices that expressed initial interest, 29 agreed to participation. Four withdrew prior to randomization, so 25 practices were randomized to IND (13) or MOC (12), stratified by practice type (family medicine/pediatrics) and resident training site (yes/no). 6 additional practices withdrew early on. Reasons included feeling already overwhelmed, compounded by the pandemic; the sole family medicine practice cited plans for universal screening within their healthcare system.

Participants by arm

Arm	Count
Independent Online Training (IND) IND refers to practitioners independently doing the online training per practice assignment. We did not conduct separate analyses on the other subsamples: practice leaders, behavioral health professionals, admin/nursing and parents. Particularly in the 1st three groups the n was too low.	90
Independent Online Training (IND) IND refers to practitioners independently doing the online training per practice assignment. We did not conduct separate analyses on the other subsamples: practice leaders, behavioral health professionals, admin/nursing and parents. Particularly in the 1st three groups the n was too low.	13
Maintenance of Certification (MOC) MOC refers to practitioners trained via the SEEK MOC-4 Activity per practice assignment. involves a guided learning experience, approved by the American Board of Pediatrics and the American Board of Family Medicine for Maintenance of Certification credits. This involves a Quality Improvement project with 3 waves of data collection to assess and improve implementation while participating in 4 monthly webinars led by Dr. Dubowitz. We did not conduct separate analyses on the other subsamples: practice leaders, behavioral health professionals, admin/nursing and parents. Particularly in the 1st three groups the n was too low.	120
Maintenance of Certification (MOC) MOC refers to practitioners trained via the SEEK MOC-4 Activity per practice assignment. involves a guided learning experience, approved by the American Board of Pediatrics and the American Board of Family Medicine for Maintenance of Certification credits. This involves a Quality Improvement project with 3 waves of data collection to assess and improve implementation while participating in 4 monthly webinars led by Dr. Dubowitz. We did not conduct separate analyses on the other subsamples: practice leaders, behavioral health professionals, admin/nursing and parents. Particularly in the 1st three groups the n was too low.	12
Total	235

Baseline characteristics

Characteristic	Independent Online Training (IND)	Maintenance of Certification (MOC)	Total
Age, Categorical <=18 years	0 Participants	0 Participants	0 Participants
Age, Categorical >=65 years	0 Participants	0 Participants	0 Participants
Age, Categorical Between 18 and 65 years	90 Participants	120 Participants	210 Participants
Ethnicity (NIH/OMB) Hispanic or Latino	4 Participants	10 Participants	14 Participants
Ethnicity (NIH/OMB) Not Hispanic or Latino	84 Participants	108 Participants	192 Participants
Ethnicity (NIH/OMB) Unknown or Not Reported	2 Participants	2 Participants	4 Participants
Race (NIH/OMB) American Indian or Alaska Native	0 Participants	0 Participants	0 Participants
Race (NIH/OMB) Asian	23 Participants	23 Participants	46 Participants
Race (NIH/OMB) Black or African American	4 Participants	5 Participants	9 Participants
Race (NIH/OMB) More than one race	4 Participants	5 Participants	9 Participants
Race (NIH/OMB) Native Hawaiian or Other Pacific Islander	0 Participants	0 Participants	0 Participants
Race (NIH/OMB) Unknown or Not Reported	4 Participants	7 Participants	11 Participants
Race (NIH/OMB) White	55 Participants	80 Participants	135 Participants
Region of Enrollment United States	90 Participants	120 Participants	210 Participants
Sex: Female, Male Sex Female	71 Participants	95 Participants	166 Participants
Sex: Female, Male Sex Male	19 Participants	25 Participants	44 Participants

Adverse events

Event type	EG000 affected / at risk	EG001 affected / at risk
deaths Total, all-cause mortality	0 / 148	0 / 212
other Total, other adverse events	0 / 148	0 / 212
serious Total, serious adverse events	0 / 148	0 / 212

Outcome results

Primary

Stages of Implementation Completion (SIC) - as Assessed by the Safe Environment for Every Kid (SEEK) Fidelity Score (at a Practice Level)

The SIC is an assessment tool with 8 stages extending from Engagement to achievement of Fidelity. Each stage maps onto 3 phases of implementation (Pre-Implementation, Implementation and Sustainment). Competence/Fidelity score reflects the extent to which a practice implemented the intervention as intended. Score range: 0-100; higher is better fidelity to the intervention as planned.

Time frame: up to 30 months

Population: The n is different because 6 practices dropped out early on and were not included in the analyses.~3 dropped out from each arm.

Arm	Measure	Value (MEAN)	Dispersion
Independent Online Training (IND)	Stages of Implementation Completion (SIC) - as Assessed by the Safe Environment for Every Kid (SEEK) Fidelity Score (at a Practice Level)	90.4 Score on a scale	Standard Deviation 6.9
Maintenance of Certification (MOC)	Stages of Implementation Completion (SIC) - as Assessed by the Safe Environment for Every Kid (SEEK) Fidelity Score (at a Practice Level)	91.0 Score on a scale	Standard Deviation 6.5

Primary

Stages of SEEK Implementation Completion (SIC) - Startup (at a Practice Level)

The SIC is an assessment tool with 8 stages extending from Engagement to achievement of Competency. Each stage maps onto 3 phases of implementation (Pre-Implementation, Implementation and Sustainment). Startup reflects whether the practice implemented SEEK - Yes/No. Yes = practice started implemented SEEK. Practices that did start are reported.

Time frame: up to 30 months

Arm	Measure	Value (NUMBER)
Independent Online Training (IND)	Stages of SEEK Implementation Completion (SIC) - Startup (at a Practice Level)	10 practices
Maintenance of Certification (MOC)	Stages of SEEK Implementation Completion (SIC) - Startup (at a Practice Level)	9 practices

Primary

Stages of SEEK Implementation Completion (SIC) - Sustainability

The SIC is an assessment tool with 8 stages extending from Engagement to achievement of Competency. Each stage maps onto 3 phases of implementation (Pre-Implementation, Implementation and Sustainability). Sustainability was determined if the practice was implementing SEEK 24 months after initiation and doing so with fidelity. Reporting the number of practices that achieved sustainability.

Time frame: up to 30 months

Population: The n is different because 6 practices dropped out early on and were not included in the analyses.~3 dropped out from each arm.

Arm	Measure	Value (NUMBER)
Independent Online Training (IND)	Stages of SEEK Implementation Completion (SIC) - Sustainability	6 practices
Maintenance of Certification (MOC)	Stages of SEEK Implementation Completion (SIC) - Sustainability	8 practices

Secondary

Overall Rates of Children With Possible Abuse and Neglect.

International Classification of Diseases - 10 (ICD-10) codes related to possible child maltreatment (CM or child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.

Time frame: 2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.

Population: Electronic Health Record (EHR) data were collected from 14 of the participating practices.

Arm	Measure	Group	Value (NUMBER)
Independent Online Training (IND)	Overall Rates of Children With Possible Abuse and Neglect.	Pre-SEEK	4.1 percentage of patients
Independent Online Training (IND)	Overall Rates of Children With Possible Abuse and Neglect.	Post-SEEK	4.3 percentage of patients
Maintenance of Certification (MOC)	Overall Rates of Children With Possible Abuse and Neglect.	Pre-SEEK	8.2 percentage of patients
Maintenance of Certification (MOC)	Overall Rates of Children With Possible Abuse and Neglect.	Post-SEEK	9.3 percentage of patients

Secondary

Rates of Children With Non-adherence to Medical Care

ICD-10 codes related to possible maltreatment (child abuse and neglect) are readily accessible through EHRs. De-identified aggregate data were gathered towards the end of the study for all children in the practice 0-66 months of age.

Time frame: 2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.

Population: EHR data were collected from 14 of the participating practices.

Arm	Measure	Group	Value (NUMBER)
Independent Online Training (IND)	Rates of Children With Non-adherence to Medical Care	Pre-SEEK	0.02 percentage of patients
Independent Online Training (IND)	Rates of Children With Non-adherence to Medical Care	Post-SEEK	0.01 percentage of patients
Maintenance of Certification (MOC)	Rates of Children With Non-adherence to Medical Care	Pre-SEEK	0.1 percentage of patients
Maintenance of Certification (MOC)	Rates of Children With Non-adherence to Medical Care	Post-SEEK	0.05 percentage of patients

Secondary

Rates of Children With Possible Abuse.

Time frame: 2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.

Population: EHR data were collected from 14 of the participating practices.

Arm	Measure	Group	Value (NUMBER)
Independent Online Training (IND)	Rates of Children With Possible Abuse.	Pre-SEEK	0.8 percentage of patients
Independent Online Training (IND)	Rates of Children With Possible Abuse.	Post-SEEK	0.7 percentage of patients
Maintenance of Certification (MOC)	Rates of Children With Possible Abuse.	Pre-SEEK	1.3 percentage of patients
Maintenance of Certification (MOC)	Rates of Children With Possible Abuse.	Post-SEEK	1.2 percentage of patients

Secondary

Rates of Children With Possible Neglect.

Time frame: 2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.

Population: EHR data were collected from 14 of the participating practices.

Arm	Measure	Group	Value (NUMBER)
Independent Online Training (IND)	Rates of Children With Possible Neglect.	Pre-SEEK	3.3 percentage of patients
Independent Online Training (IND)	Rates of Children With Possible Neglect.	Post-SEEK	3.6 percentage of patients
Maintenance of Certification (MOC)	Rates of Children With Possible Neglect.	Pre-SEEK	7.0 percentage of patients
Maintenance of Certification (MOC)	Rates of Children With Possible Neglect.	Post-SEEK	8.3 percentage of patients

Secondary

Rates of Under Immunized Children

Time frame: 2 years prior to implementing SEEK, 2 years after implementing SEEK., 48 months total.

Population: EHR data were collected from 14 of the participating practices.

Arm	Measure	Group	Value (NUMBER)
Independent Online Training (IND)	Rates of Under Immunized Children	Pre-SEEK	0.009 percentage of patients
Independent Online Training (IND)	Rates of Under Immunized Children	Post-SEEK	0.01 percentage of patients
Maintenance of Certification (MOC)	Rates of Under Immunized Children	Pre-SEEK	0.02 percentage of patients
Maintenance of Certification (MOC)	Rates of Under Immunized Children	Post-SEEK	0.018 percentage of patients

Secondary

SEEK Parent View of Relationship With Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale

Parents rate their child's PCP on this measure adapted for pediatric practice from the Patient-Doctor Interaction Scale. The Likert scale included 8 questions; the response set was from 1 (no) to 4 (always). Subscale mean scores range from 1 - 4; higher scores indicate greater satisfaction. The mean of the Likert subscale response is reported.

Time frame: 24 months

Population: Represents all the data collected from parents.

Arm	Measure	Value (MEAN)	Dispersion
Independent Online Training (IND)	SEEK Parent View of Relationship With Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.53 score on a scale	Standard Deviation 0.5
Maintenance of Certification (MOC)	SEEK Parent View of Relationship With Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.48 score on a scale	Standard Deviation 0.44

Secondary

SEEK Parent View of the Attentiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale

Time frame: 24 months

Population: Represents all the data collected from parents.

Arm	Measure	Value (MEAN)	Dispersion
Independent Online Training (IND)	SEEK Parent View of the Attentiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.67 score on a scale	Standard Deviation 0.38
Maintenance of Certification (MOC)	SEEK Parent View of the Attentiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.62 score on a scale	Standard Deviation 0.44

Secondary

SEEK Parent View of the Communication Skills of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale

Time frame: 24 months

Population: Represents all the data collected from parents.

Arm	Measure	Value (MEAN)	Dispersion
Independent Online Training (IND)	SEEK Parent View of the Communication Skills of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.61 score on a scale	Standard Deviation 0.41
Maintenance of Certification (MOC)	SEEK Parent View of the Communication Skills of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.51 score on a scale	Standard Deviation 0.43

Secondary

SEEK Parent View of the Comprehensiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale

Time frame: 24 months

Population: Represents all the data collected from parents.

Arm	Measure	Value (MEAN)	Dispersion
Independent Online Training (IND)	SEEK Parent View of the Comprehensiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	3.08 score on a scale	Standard Deviation 0.56
Maintenance of Certification (MOC)	SEEK Parent View of the Comprehensiveness of the Child's PCP Subscale - Adapted From the Patient-Doctor Interaction Scale	2.92 score on a scale	Standard Deviation 0.56

Secondary

SEEK PCP Questionnaire (PCPQ) - Addressing Food Insecurity

The PCPQ, used in both SEEK RCTs has 5 vignettes, with 7-12 statements assessing PCPs' thinking and practice with regard to addressing the targeted CM risk factors. Items were grouped conceptually into 6 topical scales (e.g., Substance Abuse) and 5 cross-cutting themes (e.g., Perceived Competence). The mean subscale scores ranged from 0-3. Higher scores reflect PCPs more optimal capability to address the problem.

Time frame: 30 months