Research Capacity and Culture Development in a Small Rural Health Service
Dai Pu, Stephen Gill, Michael Field, Alison Buccheri, Olivia A. King, Catherine E. Huggins, Terry Haines, Joanne E. Porter, Vincent L. Versace, Laura Alston
TL;DR
This study shows that integrating research into a small rural health service in Australia improved organizational and team-level research culture over time, but not individual skills.
Contribution
The study provides empirical evidence on the impact of embedding research into rural health service structures on research capacity and culture.
Findings
Improvements were observed in 11 out of 18 organizational-level research capacity and culture measures.
Team-level improvements were seen in 6 out of 19 measures, but no individual-level improvements were found.
Strategies like embedded research units and external partnerships may enhance research culture in rural settings.
Abstract
This study aimed to measure changes in staff perceptions of research capacity and culture in a small rural health service in Australia over time. Staff completed the Research Capacity and Culture Tool, a valid and reliable survey that measures individuals' perceptions of their own research capacity and the research capacity and culture of their team and organisation. Data from 2015 was compared to 2023, following significant changes at the health service that focused on integrating research into the organisational structure. This was a repeated cross‐sectional study in which data were collected from different individuals. Data were collected from a rural health service in Victoria (Modified Monash Model 4–5). All staff working in the health service were invited to complete the survey. Research Capacity and Culture Tool. Results demonstrated improvements in eleven out of eighteen…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
| 2015 ( | 2023 ( | |
|---|---|---|
|
Male Female Unreported |
3 (6.0%) 33 (66.0%) 14 (28.0%) |
10 (13.5%) 56 (75.7%) 8 (10.8%) |
|
Clinical Staff Management Administration Others/Unreported |
31 (62.0%) 0 4 (8.0%) 15 (30.0%) |
35 (47.3%) 7 (9.5%) 14 (19%) 18 (24.3%) |
| 2015 ( | 2023 ( |
| |
|---|---|---|---|
|
| |||
|
| |||
| provides resources to support staff research training. | 5 (3–7) ( | 6 (4–7.5) ( |
|
| provides funds, equipment or admin to support research activities. | 4 (2–7) ( | 5 (4–8) ( |
|
| has a plan or policy for research development. | 5 (2.25–7) ( | 7 (5–9) ( |
|
| provides access to literature search and article retrieval. | N/A | 6 (4–8.5) ( | N/A |
| has senior managers that support research. | 6 (3–8) ( | 7 (5–9) ( |
|
| ensures staff career pathways are available in research. | 4 (2–7) ( | 5 (4.5–8) ( |
|
| ensures organisation planning is guided by evidence. | 6 (4–8) ( | 7 (5–8) ( |
|
| has consumers involved in research. | 5.5 (3–7) ( | 7 (5–8) ( |
|
| accesses external funding for research. | 5 (2–7) ( | 7 (5–9) ( |
|
| promotes clinical practice based on evidence. | 7 (5–8) ( | 7.5 (6–9) ( |
|
| encourages research activities relevant to practice. | 5 (3–7) ( | 7 (5–9) ( |
|
| provides software programs for analysing research data. | 3 (2–6) ( | 5 (4–7) ( |
|
| has mechanisms to monitor research quality. | 4 (2–7) ( | 6 (5–8) ( |
|
| provides experts for research advice. | 2 (1.5–6) ( | 7 (5–9) ( |
|
| supports a multi‐disciplinary approach to research. | 5 (2–7) ( | 6.5 (5–9) ( |
|
| provides forums or bulletins to present research findings. | 5 (1–7) ( | 7 (5.25–9) ( |
|
| engages external partners (e.g., Universities) in research. | 5 (1.5–7) ( | 8 (6–9) ( |
|
| supports applications for research scholarships or degrees. | 5 (3–7) ( | 6.5 (5–9) ( |
|
| supports the peer‐reviewed publication of research. | 5 (2–7) ( | 7 (5–9) ( |
|
| requires ethics approval for research activities. | N/A | 8.5 (7–10) ( | N/A |
|
| |||
| provides resources to support staff research training. | 4 (1.25–6.75) ( | 6 (5–8) ( |
|
| provides funds, equipment or administration to support research activities. | 3.5 (1–6) ( | 5 (5–7.5) ( |
|
| does team level planning for research development. | 4 (2–6) ( | 5 (5–7.5) ( |
|
| ensures staff involvement in developing that plan. | 4 (2–7) ( | 5 (5–8) ( |
|
| has team leaders that support research. | 5 (3–7) ( | 6 (5–8) ( |
|
| provides opportunities to get involved in research. | 4 (1.25–7) ( | 6 (5–8) ( |
|
| does planning that is guided by evidence. | 6 (4–8) ( | 7 (5–8) ( |
|
| has consumer involvement in research activities or planning. | 5 (2–8) ( | 5 (5–7.75) ( |
|
| has applied for external funding for research. | 4 (1–6) ( | 5.5 (4–8) ( |
|
| provides access to literature searching and article retrieval. | N/A | 5 (3.5–8) ( | N/A |
| conducts research activities relevant to practice. | 4 (1.25–7) ( | 6 (4.5–8) ( |
|
| supports applications to research scholarships or degrees. | 5 (3–7) ( | 5 (5–8) ( |
|
| has mechanisms to monitor research quality. | 3.5 (1–7) ( | 5 (4–8) ( |
|
| provides experts accessible for research advice. | 3 (1–6.75) ( | 5 (5–8) ( |
|
| disseminates research results at research forums or seminars. | 4 (1–7) ( | 6 (5–8) ( |
|
| supports a multi‐disciplinary approach to research. | 5 (2–7) ( | 5 (5–8) ( |
|
| has incentives and support for mentoring activities. | 4 (2–7) ( | 5 (5–7) ( |
|
| has external partners (e.g., universities engaged in research). | 4.5 (2–7) ( | 5 (5–8) ( |
|
| supports peer‐reviewed publication of research. | 5 (1.25–7) ( | 5 (5–8) ( |
|
| provides software to support research activities. | 3 (1–7) ( | 5 (3–6) ( |
|
|
| |||
| find relevant literature. | 8 (6–8) ( | 8 (6–9) ( |
|
| critically review the literature. | 7 (4–8) ( | 7 (5–8) ( |
|
| use a computer referencing system (e.g., Endnote). | 6 (2–8) ( | 5 (3–8) ( |
|
| write a research protocol. | 5 (2–6.5) ( | 5 (3–8) ( |
|
| secure research funding. | 3 (2–5.75) ( | 5 (2–8) ( |
|
| submit an ethics application. | 3 (1.25–5.75) ( | 5 (2–8) ( |
|
| design questionnaires. | 6 (2.5–7.5) ( | 7 (5–9) ( |
|
| collect data, e.g., surveys, interviews. | 7 (3.5–8) ( | 7 (5–9) ( |
|
| use computer data management systems. | 6 (2–8) ( | 6 (5–9) ( |
|
| analyse qualitative research data. | 5 (2.5–7) ( | 7 (5–9) ( |
|
| analyse quantitative research data. | 5 (2.25–7) ( | 6.5 (3.5–8) ( |
|
| write a research report. | 6 (2–7) ( | 6 (3–8) ( |
|
| write for publication in peer‐reviewed journal. | 2.5 (1–6) ( | 5 (2.25–7.75) ( |
|
| integrate research findings into practice. | N/A | 7 (5–8) ( | N/A |
| provide advice to less experienced researchers. | 3 (1–6) ( | 5 (3.25–8) ( |
|
- —Medical Research Future Fund10.13039/501100025520
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHealth Policy Implementation Science · Health Sciences Research and Education · Primary Care and Health Outcomes
Introduction
1
Research capacity and culture can influence the clinical and public health practices of an organisation [1, 2, 3]. Good clinical practice relies on the accurate understanding, interpretation, and translation of research evidence. At the same time, the knowledge and skills needed to design and conduct scientifically sound research are important for clinicians and their organisations to contribute to evidence‐based practice. Understanding the research skill levels and the supports required will identify the training needs in an organisation and inform targeted strategies to address these needs.
Research capacity building has been defined as a process of developing sustainable abilities and skills enabling individuals and organisations to perform high quality research [4]; and research culture can be considered the “research environment” [4] that supports the research and can be indicative of the value placed on research by the organisation. Research capacity is dynamic and can be observed at multiple levels of a health care organisation, including individuals, teams and departments [5].
Programs designed to improve health service research capacity and culture can struggle with sustainability due to lack of funding and inconsistent evaluation [6]. Key to ensuring ongoing progress and improvement is to systematically track and evaluate these investments and their outcomes [6, 7].
Research capacity in rural and regional areas is consistently lower compared to metropolitan areas where large research universities and health services are concentrated [3, 8, 9, 10]. Research education is also less accessible in these areas [9]. Without intervention, research culture and capacity can remain stagnant over time [11]. Increasingly, rural health services are building research into their organisations to improve research capacity and culture and attract and retain critical health workforce. Colac Area Health (CAH) provides health care services to communities in areas classified as MM4 and MM5 by the Modified Monash Model. Since 2020, the organisation has established a research unit, which integrated research into the organisational structure, strengthened by external partnerships with the local University Department of Rural Health (Deakin Rural Health) and the Western Alliance Academic Health Science Centre, and has been described in detail elsewhere [12, 13, 14, 15]. In short, the research unit was included in the organisational structure and the strategic plan, with direct reporting to the CEO via the research unit director. Its aim was to support staff to develop their research skills and ideas, as well as house research staff leading externally funded projects at CAH. The research unit started with two staff members in 2020 and has since included up to 8 staff members depending on research funding.
This study aimed to measure changes in staff perceptions of the research capacity and culture at CAH over time.
Materials & Methods
2
Research capacity and culture was compared across two time points. Historical data was obtained from a study that used the Research Capacity and Culture (RCC) Tool [4] to survey eight health services in Victoria in 2015 [3]; RCC data was collected at one of these health services again in 2023 [13]. Both studies were approved by Barwon Health Human Research Ethics Committee (2015 reference: 14/136; 2023 reference: 23/29), with additional approvals for data sharing and comparisons for this study.
Participants and Setting
2.1
Data were collected from CAH, a rural health service that serves a region classified as small‐medium rural towns on the Modified Monash Model [16] (MM4‐5). CAH services a population of approximately 32 000 via one urgent care centre, 28 inpatient beds that encompass medical, surgical and maternity care, specialist clinics, aged care, and community health services [17]. At both time points of data collection, all staff in the health service were invited to participate. There were 361 non‐casual staff members employed at CAH in 2015, and 598 staff in 2023.
Survey Tool
2.2
The RCC tool [4] was used to measure research capacity and culture. The RCC tool is based on the research capacity building framework proposed by Cooke in 2005 [18], and asks health service staff 55 questions: 20 questions each for how they perceived their team and organisation's research capacity and culture, and 15 questions for how they perceived their own research skills. Each question asks for a rating from 1 (least success) to 10 (most success). The 2015 survey excluded four questions: one question about individual research capacity (You integrate research findings into practice), one question about team‐level research culture (Your team provides access to literature searching and article retrieval), and two questions about organisational research culture (Your organisation provides access to literature searching and article retrieval and Your organisation requires ethics approval for research activities). A total of 51 questions were compared between the two survey rounds.
Data Collection
2.3
The survey was distributed online using REDCap at both timepoints. Recruitment was conducted via emails sent to individual staff members' organisation email. One email was sent to invite staff to complete the survey in 2015; up to two emails were sent in 2023. Respondents did not have to provide a rating for every RCC question if they chose not to.
Analysis
2.4
Ratings for RCC tool questions were described as medians and interquartile ranges. Comparisons across time were conducted using the non‐parametric Kruskal–Wallis test. Paired analysis was not conducted as individuals were not matched across time. Holm's–Bonferroni corrections were used to determine statistical significance for each level of the RCC Tool to account for the higher probability of Type 1 errors due to multiple comparisons.
Results
3
In the 2015 dataset, there were 50 responses (13.9%) from a total of 361 non‐casual staff members who were sent the email inviting survey completion; in the 2023 dataset, there were 74 responses (12.4%) from a total of 598 staff employed by CAH. Respondent characteristics are reported in Table 1.
From 2015 to 2023, CAH staff reported perceived improvements in 11 out of 18 measures of research capacity and culture at the organisation level, 6 out of 19 measures at the team level, but none at the individual level (Table 2). Significant improvements were most commonly a median of two points. One item (My organisation provides experts for research advice) demonstrated a median improvement of 5 points. No deteriorations were observed for any measures.
Discussion
4
This study found improvements in research capacity and culture over time in a rural health service in Victoria, after significant efforts were made to integrate research into the organisation. These improvements were reported primarily for items at the team and organisation levels, with non‐significant changes at the individual level. While team and organisational improvements were statistically significant, these changes were typically modest in size and between 1 to 3 points. The largest improvement was reported for CAH's provision of experts for research advice, which increased from a median rating of 2 to 7. This improvement could be the consequence of introducing the research unit which increased the availability of research experts to provide advice to the organisation.
The introduction of an embedded research unit in 2020 dedicated to supporting research activities in the rural setting [12, 15, 19] may have contributed to improvements to research capacity and culture. The 2023 dataset was analysed and reported in a previous study [13], which found that staff who had received support from the research unit reported higher ratings for organisational research culture and individual research capacity compared to those who had not [13]. This result was similar to a study that reported the introduction of a dedicated research support office was associated with increased individual research capacity in a metropolitan health service [20]. In contrast, the current comparison across time only found change at the organisational and team levels, but not the individual level. This might be due to a few reasons. First, individual RCC levels have tended to be higher than team and organisation levels in previous studies [10], indicating a possible ceiling effect. Second, the high turnover rate in more remote health services [8] could mean that the same staff who might have reported improvements in research skills were not captured after leaving the health service. Finally, a research unit integrated into the organisational structure with supporting policies and locally employed staff may be most effective in improving research culture at higher levels, but benefits may not always filter to team/departmental and individual levels in short time periods [15].
Future research should monitor the reach and effects of having integrated research units in rural health services. This could continue to utilise the RCC tool survey used in the current study, as well as qualitative interviews to gather more detailed information from staff. Traditional research metrics such as the number of peer‐reviewed publications and grants received could also be used to evaluate the effects of initiatives to improve research culture and capacity.
Limitations
5
Data used in the current study was limited to a single rural health service, limiting the generalisability of the findings. The RCC tool measures staff‐perceived research culture and capacity, which is subjective and may not be reflective of the actual research support infrastructure and processes in place. While associations may be drawn between the embedded research unit in CAH and improvements in staff‐perceived research capacity and culture, no direct causal relationship can be concluded. The repeated cross‐sectional design of this study means that the same participants were not followed over time, so the observed changes could be due to the characteristics of each sample rather than changes over time. While the survey was sent to all health service staff and response rates were similar across time (13.9% in 2015 and 12.4% in 2023), it is possible that only those with an interest and active engagement in research activities responded to the survey, thereby potentially limiting the representativeness of the findings.
Conclusion
6
Embedded research units that support research conduct in rural health services may be a promising approach to supporting the growth of research culture and capacity for the rural health workforce.
Author Contributions
Alison Buccheri: data curation, methodology, writing – review and editing. Laura Alston: data curation, methodology, writing – review and editing. Vincent L. Versace: data curation, writing – review and editing. Terry Haines: methodology, writing – review and editing. Dai Pu: conceptualization, data curation, formal analysis, investigation, methodology, project administration, writing – original draft, writing – review and editing.
Funding
This work was supported by The Medical Research Future Fund, RARUR000072.
Disclosure
The data used in this study were previously published in two different papers that reported single timepoint analyses.
Conflicts of Interest
Authors Alston, Field and Buccheri have roles within the health services from which this study's data was collected. Data analysis was conducted by Dai Pu, who has no current or prior relationship with these health services to address this conflicts of interest.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1A. Boaz , B. Goodenough , S. Hanney , and B. Soper , “If Health Organisations and Staff Engage in Research, Does Healthcare Improve? Strengthening the Evidence Base Through Systematic Reviews,” Health Research Policy and Systems 22, no. 1 (2024): 113.39160553 10.1186/s 12961-024-01187-7PMC 11331621 · doi ↗ · pubmed ↗
- 2K. Harding , L. Lynch , J. Porter , and N. F. Taylor , “Organisational Benefits of a Strong Research Culture in a Health Service: A Systematic Review,” Australian Health Review 41, no. 1 (2016): 45–53.10.1071/AH 1518027074113 · doi ↗ · pubmed ↗
- 3S. D. Gill , S. M. Gwini , R. Otmar , S. E. Lane , F. Quirk , and G. Fuscaldo , “Assessing Research Capacity in Victoria's South‐West Health Service Providers,” Australian Journal of Rural Health 27, no. 6 (2019): 505–513.31814198 10.1111/ajr.12558 · doi ↗ · pubmed ↗
- 4L. Holden , S. Pager , X. Golenko , and R. S. Ware , “Validation of the Research Capacity and Culture (RCC) Tool: Measuring RCC at Individual, Team and Organisation Levels,” Australian Journal of Primary Health 18, no. 1 (2012): 62–67.22394664 10.1071/PY 10081 · doi ↗ · pubmed ↗
- 5J. Cooke , “Building Research Capacity for Impact in Applied Health Services Research Partnerships Comment on “Experience of Health Leadership in Partnering With University‐Based Researchers in Canada–A Call to re‐Imagine Research”,” International Journal of Health Policy and Management 10, no. 2 (2020): 93.10.15171/ijhpm.2020.11PMC 794766332610775 · doi ↗ · pubmed ↗
- 6O. King , E. West , L. Alston , et al., “Models and Approaches for Building Knowledge Translation Capacity and Capability in Health Services: A Scoping Review,” Implementation Science 19, no. 1 (2024): 7.38287351 10.1186/s 13012-024-01336-0PMC 10823722 · doi ↗ · pubmed ↗
- 7A. Brown , A. Edelman , T. Pain , S. Larkins , and G. Harvey , ““We're Not Providing the Best Care if we Are Not on the Cutting Edge of Research”: A Research Impact Evaluation at a Regional Australian Hospital and Health Service,” International Journal of Health Policy and Management 11, no. 12 (2022): 3000–3011.35643418 10.34172/ijhpm.2022.6529 PMC 10105178 · doi ↗ · pubmed ↗
- 8D. J. Russell , Y. Zhao , S. Guthridge , et al., “Patterns of Resident Health Workforce Turnover and Retention in Remote Communities of the Northern Territory of Australia, 2013–2015,” Human Resources for Health 15, no. 1 (2017): 52.28810919 10.1186/s 12960-017-0229-9PMC 5558760 · doi ↗ · pubmed ↗