Examining the health of family medicine research

Part 1

By David A. Katerndahl, M.D., M.A.

In the 35 years that family medicine has been a specialty, it has struggled with the tensions between its commitment to patient care and its position as an academic discipline. There is an inherent conflict between family medicine as a “counterculture” focused on patient care and family medicine as an “evidence-based” specialty focused on generating that evidence. Consequently, the growth of family medicine research has been slow and often unsupported by federal granting agencies and practitioners.

Yet, from the start, many of the leaders in family medicine recognized the necessity of research if the discipline was to achieve equal recognition within the academic community and if the specialty was to provide cutting-edge quality of care to its constituency. Hence, the 1970s was the “age of the research trailblazer” while the ’80s was the “age of the research evangelist,” trying to get the word out and grow converts. The ’90s finally saw the “age of research commitment” with family medicine organizations like AAFP throwing their full support behind the movement. But did 2000 mark the blossoming of those efforts? Has the 30-year process finally come to fruition?

Status of Research Productivity in the United States

The most recent studies of the family medicine research literature summarized the research productivity in 2000 (Pathman et al., 2005) and again in 2003 (Pathman et al., 2008). The 2003 study found that 801 family medicine researchers generated 790 articles published in 285 journals (see Table 1). Compared with the 2000 results, the total number of articles increased by 58 percent and the number of researcher-authors had increased by 41 percent. Although only 60 percent of these articles included a family medicine researcher as lead author and the increase in physician researchers was only 29 percent, these still represent significant increases in both the volume of research articles and the number of family medicine researchers publishing. As might be expected, family medicine journals could not meet this growth so the proportion of these articles published in family medicine journals declined.

In addition to an increase in volume, there are signs that the quality of the research is also improving. Sonis and Joines (1994) found that the quality of the clinical trials published from 1974 to 1991 in the Journal of Family Practice was increasing. Pathman reported that the number of articles published in top-tier general medicine journals (i.e. JAMA, New England Journal of Medicine), though small, increased by 57 percent from 2000 to 2003. Yet, when the content of family medicine research articles was compared with the rank order of the content of family practice patients, good agreement was found. The only major differences found were an under-representation in the research literature of minor but common problems (e.g. cerumen) but an over-representation of serious but uncommon problems (e.g. cancer and AIDS). This speaks to the relevance of the family medicine research literature (Katerndahl et al., 1998).

Status of the Practice-based Research Network

The research tool that was developed by family medicine is the practice-based research network, or PBRN, linking physician practices together to conduct research studies in representative settings. The 86 primary care PBRNs that responded to a survey in 2003 reported that they included 1,871 practices, 12,957 physicians and 14.7 million patients. These PBRNs had generated over 600 publications. However, only 20 were PBRNs consisting exclusively of family physicians. As a group, they were young PBRNs with little research experience and continued to struggle for funding (Tierney et al., 2007).

Continuing Barriers

Although family medicine has finally begun to achieve a substantial level of research productivity and its PBRNs have reached a critical mass to ensure an ongoing supply of generalizable studies, significant barriers to research still exist. Although we may have a critical mass of experienced researchers in academic departments, many of the beginning researchers and those affiliated with community-based residencies still lack adequate research training and skills. With the termination of Title VII federal funds that were used for faculty development and research infrastructure, the lack of research skill development and infrastructure opportunities is critical to the ongoing support of our research effort.

In addition, there remains a lack of research mentors, particularly for non-university-based researchers. Opportunities for mentor development and non-local mentor-protégé pairing must be sought.

Finally, without an Institute of Primary Care in NIH, there remain serious funding barriers to the conduct of major primary care research. The Agency for Healthcare Research and Quality is underfunded to meet this need. Even though existing NIH institutes are beginning to fund some family medicine projects on a modest scale, research funding continues to be the single greatest barrier to family medicine research (Katerndahl, 2006).

Evidence shows that family medicine continues to increase its research productivity. The number of PBRNs has reached a critical mass and should yield important results for practice. However, in addition to the stress imposed by the end of Title VII funds, the perpetual problems of lack of mentorship and federal research funds continue to represent important barriers to further development of the family medicine research base.

References

Katerndahl DA, Burge S, Schneider FD, Legler J: Clinical content of the Family Practice research literature. Texas Fam Physician 1998; July/August issue:26-27.

Katerndahl DA: Directing Research in Primary Care. Seattle: Radcliffe Publishing, 2006.

Pathman DE, Gamble G, Thaker S, Newton WP: Metric of progress for Family Medicine research. Ann Fam Med 2005; 3:88-9.

Pathman DE, Viera AJ, Newton WP: Research published in 2003 by U.S. family medicine authors. J Am Board Fam Med 2008; 21:6-16.

Sonis J, Joines J: Quality of clinical trials published in the Journal of Family Practice, 1974-1991. J Fam Pract 1994; 39:225-35.

Tierney WM, Oppenheimer CC, Hudson BL, Benz J, Finn A, Hickner JM, Lanier D, Gaylin DS: National survey of primary care practice-based research networks. Ann Fam Med 2007; 5:242-50.

---

Outcomes of Foundation-supported research

Part 2

Of the 58 studies funded by the TAFP Foundation, 47 were completed by 2006 and can be evaluated in terms of their outcomes. Did these completed studies lead to presentations at national conferences or publications? Did they serve as pilot studies for larger, grant-supported research?

Recent Outcomes

Of the 21 studies completed between 2002 and 2006, information about outcomes was provided for 15 studies. Table 2 summarizes the numbers of conference, manuscript and grant submissions resulting from these 15 studies. Overall, there were a total of 42 submissions from 13 studies with almost half being manuscripts submitted for publication. The last column displays the mean and range of submissions over all 15 completed studies reporting. Overall, the 15 studies produced an average of 2.8 submissions.

Not every submission leads to a presentation, publication or funded grant. Table 3 presents the results of these submissions. Overall, there were 33 outcomes from 12 studies. Again, almost half of the outcomes were publications. The final column displays the mean and range of outcomes across all 15 completed studies. The 15 studies produced an average of 2.2 outcomes.

In addition to the 42 submissions reported above, eight researchers say they are planning to make 18 more submissions based on these studies, nine of which are manuscripts. Taken together then, these 15 completed studies have or will generate a total 60 submissions, an average of four submissions per study. Thus, 60 percent of the studies will produce at least one conference submission, 14 will result in at least one manuscript, and seven will lead to at least one grant application. Only one study will not result in at least one submission.

Comparison With Older Studies

Comparing these results with those of 26 studies completed prior to 2002, Table 4 summarizes the outcomes. Fewer recently-completed studies (53 percent versus 63 percent) produced fewer presentations (13 versus 18). However, more recent studies (60 percent versus 25 percent) yielded more publications (15 versus seven). There was little difference in funded grants (five versus four) or the number of studies yielding them (20 percent versus 25 percent). Overall, the more recently-completed studies resulted in more outcomes, particularly publications (2.2 versus 1.81 outcomes, on average).

Implications

The right-hand columns of Tables 2 and 3 reveal the current yield for this grant support. With a maximum of $5,000 given per study, the 15 studies completed between 2002 and 2006 cost the Foundation less than $75,000. For that $75,000, the discipline received 13 conference presentations and 15 publications. In addition, these studies resulted in five funded grants for a total of $659,000 in funding, a nine-fold return on investment! This analysis suggests a high yield in outcomes for the modest financial support offered by the Foundation.

---

TAFP Foundation-based support for research
in Texas

Part 3

As the philanthropic arm of the Texas Academy of Family Physicians, the TAFP Foundation emphasizes scientific and educational activities in support of the discipline and the promotion of health among patients in Texas. The objective of the Research Grants Program is to stimulate, support, and disseminate family medicine research that promotes high-quality family medicine care and health for the people of Texas. Funding priorities for the program include:

• family medicine manpower and distribution,
• family medicine outcome studies,
• ambulatory care studies, and
• other studies in support of the discipline.

Since 1988, the TAFP Foundation has offered funding support for family medicine research in Texas.

Record of Support

Despite the limited availability of funds, the Research Grants Program has provided a total of about $210,000 in support of 58 studies to date. At any point in time, the Foundation supports up to 12 studies. Table 5 summarizes the distribution of areas supported through this program.

Half of the studies focused on ambulatory care for a variety of clinical problems. In addition, these 58 funded studies represent the work of 44 different researchers from across the state of Texas. About 75 percent of the completed studies have published a summary in TEXAS FAMILY PHYSICIAN.

Financial Stewardship

Due to the limited availability of funds, the Research Grants Program attempts to ensure the maximum value in its funding decisions. This policy enables the Foundation to support as many studies as possible.

Grant applications are reviewed by the Research Grants Committee based upon the following criteria:

• relevance to family medicine,
• quality of study design,
• appropriateness of data analysis, and
• cost versus benefit.

Funding of approved studies is dependent upon their approval by the local Institutional Review Board to ensure protection of human subjects. In addition, researchers are required to submit progress reports twice annually and a final report at the conclusion of the study.

The Foundation employs several budgetary policies designed to limit funding expenses. For instance, requested direct costs are limited to a maximum of $5,000 per study. TAFP Foundation funds can only be used for direct costs; no indirect costs are supported. Travel funds for the presentation of results at conferences are not provided and because conducting research is an expected part of employment, faculty in academic or teaching programs cannot receive funding to support their time spent on research. Funds cannot be used to purchase computer equipment; this should be provided by their institutions. All expenses must be clearly justified in the application, any unused funds must be returned to the Foundation and the Research Grants Committee must approve any rebudgeting of funds. In addition, when funded studies are to be conducted at least partially outside of Texas, the Foundation provides half of the funding and encourages the investigator to seek matching funds from AAFP.

These policies have resulted in a high yet conservative funding rate. Since 2005, 18 grant applications have been funded. However, 22 percent of these funded studies received less than the amount requested. Thus, in 20 years, the Foundation has supported 44 researchers involved in 58 studies for little over $200,000.

---

Supporting the future development of research
in Texas

Part 4

As the previous articles have demonstrated, there has been steady growth in family medicine research activity and outcomes in Texas over the past two decades. In 2007, the TAFP Foundation developed a strategic plan for research support to provide guidance for the Research Grants Committee and a blueprint for the future. This plan focuses on three areas of research development: funding support for research, support for research infrastructure and dissemination of research results.

Funding Support for Research Studies

Currently, the Foundation commits between $25,000 and $30,000 annually to support unsolicited research studies with a maximum of $5,000 per study. The plan seeks to expand the support for unsolicited studies as well as instituting an annual solicitation of studies on an area of Texas-related medical need. Eventually, the Foundation seeks to expand its support for unsolicited studies from its current annual level to $100,000 with a maximum limit of $20,000 per study. This would enable us to permit financial support for investigator research time as well as direct expenses of the research.

In addition to supporting unsolicited studies on any relevant topic, the Foundation wants to begin focusing on specific medical needs for patients in Texas. To accomplish this, the Foundation would annually seek grant applications on a different clinical topic each year chosen with input from the TAFP membership. This program would commit $50,000 each year to support one to two studies.

Support for Research Infrastructure

Major research often depends upon results of pilot work to establish feasibility and guide research design. Such preliminary study requires continual support of research staff, even between periods of external funding. In addition, the presence of permanent research staff facilitates the participation of residents in research. Since the termination of Title VII funding, federal support for research infrastructure has largely ended. Ideally, each Texas family medicine residency program should have a half-time research assistant available permanently to meet this need. However, based upon minimal salary with fringes of around $10,500, providing all 28 Texas-based residency programs with such research infrastructure would require $294,000 annually. Although a worthwhile goal, such support is currently beyond the capabilities of the Foundation.

However, the Foundation can support research infrastructure in other ways. The Foundation seeks measures to support faculty development of research skills, practice-based research networks, or PBRNs, in Texas and for presentation of resident research results at national research meetings.

Because many faculty in community-based residencies lack research skills, the plan would provide travel support for five such faculty members to attend one of two annual research meetings that focus on development of research skills: the AAFP PBRN Convocation and the Primary Care Research Methods and Statistics Conference. Recipients would be chosen by the Foundation’s Board of Trustees. Second, the Foundation would support Texas-based PBRNs in two ways. To facilitate cooperation among these PBRNs, an annual meeting of their directors would occur in conjunction with the Interim Session to promote exchange of information. The Foundation would also provide $50,000 annually to support one to two studies conducted by TAFP’s PBRN, the Statewide Primary care Ambulatory Research and Resource Consortium, or SPARRC. Finally, recent surveys among U.S. family medicine residents show that although many residents express interest in attending research-related conferences, travel funds to attend research conferences are generally not available to residents, even if they are to present results of research they have conducted. The Foundation plans to support two residents each year to present results of their research at an annual family medicine research meeting, like the AAFP Scientific Assembly, the Society of Teachers of Family Medicine annual conference and the North American Primary Care Research Group.

Dissemination of Research Results

To promote recognition of the value of Foundation-supported research and encourage research-related donations to the Foundation, it is important to periodically evaluate and disseminate the results and impact of Foundation-supported research to the TAFP membership. The Foundation proposes dissemination of research results at Annual Session and the routine assessment of the impact of Foundation-supported research.

Currently, research results do not figure prominently in the program of the TAFP scientific assembly. The Foundation proposes that the presentation of research posters be re-established into the TAFP Annual Session. In addition, the Foundation plans to work with the TAFP Program Committee to establish a plenary presentation at the scientific assembly that includes research results generated through a Foundation-supported study. This presentation should occur during a high-profile position in the schedule and the research results should be integrated into a more general clinical topic.

Previous assessment of the outcomes of studies receiving Foundation support revealed that most led to publication and/or presentation of results, or were instrumental in the pursuit of subsequent major funding from external agencies. However, such evaluations are not currently performed on a regular basis. The Foundation will contact funded investigators every five years to determine the impact of their studies in terms of subsequent publications, national presentations, grant proposals and research awards. Such results should be disseminated in TEXAS FAMILY PHYSICIAN and at the TAFP Annual Session.

Cost of the Proposed Developments

Although the plans for dissemination of results and for an annual meeting of Texas-based PBRNs are without cost and currently being pursued, the other measures involve significant costs. The proposals for funding support for research studies will cost $150,000 annually. The proposals for support of research infrastructure would cost $5,000 in support of faculty development for community-based residency faculty and $3,000 for resident travel to present the results of their research. The proposals in support of Texas-based PBRNs would cost $50,000. Thus, full implementation of this strategic plan will cost the Foundation $208,000 annually.

Seeking to build upon the success of efforts to develop and expand family medicine research in Texas, this strategic plan attempts to take research in Texas to the next level. But, to succeed, this will require significant commitment from the Foundation and its financial supporters.