Identifying Trends in Training Needs of Food Safety Professionals in Virginia

Introduction

The Food Safety Modernization Act (FSMA) was passed in 2011 and included the Preventive Controls for Human Food (PCHF) Rule (21 CFR Part 117). Under this rule, covered food businesses must employ at least one individual who has successfully completed a food safety training that the Food and Drug Administration (FDA) has deemed adequate or otherwise have sufficient job experience to develop and apply a food safety system (21 CFR §117.3).

Currently, the only recognized training course to meet this regulatory requirement is the Food Safety Preventive Controls Alliance (FSPCA) PCHF course. Businesses continue to struggle to apply learned concepts to ensure regulatory compliance. To efficiently target extension and outreach efforts that support the continued regulatory compliance needs of Virginia food businesses, it is necessary to evaluate the short-term outcomes of the PCHF trainings to better understand gaps in food safety knowledge.

To evaluate the short-term impact of this training, the FSMA Southern Center for Food Safety Training, Outreach and Technical Assistance developed a voluntary assessment to be administered before and after the course to assess participant change in knowledge as a result of instruction. It covered food safety concepts addressed in the course (Table 1) and, in 2017, was refined from a 23- to a 15-question version.

Methods

For every PCHF training hosted by Virginia Cooperative Extension in Virginia, pre-and post-tests were administered before and after trainings were delivered, respectively. Questions were categorized by module and topic area (Table 2). The surveys (n=480) were examined to assess the change in knowledge between pre- (244) and post- (236) test scores.

Metadata about the year the training took place (2017- 2022), the test version used (23- vs. 15-questions), and the format of the training (in-person, virtual, hybrid) were also collected. Modules and topics were sorted according to change in knowledge (low: <15%; high: >15%) and post-test score (low: <50%; high: >50%) before ranking. Rankings were assigned according to prioritized need, with highest priority given to modules and topics with low knowledge change and post-test scores and lowest priority given to those with high changes in knowledge and post-test scores. The data were analyzed using Crosstabs iQ in Qualtrics.

Results

Between 2017-2022, 244 pre-tests and 236 post-tests were taking in Virginia. In 2017, 81 pre- and 81 post- tests were taken on the 23-question test version and 18 pre- and 16 post-tests were taken on the 15-question version. After 2017, only the 15-question test was used. From 2018-2022, 145 pre- and 139 post-tests were collected.

The test did not include questions from each module (Table 2), and questions were not distributed evenly among the modules represented (Figure 1).

On average, 3 additional questions were answered correctly on the 23-question post-test (57.96%), and 3 additional questions were answered correctly on the 15-question post-test (54.87%) when compared to the pre-test. Given that (i) the number of questions answered correctly were similar across both test versions, (ii) the identical question content (except for the 8 questions that were removed), and (iii) the greater quantity of 15-question tests, the following analysis and recommendations are based on the 15-question test version only. For the 8 questions that were not carried over to the 15-question test, a short summary of those findings is included at the end of this section.

Overall, modules 3 (Good Manufacturing Practices and Other Prerequisite Programs), 12 (Supply-Chain Preventive Controls) and 15 (Recall Plan), resulted in consistently high scores ranging from 60.0-100.0%. Modules 5 (Chemical, Physical, and Economically Motivated Food Safety Hazards), 11 (Sanitation Preventive Controls), and 14 (Record-keeping Procedures all had consistently low scores ranging from 33.5-48.4% on both test versions. Modules 9 (Process Preventive Controls), 11, and 14 resulted in low changes in knowledge ranging from 14.0-19.9% and low post-test scores ranging from 33.5-43.9%. Knowledge change also varied within and between modules. For example, questions addressing module 4 content resulted in both low (8.1%) and high (11.5- 38.4%) changes in knowledge, depending on the question (Table 2).

Questions and their associated topics were then ranked based on priority, with 3 questions identified as high, 9 questions identified as medium, and 3 questions identified as low (Figure 1). High priority topics included corrective actions, while prerequisite programs, and food safety plans were low priorities. Other topics like hazard analysis, preventive controls, recalls, recordkeeping, and validation, while important, were identified as medium priority topics. Interestingly, food safety hazards were simultaneously identified as high, medium, and low priorities. Given that understanding of this topic was not consistently satisfactory (high post-test score), it is recommended that additional training, resources, or other guidance is provided to scaffold current professionals’ knowledge to enhance food safety outcomes.

Of the 8 questions that were not addressed on the 15- question test, food safety plans were identified as a potential high priority need based on post-test and knowledge change alone.

Discussion and Conclusions

Since the PCHF Rule, there have been improvements in Virginia professionals’ food safety knowledge. Despite these improvements, it is apparent that there are still gaps in understanding of food safety concepts. Additional educational resources (e.g., fact sheets, trainings, and other professional development opportunities) are needed in order to help businesses better understand and identify food safety hazards and implement appropriate food safety practices for their businesses. It is recommended that Extension educators prioritize educational initiatives that address content related to food safety hazards and corrective actions, in addition to emphasizing these content areas more in PCHF trainings. It will also be important to continue to monitor both change in participant knowledge and post-test scores to adequately assess if proposed educational initiatives are truly fostering better food safety outcomes across Virginia.

References

21 CFR Part 117