EVALUATING
YOUR SAFETY AND HEALTH PROGRAM
INTRODUCTION
The
scenario: Your safety and health program
is in place. You have set your goal for
the year and clearly stated the objectives,
procedures, and activities necessary to
meet that goal. Responsibilities have been
defined and clearly assigned. Adequate authority
and resources have been allocated. People
have been trained in their safety and health
program roles, and they understand the consequences
of failing to perform their assignments.
Your
responsibility for employee safety and health
does not stop here. The next step -- a critical
one -- is to evaluate how well your safety
and health program is working.
This
process is more than an inspection or an
audit. Inspections are necessary to look
at the facility, the process and the individual
jobs in order to identify and then to eliminate
or control any hazards that may exist. Audits
focus on program activities and seek to
determine whether specific objectives have
been met. For example, if you are assessing
employee participation by looking at the
activities of the safety committee you will
want to know if that committee met at the
intervals specified, and if most of the
members attended each meeting. These are
audit questions.
But
beyond this simple accounting are larger
questions. For example, has employee participation
at safety committee meetings helped improve
the worksite's safety and health program?
How is the work of the safety committee
helping you meet your goal? These are the
kinds of issues addressed by an evaluation.
A
safety and health evaluation looks at the
systems you have created to carry out your
safety and health program. It asks if these
systems are working effectively and efficiently.
All systems that contribute to your safety
and health program should be reviewed. These
should include management leadership and
the evaluation of that leadership, the analysis
of the worksite to identify hazards, hazard
prevention and control, accident and near
miss investigations, employee involvement,
safety and health training, use of personal
protective equipment (PPE), the health program,
and the emergency response program. The
site may have additional programs or systems
that contribute to the safety and health
program. You will need to evaluate these
also.
Who
should conduct the evaluation? Although
evaluations can be performed by worksite
employees, they are best done by people
who are knowledgeable about the site's processes
and managing safety and health programs.
The evaluator should not work at the site
being evaluated. The fresh look an outsider
brings produces a more accurate and helpful
evaluation. This outsider may come from
corporate headquarters, another worksite
within the company, an insurance company
or a consulting firm.
Evaluation
often causes anxiety for workers. You may
be able to reduce that anxiety by letting
your employees know that the evaluator will
be focusing on systems and not on people.
Three
useful tools for this evaluation are document
review, employee interviews and review of
site conditions. These tools will provide
the basis for an evaluation report. This
report should contain a list of the programs
or systems reviewed and a narrative account
of the examination of each system or program.
It also should contain a schedule of needed
changes with target completion dates, responsible
parties, and space to indicate the date
when changes are actually completed. Some
reports include pictures of excellent situations
and those needing improvement. Some provide
a grading system, so that each year's results
can be compared quickly to previous years.
This report should be available to any employee
who wants to read it. This chapter will
explain in detail how you can accomplish
this evaluation.
WHAT
SHOULD BE EVALUATED?
Ideally,
everything that you know to be contributing
directly to your safety and health program
should be evaluated. OSHA's Safety and Health
Program Management Guidelines can help you
determine which areas of your program need
evaluation. These are the four major areas
of the Guidelines, that are called Major
Elements:
1.The
demonstration of management leadership and
employee involvement through:
-
Setting and communicating the safety and
health policy;
-
Setting and communicating a clear goal
and objectives;
-
Being visibly involved in employee safety
and health;
-
Assuring employee involvement in safety
and health problem identification and
resolution;
-
Assigning clear responsibility for safety
and health;
-
Giving adequate authority and assuring
efficient use of resources;
-
Holding all personnel accountable; and
-
Assuring quality.
2.Worksite
analysis to identify existing and potential
hazards through:
-
Comprehensive safety and health hazard
surveys;
-
Analysis of planned changes to identify
hazards that might be introduced;
-
Routine hazard analyses, such as:
-
Job hazard analysis (also known
as job safety analysis),
- Process hazard analysis (used in industries
with complex and hazardous processes),
and
- Phase hazard analysis (used mainly
in construction);
Periodic worksite inspections, including:-
Self-inspections conducted by supervisors
in their work areas, and
-
General inspections of the entire site
conducted by safety and health staff;
-
Employee reports of hazards;
-
Work practice control; and
-
Analysis of injury/illness trends.
3.
Hazard prevention and control through:
-
Engineering controls;
-
Work practice control;
-
Personal protective equipment;
-
Administrative controls;
-
Disciplinary systems to enforce controls;
-
Preventive maintenance;
-
Emergency preparedness; and
-
Medical program.
4.
Safety and health training to ensure that
all employees know how to protect themselves
and others from existing and potential hazards
of the worksite.
WHO
SHOULD EVALUATE
Evaluators
can be drawn from the workplace safety and
health department or the safety committee,
but the best evaluators will be people possessing
fresh vision. They will not be involved
in the day-to-day operations of the site.
Look in the corporate safety department,
another worksite of the company, insurance
companies and outside consulting firms.
Or have two activity managers switch places
and evaluate each other's results.
Many
workers' compensation insurance carriers
offer their clients the services of a certified
safety professional and a certified industrial
hygienist. These experts are qualified to
review your program activities.
Evaluators
should be knowledgeable in the technical
aspects of occupational safety and health,
the management of safety and health, and
the evaluation of programs. Of these three
areas, management of safety and health is
the most important.
TOOLS
FOR COLLECTING INFORMATION USED IN EVALUATION
There
are three indispensable evaluation tools
for judging the effectiveness of occupational
safety and health program management. These
are:
-
Document review,
-
Interviews with employees at different
levels, and
-
Review of site conditions.
See
Appendix 12-2 for a detailed description of
how to use these tools.
Documentation.
Every worksite will have, at an absolute
minimum, written accident reports and the
OSHA log of injuries and illnesses as required
by law. Major companies should have written
procedures and records of all their safety
and health programs. The evaluator should
compare the written program to the written
records of what occurred.
Interviews.
In addition to the documentation, interviews
can be very helpful in establishing what
has occurred. We use two kinds of interviews,
formal and informal. The formal interviews
are conducted privately with randomly selected
employees who are asked preselected questions.
Informal interviews occur at employee work
stations and generally follow a list of
topics.
To
assess how well the worksite safety and
health policy is communicated and understood
and how well the disciplinary system is
working, ask the employees to explain them.
To
gauge the effectiveness of safety and health
training, interview hourly employees and
first-line supervisors. Ask employees to
describe what hazards they are exposed to
and how they are protected. Ask them to
explain what they are supposed to do in
several different types of emergencies.
Ask supervisors how they teach, how they
reinforce the teaching, how they enforce
safety and health rules and safe work practices
and what their responsibilities are during
emergency situations.
Interviews
with management should focus on its involvement
in and commitment to the safety and health
program. Ask how the policy statement was
created and how that statement is communicated
to all employees. Ask what information management
receives about the safety and health activities
and what action management takes as a result
of that information. Ask how management's
commitment to safety and health is demonstrated
to the workforce.
Review
of Site Conditions. The conditions
at the worksite reveal much about the safety
and health program's effectiveness. Worksite
conditions can be observed indirectly by
examining documents such as inspection reports
of hazards, employee reports of hazards
and accident/incident investigations.
Site
tours also may reveal hazards. Be careful,
however, that the site tour does not become
a routine inspection with emphasis only
on hazard correction. When a hazard is found,
certainly take steps to ensure its correction.
But in addition, ask what management system(s)
should have prevented or controlled the
hazard. Determine why system(s) failed and
either change them or take other appropriate
corrective measures. Chapter 8 has more
information on this technique. See especially
the hazard analysis flow charts, Appendix
9-4.
DO
PROGRAM ACTIVITIES GET RESULTS?
THE
TARGET OF EVALUATION
Time
and resources can be wasted when safety
and health program activities do not achieve
the desired results. Each year activities
should be planned with the intention of
achieving specific objectives. These objectives,
in turn, are geared toward reaching the
year's safety and health program goal.
As
an example, a company's goal is:
Develop
a comprehensive safety and health program
that effectively protects employees by
preventing or controlling existing and
potential workplace hazards.
To
reach this goal, one objective this
year is:
Develop
a comprehensive preventive maintenance
program.
The
company expects that achieving this objective
will require more than one year. For the
current year the company plans to undertake
two activities, each with governing
procedures:
Activity
1: Create preventive maintenance checklists
for all classes of company vehicles.
Procedure:
By February 1, Transportation Department
Chief will hold joint meeting of all drivers
and vehicle maintenance mechanics to determine
maintenance needs and create checklist
of preventive maintenance tasks. Checklist
will assign responsibilities to appropriate
staff, indicate required time frames and
provide for sign-off.
Activity
2: Conduct a survey of non-vehicle
machinery throughout the worksite to determine
preventive maintenance needs.
Procedure:
By February 1, each Department will submit
to Maintenance Department a list of all
machinery located within the Department,
together with notations regarding operating
problems, hazards and maintenance needs.
By March 1, Maintenance Department Chief
and staff will visit each department to
examine machinery and to discuss needs
with operators. By April 1, a comprehensive
report will be submitted to Vice-President
for Operations, inventorying machinery
and indicating maintenance needs and suggested
maintenance schedule.
The
end-of-year safety and health program evaluation
will determine whether these activities
were conducted and whether they had the
desired effect, i.e., successfully began
the process of developing a comprehensive
preventive maintenance program.
The
evaluation then will examine the value of
this objective: did the achievement of this
phase of a preventive maintenance program
move the company closer to its targeted
safety and health goal? If this analysis
finds program efforts that are ineffective
and do not contribute to the goal, the evaluation
should include recommendations for program
changes for the next year. For more information
about setting a goal and objectives, see
Chapter 2.
ACTIVITIES
AND PROCEDURES
Do
the actual safety and health program activities
and the procedures for implementing them
bring the expected results?
Larger
worksites. Large companies will
have written procedures for the major activities
of their safety and health program. They
also will have written records of those
activities as they were performed. Evaluating
whether the written procedures were followed
in the period evaluated, or how well they
were followed, is an audit function of quality
assurance or program evaluation.
Smaller
worksites. Even if yours is a smaller
business with more limited recordkeeping
you still should put some effort into thinking
about how safety and health activities were
carried out for the period evaluated and
whether the results achieved were those
expected at the outset.
Sample
questions. The precise questions
you should ask will depend upon the activity
being audited and the way the activity was
to be accomplished. For example, if plans
call for a certain person to carry out the
inspection program, did that person actually
conduct the inspections? In many workplaces
inspections are conducted by the person
with the most expertise along with members
of the site safety and health committee.
Was the expert present during every inspection?
Did the employee members always participate?
Other
questions about inspections might include
the following:
-
Is there evidence that the inspectors
went to every part of the worksite that
was specified in the inspection plans?
-
Did their reports indicate that the inspectors
were finding the kinds of hazards they
were trained to recognize?
-
Was hazard correction appropriately assigned?
-
Were the hazards corrected in an appropriate
and timely manner?
-
Was the correction tracked to completion?
Similar
questions should be asked about each activity
under the safety and health program. When
a discrepancy is found between the original
plan and actual execution of the activity,
assess which way best meets the safety and
health objectives and goal. Then make sure
that everyone follows that procedure.
OBJECTIVES
The
objectives connect the goal for the safety
and health program to the program procedures
and activities.
Objectives
that can be audited. Sometimes a
program objective will be to complete a
new or improved activity. For example, suppose
the objective states, "Complete one job
safety analysis each month, with follow-up
revision of safe work procedures and employee
training in the following month." In this
case the objective describes the frequency
of activities rather than the desired result.
An evaluation of this objective involves
no more than determining whether the activities
occurred. Therefore, an audit will be appropriate.
Look for evidence that job safety analyses
were done each month. Is there evidence
that revisions of procedures and training
also were made each month as a result of
the previous month's job safety analysis?
If the answer is yes and other program evaluation
reveals no need to do anything differently,
the frequency of these activities will become
an ongoing subject of audit.
Objectives
that must be evaluated. Ordinarily,
objectives should focus on the results desired
from the program activities. For example,
an objective might state, "Identify and
assign all areas of safety and health responsibility
that are not presently clearly assigned,
so that all safety and health responsibilities
can be successfully carried out." A set
of activities will be needed to accomplish
this objective. The activities might include
assigning a committee to list all the safety
and health responsibilities; reviewing assignment
of those responsibilities; identifying missing,
duplicate and unclear assignments; and recommending
clearer assignments.
In
this case the evaluation will focus on whether
the objective was accomplished. That is,
were all areas of safety and health responsibilities
that were not clearly defined actually identified
and clarified? Perhaps these areas were
identified but the corrections not made
because the Personnel Department needed
to first rewrite seven major job descriptions
to get the responsibilities correctly assigned.
This, then, leads to a new objective to
get the seven job descriptions rewritten
with clearer assignment of safety and health
responsibilities. If all safety and health
areas have been identified and clarified
this objective will not lead to another
objective.
As
another example, say the objective is to
increase employee safety and health awareness
by involving the employees' families in
a safety and health awareness program. The
evaluation will seek to measure the difference
between employee awareness before the family
program and after it. One way to measure
that difference is to ask them if they thought
their awareness was increased. Still another
is to ask supervisors if they perceived
a difference in employee awareness after
the family program began. If, after using
some type of meaningful measurement tool,
the findings indicate that the program has
increased awareness it will make sense to
designate the family program an ongoing
activity subject to audit. If the findings
show no measurable increase in awareness
the family program can be changed or another
activity substituted. The altered objective
then will be to increase employee awareness
by this new means.
GOAL
The
goal is the ultimate intention of the safety
and health program, its basic aim.
Objectives
should be evaluated to make sure they are
leading to the program goal. For example,
suppose the goal is the reduction of employee
exposure to hazards and one of the objectives
to achieve that goal is to hold monthly
safety meetings for all employees. Since
this is an activity objective it can be
audited to determine whether safety meetings
are actually being held. But the next question
is, "Did achieving this objective help fulfill
the goal of the safety and health program?"
In other words, did the safety meetings
help employees understand the hazards to
which they are exposed and result in plans
to reduce exposure?
There
are various ways to collect such information.
A sign-off sheet can indicate who attended
the training. The same sheet can ask employees
to describe hazards and potential hazards
that exist in their work area and ways to
better control them. If there is no sign-off
sheet interviews with some randomly selected
employees can reveal their opinion of whether
the meetings improved their understanding
of hazards and resulted in plans to control
them. Interviews with supervisors can reveal
whether employees exhibited better understanding
after training.
If
the results indicate that not much was learned
at these sessions, ask further questions
to see what went wrong. In this way, all
the objectives can and should be checked
to see if they are helping achieve the goal.
Program
evaluation can identify activities that
are not really helping to improve worksite
safety and health. In so doing, evaluation
can save you time, effort and money.
EVALUATION
JUDGMENTS
The
important work of gathering information
about safety and health program activities
is the most time-consuming part of program
evaluation. It is, however, the easiest
to understand and accomplish. The hardest
part is making judgments about program effectiveness.
To
assist smaller businesses in making these
judgments, a sample checklist is provided
in Appendix 12-1. It can be used in deciding
what information should be gathered and
what judgments need to be made. Suggested
evaluation questions in the previously cited
examples and a sample section of evaluation
instructions in Appendix 12-2 can provide
additional guidance.
Employers
should draw up appropriate, site-specific
procedures for gathering information and
making judgments. You may also want to add
environmental, product safety or security
considerations to the evaluation process.
The questions you ask should be based on
individual site program activities and site
objectives. The sample question in Appendix
12-3 may be helpful.
Insist
that the evaluator determine the program's
bottom line profitability, its real benefit.
In other words, which activities contribute
to the safety and health goal and which
do not? Judgments and decisions made by
evaluators should be driven by this quest
for profitability, by which we mean improvement
in safety and health program. Insist that
the hard questions about program effectiveness
be addressed.
HOW
TO USE THE EVALUATION
The
evaluation will prove valuable only if it
leads to improved performance in meeting
the safety and health goal. Some of the
recommendations that result from the evaluation
will be for one-time corrections. Many,
however, will involve changing emphasis
or trying new activities. These recommendations
should be incorporated into the objectives
for the next year. Consider establishing,
as a permanent objective, an audit of the
procedures that your program sets for safety
and health program activities. See Chapter
2 for more information about establishing
objectives.
Larger
Worksites. The evaluation should
result in a written report with written
recommendations and documented follow-up
to those recommendations. It may be useful
to refer to past years' evaluations when
preparing new ones or when planning new
objectives. If you find that the same recommendations
are being made year after year the process
of implementing and tracking recommendations
to completion needs improvement.
Smaller
Worksites. At smaller sites a written
evaluation report may not be practical.
However, it is important to set aside time
to think about desired changes. The evaluation
process already has involved considering
what was done during the course of the year,
talking to people, looking at the site's
working conditions and reviewing available
documents. Next, decide what you want to
do differently and make sure that everyone
understands what is expected.
SUMMARY
This
chapter has defined a safety and health
program evaluation. It has described what
should be evaluated, who should do the evaluation
and with what tools, how the evaluation
should be conducted and how to use the results.
The
following appendices offer additional examples,
guidelines and instructions:
Appendix
12-1 provides an example of a self-evaluation
checklist for small businesses. This can
be used to perform a comprehensive evaluation
of the safety and health program.
Appendix
12-2 is a step-by-step guide to using the
three evaluation tools: documentation review,
employee interviews and hazardous conditions
review and analysis. These tools are used
to assess each element and subsidiary component
of a safety and health program.
Appendix
12-3 provides a sample evaluation instruction
sheet that can be used to evaluate routine
inspections when such inspections are part
of your safety and health program.
By
using this information to perform annual
evaluations you will be able to compute
your company's safety and health bottom
line, just as you now can calculate your
organization's financial bottom line. You
will have the information needed to make
knowledgeable and effective decisions promoting
workplace safety and health.
APPENDIX
12-1
EXAMPLE
OF A SELF-EVALUATION CHECKLIST FOR SMALL
BUSINESSES
Instructions:
Fill out the last five columns after judging
the effectiveness of each part of your safety
and health program. If the part of your
program indicated in the first column is
fully effective enter "YES" in the second
column. If not fully effective enter "NO".
-
Explain the deficiencies in the "Comments/Improvements"
column and list any specific, planned
improvements.
-
Indicate the persons responsible for these
improvements in the next column.
-
Enter the target date for these improvements.
-
Finally, when the improvements are actually
completed enter the completion date in
the last column.
ELEMENT
I. MANAGEMENT LEADERSHIP AND EMPLOYEE INVOLVEMENT
ELEMENT
II. WORKSITE ANALYSIS
ELEMENT
II. WORKSITE ANALYSIS (CON'T)
ELEMENT
III. HAZARD PREVENTION AND CONTROL
ELEMENT
III. HAZARD PREVENTION AND CONTROL (CON'T)
ELEMENT
IV. SAFETY AND HEALTH TRAINING
SPECIAL
OBJECTIVES (LIST AND EVALUATE INDIVIDUALLY)
|
INDICATORS
|
YES/NO
|
COMMENTS
IMPROVEMENTS
|
RESPONSIBLE
PARTY
|
TARGET
DATE
|
DATE
COMPLETED
|
| |
|
|
|
|
|
APPENDIX
12-2
FURTHER
DESCRIPTION OF
A SAFETY
AND HEALTH PROGRAM ASSESSMENT
INTRODUCTION
There
are three basic methods for assessing safety
and health program effectiveness. This description
will explain each of them. It also will
provide more detailed information on how
to use these tools to evaluate each element
and subsidiary component of a safety and
health program. The outlined information
that begins on page 12-22 corresponds in
format to pages 12-2 to 12-4.
The
three basic methods for assessing safety
and health program effectiveness are:
