Books : Managing Worker Safety and Health : Evaluating Your Safety and Health Program

EVALUATING YOUR SAFETY AND HEALTH PROGRAM

INTRODUCTION

The scenario: Your safety and health program is in place. You have set your goal for the year and clearly stated the objectives, procedures, and activities necessary to meet that goal. Responsibilities have been defined and clearly assigned. Adequate authority and resources have been allocated. People have been trained in their safety and health program roles, and they understand the consequences of failing to perform their assignments.

Your responsibility for employee safety and health does not stop here. The next step -- a critical one -- is to evaluate how well your safety and health program is working.

This process is more than an inspection or an audit. Inspections are necessary to look at the facility, the process and the individual jobs in order to identify and then to eliminate or control any hazards that may exist. Audits focus on program activities and seek to determine whether specific objectives have been met. For example, if you are assessing employee participation by looking at the activities of the safety committee you will want to know if that committee met at the intervals specified, and if most of the members attended each meeting. These are audit questions.

But beyond this simple accounting are larger questions. For example, has employee participation at safety committee meetings helped improve the worksite's safety and health program? How is the work of the safety committee helping you meet your goal? These are the kinds of issues addressed by an evaluation.

A safety and health evaluation looks at the systems you have created to carry out your safety and health program. It asks if these systems are working effectively and efficiently. All systems that contribute to your safety and health program should be reviewed. These should include management leadership and the evaluation of that leadership, the analysis of the worksite to identify hazards, hazard prevention and control, accident and near miss investigations, employee involvement, safety and health training, use of personal protective equipment (PPE), the health program, and the emergency response program. The site may have additional programs or systems that contribute to the safety and health program. You will need to evaluate these also.

Who should conduct the evaluation? Although evaluations can be performed by worksite employees, they are best done by people who are knowledgeable about the site's processes and managing safety and health programs. The evaluator should not work at the site being evaluated. The fresh look an outsider brings produces a more accurate and helpful evaluation. This outsider may come from corporate headquarters, another worksite within the company, an insurance company or a consulting firm.

Evaluation often causes anxiety for workers. You may be able to reduce that anxiety by letting your employees know that the evaluator will be focusing on systems and not on people.

Three useful tools for this evaluation are document review, employee interviews and review of site conditions. These tools will provide the basis for an evaluation report. This report should contain a list of the programs or systems reviewed and a narrative account of the examination of each system or program. It also should contain a schedule of needed changes with target completion dates, responsible parties, and space to indicate the date when changes are actually completed. Some reports include pictures of excellent situations and those needing improvement. Some provide a grading system, so that each year's results can be compared quickly to previous years. This report should be available to any employee who wants to read it. This chapter will explain in detail how you can accomplish this evaluation.

WHAT SHOULD BE EVALUATED?

Ideally, everything that you know to be contributing directly to your safety and health program should be evaluated. OSHA's Safety and Health Program Management Guidelines can help you determine which areas of your program need evaluation. These are the four major areas of the Guidelines, that are called Major Elements:

1.The demonstration of management leadership and employee involvement through:

Setting and communicating the safety and health policy;

Setting and communicating a clear goal and objectives;

Being visibly involved in employee safety and health;

Assuring employee involvement in safety and health problem identification and resolution;

Assigning clear responsibility for safety and health;

Giving adequate authority and assuring efficient use of resources;

Holding all personnel accountable; and

Assuring quality.

2.Worksite analysis to identify existing and potential hazards through:

Comprehensive safety and health hazard surveys;

Analysis of planned changes to identify hazards that might be introduced;

Routine hazard analyses, such as:

- Job hazard analysis (also known as job safety analysis),
- Process hazard analysis (used in industries with complex and hazardous processes), and
- Phase hazard analysis (used mainly in construction);

Periodic worksite inspections, including:- Self-inspections conducted by supervisors in their work areas, and

- General inspections of the entire site conducted by safety and health staff;

Employee reports of hazards;

Work practice control; and

Analysis of injury/illness trends.

3. Hazard prevention and control through:

Engineering controls;

Work practice control;

Personal protective equipment;

Administrative controls;

Disciplinary systems to enforce controls;

Preventive maintenance;

Emergency preparedness; and

Medical program.

4. Safety and health training to ensure that all employees know how to protect themselves and others from existing and potential hazards of the worksite.

WHO SHOULD EVALUATE

Evaluators can be drawn from the workplace safety and health department or the safety committee, but the best evaluators will be people possessing fresh vision. They will not be involved in the day-to-day operations of the site. Look in the corporate safety department, another worksite of the company, insurance companies and outside consulting firms. Or have two activity managers switch places and evaluate each other's results.

Many workers' compensation insurance carriers offer their clients the services of a certified safety professional and a certified industrial hygienist. These experts are qualified to review your program activities.

Evaluators should be knowledgeable in the technical aspects of occupational safety and health, the management of safety and health, and the evaluation of programs. Of these three areas, management of safety and health is the most important.

TOOLS FOR COLLECTING INFORMATION USED IN EVALUATION

There are three indispensable evaluation tools for judging the effectiveness of occupational safety and health program management. These are:

Document review,

Interviews with employees at different levels, and

Review of site conditions.

See Appendix 12-2 for a detailed description of how to use these tools.

Documentation. Every worksite will have, at an absolute minimum, written accident reports and the OSHA log of injuries and illnesses as required by law. Major companies should have written procedures and records of all their safety and health programs. The evaluator should compare the written program to the written records of what occurred.

Interviews. In addition to the documentation, interviews can be very helpful in establishing what has occurred. We use two kinds of interviews, formal and informal. The formal interviews are conducted privately with randomly selected employees who are asked preselected questions. Informal interviews occur at employee work stations and generally follow a list of topics.

To assess how well the worksite safety and health policy is communicated and understood and how well the disciplinary system is working, ask the employees to explain them.

To gauge the effectiveness of safety and health training, interview hourly employees and first-line supervisors. Ask employees to describe what hazards they are exposed to and how they are protected. Ask them to explain what they are supposed to do in several different types of emergencies. Ask supervisors how they teach, how they reinforce the teaching, how they enforce safety and health rules and safe work practices and what their responsibilities are during emergency situations.

Interviews with management should focus on its involvement in and commitment to the safety and health program. Ask how the policy statement was created and how that statement is communicated to all employees. Ask what information management receives about the safety and health activities and what action management takes as a result of that information. Ask how management's commitment to safety and health is demonstrated to the workforce.

Review of Site Conditions. The conditions at the worksite reveal much about the safety and health program's effectiveness. Worksite conditions can be observed indirectly by examining documents such as inspection reports of hazards, employee reports of hazards and accident/incident investigations.

Site tours also may reveal hazards. Be careful, however, that the site tour does not become a routine inspection with emphasis only on hazard correction. When a hazard is found, certainly take steps to ensure its correction. But in addition, ask what management system(s) should have prevented or controlled the hazard. Determine why system(s) failed and either change them or take other appropriate corrective measures. Chapter 8 has more information on this technique. See especially the hazard analysis flow charts, Appendix 9-4.

DO PROGRAM ACTIVITIES GET RESULTS?

THE TARGET OF EVALUATION

Time and resources can be wasted when safety and health program activities do not achieve the desired results. Each year activities should be planned with the intention of achieving specific objectives. These objectives, in turn, are geared toward reaching the year's safety and health program goal.

As an example, a company's goal is:

Develop a comprehensive safety and health program that effectively protects employees by preventing or controlling existing and potential workplace hazards.

To reach this goal, one objective this year is:

Develop a comprehensive preventive maintenance program.

The company expects that achieving this objective will require more than one year. For the current year the company plans to undertake two activities, each with governing procedures:

Activity 1: Create preventive maintenance checklists for all classes of company vehicles.

Procedure: By February 1, Transportation Department Chief will hold joint meeting of all drivers and vehicle maintenance mechanics to determine maintenance needs and create checklist of preventive maintenance tasks. Checklist will assign responsibilities to appropriate staff, indicate required time frames and provide for sign-off.

Activity 2: Conduct a survey of non-vehicle machinery throughout the worksite to determine preventive maintenance needs.

Procedure: By February 1, each Department will submit to Maintenance Department a list of all machinery located within the Department, together with notations regarding operating problems, hazards and maintenance needs. By March 1, Maintenance Department Chief and staff will visit each department to examine machinery and to discuss needs with operators. By April 1, a comprehensive report will be submitted to Vice-President for Operations, inventorying machinery and indicating maintenance needs and suggested maintenance schedule.

The end-of-year safety and health program evaluation will determine whether these activities were conducted and whether they had the desired effect, i.e., successfully began the process of developing a comprehensive preventive maintenance program.

The evaluation then will examine the value of this objective: did the achievement of this phase of a preventive maintenance program move the company closer to its targeted safety and health goal? If this analysis finds program efforts that are ineffective and do not contribute to the goal, the evaluation should include recommendations for program changes for the next year. For more information about setting a goal and objectives, see Chapter 2.

ACTIVITIES AND PROCEDURES

Do the actual safety and health program activities and the procedures for implementing them bring the expected results?

Larger worksites. Large companies will have written procedures for the major activities of their safety and health program. They also will have written records of those activities as they were performed. Evaluating whether the written procedures were followed in the period evaluated, or how well they were followed, is an audit function of quality assurance or program evaluation.

Smaller worksites. Even if yours is a smaller business with more limited recordkeeping you still should put some effort into thinking about how safety and health activities were carried out for the period evaluated and whether the results achieved were those expected at the outset.

Sample questions. The precise questions you should ask will depend upon the activity being audited and the way the activity was to be accomplished. For example, if plans call for a certain person to carry out the inspection program, did that person actually conduct the inspections? In many workplaces inspections are conducted by the person with the most expertise along with members of the site safety and health committee. Was the expert present during every inspection? Did the employee members always participate?

Other questions about inspections might include the following:

Is there evidence that the inspectors went to every part of the worksite that was specified in the inspection plans?

Did their reports indicate that the inspectors were finding the kinds of hazards they were trained to recognize?

Was hazard correction appropriately assigned?

Were the hazards corrected in an appropriate and timely manner?

Was the correction tracked to completion?

Similar questions should be asked about each activity under the safety and health program. When a discrepancy is found between the original plan and actual execution of the activity, assess which way best meets the safety and health objectives and goal. Then make sure that everyone follows that procedure.

OBJECTIVES

The objectives connect the goal for the safety and health program to the program procedures and activities.

Objectives that can be audited. Sometimes a program objective will be to complete a new or improved activity. For example, suppose the objective states, "Complete one job safety analysis each month, with follow-up revision of safe work procedures and employee training in the following month." In this case the objective describes the frequency of activities rather than the desired result. An evaluation of this objective involves no more than determining whether the activities occurred. Therefore, an audit will be appropriate. Look for evidence that job safety analyses were done each month. Is there evidence that revisions of procedures and training also were made each month as a result of the previous month's job safety analysis? If the answer is yes and other program evaluation reveals no need to do anything differently, the frequency of these activities will become an ongoing subject of audit.

Objectives that must be evaluated. Ordinarily, objectives should focus on the results desired from the program activities. For example, an objective might state, "Identify and assign all areas of safety and health responsibility that are not presently clearly assigned, so that all safety and health responsibilities can be successfully carried out." A set of activities will be needed to accomplish this objective. The activities might include assigning a committee to list all the safety and health responsibilities; reviewing assignment of those responsibilities; identifying missing, duplicate and unclear assignments; and recommending clearer assignments.

In this case the evaluation will focus on whether the objective was accomplished. That is, were all areas of safety and health responsibilities that were not clearly defined actually identified and clarified? Perhaps these areas were identified but the corrections not made because the Personnel Department needed to first rewrite seven major job descriptions to get the responsibilities correctly assigned. This, then, leads to a new objective to get the seven job descriptions rewritten with clearer assignment of safety and health responsibilities. If all safety and health areas have been identified and clarified this objective will not lead to another objective.

As another example, say the objective is to increase employee safety and health awareness by involving the employees' families in a safety and health awareness program. The evaluation will seek to measure the difference between employee awareness before the family program and after it. One way to measure that difference is to ask them if they thought their awareness was increased. Still another is to ask supervisors if they perceived a difference in employee awareness after the family program began. If, after using some type of meaningful measurement tool, the findings indicate that the program has increased awareness it will make sense to designate the family program an ongoing activity subject to audit. If the findings show no measurable increase in awareness the family program can be changed or another activity substituted. The altered objective then will be to increase employee awareness by this new means.

GOAL

The goal is the ultimate intention of the safety and health program, its basic aim.

Objectives should be evaluated to make sure they are leading to the program goal. For example, suppose the goal is the reduction of employee exposure to hazards and one of the objectives to achieve that goal is to hold monthly safety meetings for all employees. Since this is an activity objective it can be audited to determine whether safety meetings are actually being held. But the next question is, "Did achieving this objective help fulfill the goal of the safety and health program?" In other words, did the safety meetings help employees understand the hazards to which they are exposed and result in plans to reduce exposure?

There are various ways to collect such information. A sign-off sheet can indicate who attended the training. The same sheet can ask employees to describe hazards and potential hazards that exist in their work area and ways to better control them. If there is no sign-off sheet interviews with some randomly selected employees can reveal their opinion of whether the meetings improved their understanding of hazards and resulted in plans to control them. Interviews with supervisors can reveal whether employees exhibited better understanding after training.

If the results indicate that not much was learned at these sessions, ask further questions to see what went wrong. In this way, all the objectives can and should be checked to see if they are helping achieve the goal.

Program evaluation can identify activities that are not really helping to improve worksite safety and health. In so doing, evaluation can save you time, effort and money.

EVALUATION JUDGMENTS

The important work of gathering information about safety and health program activities is the most time-consuming part of program evaluation. It is, however, the easiest to understand and accomplish. The hardest part is making judgments about program effectiveness.

To assist smaller businesses in making these judgments, a sample checklist is provided in Appendix 12-1. It can be used in deciding what information should be gathered and what judgments need to be made. Suggested evaluation questions in the previously cited examples and a sample section of evaluation instructions in Appendix 12-2 can provide additional guidance.

Employers should draw up appropriate, site-specific procedures for gathering information and making judgments. You may also want to add environmental, product safety or security considerations to the evaluation process. The questions you ask should be based on individual site program activities and site objectives. The sample question in Appendix 12-3 may be helpful.

Insist that the evaluator determine the program's bottom line profitability, its real benefit. In other words, which activities contribute to the safety and health goal and which do not? Judgments and decisions made by evaluators should be driven by this quest for profitability, by which we mean improvement in safety and health program. Insist that the hard questions about program effectiveness be addressed.

HOW TO USE THE EVALUATION

The evaluation will prove valuable only if it leads to improved performance in meeting the safety and health goal. Some of the recommendations that result from the evaluation will be for one-time corrections. Many, however, will involve changing emphasis or trying new activities. These recommendations should be incorporated into the objectives for the next year. Consider establishing, as a permanent objective, an audit of the procedures that your program sets for safety and health program activities. See Chapter 2 for more information about establishing objectives.

Larger Worksites. The evaluation should result in a written report with written recommendations and documented follow-up to those recommendations. It may be useful to refer to past years' evaluations when preparing new ones or when planning new objectives. If you find that the same recommendations are being made year after year the process of implementing and tracking recommendations to completion needs improvement.

Smaller Worksites. At smaller sites a written evaluation report may not be practical. However, it is important to set aside time to think about desired changes. The evaluation process already has involved considering what was done during the course of the year, talking to people, looking at the site's working conditions and reviewing available documents. Next, decide what you want to do differently and make sure that everyone understands what is expected.

SUMMARY

This chapter has defined a safety and health program evaluation. It has described what should be evaluated, who should do the evaluation and with what tools, how the evaluation should be conducted and how to use the results.

The following appendices offer additional examples, guidelines and instructions:

Appendix 12-1 provides an example of a self-evaluation checklist for small businesses. This can be used to perform a comprehensive evaluation of the safety and health program.

Appendix 12-2 is a step-by-step guide to using the three evaluation tools: documentation review, employee interviews and hazardous conditions review and analysis. These tools are used to assess each element and subsidiary component of a safety and health program.

Appendix 12-3 provides a sample evaluation instruction sheet that can be used to evaluate routine inspections when such inspections are part of your safety and health program.

By using this information to perform annual evaluations you will be able to compute your company's safety and health bottom line, just as you now can calculate your organization's financial bottom line. You will have the information needed to make knowledgeable and effective decisions promoting workplace safety and health.

APPENDIX 12-1

EXAMPLE OF A SELF-EVALUATION CHECKLIST FOR SMALL BUSINESSES

Instructions: Fill out the last five columns after judging the effectiveness of each part of your safety and health program. If the part of your program indicated in the first column is fully effective enter "YES" in the second column. If not fully effective enter "NO".

Explain the deficiencies in the "Comments/Improvements" column and list any specific, planned improvements.

Indicate the persons responsible for these improvements in the next column.

Enter the target date for these improvements.

Finally, when the improvements are actually completed enter the completion date in the last column.

ELEMENT I. MANAGEMENT LEADERSHIP AND EMPLOYEE INVOLVEMENT

INDICATORS YES/
NO
COMMENTS
IMPROVEMENTS
RESPONSIBLE PARTY TARGET DATE DATE COMPLETED

Policies and objectives are established and communicated to all employees Yes No

Top management is visibly involved in safety and health Yes No

Employees are involved in identifying and solving safety and health problems Yes No

All safety and health responsibilities are clearly assigned Yes No

Adequate authority and resources are provided to those with responsibility Yes No

Managers, supervisors and employees are held accountable Yes No

Program operations are reviewed at least annually to evaluate success in meeting goals and objectives and to prepare new objectives Yes No

ELEMENT II. WORKSITE ANALYSIS

There are three basic methods for assessing safety and health program effectiveness. This description will explain each of them. It also will provide more detailed information on how to use these tools to evaluate each element and subsidiary component of a safety and health program. The outlined information that begins on page 12-22 corresponds in format to pages 12-2 to 12-4.

The three basic methods for assessing safety and health program effectiveness are:

1. Checking documentation of activity;

2. Interviewing employees at all levels for knowledge, awareness and perceptions; and

3. Reviewing site conditions and, where hazards are found, finding the weaknesses in management systems that allowed the hazards to occur or to be "uncontrolled."

Some elements of the safety and health program are best assessed by using one of these methods. Others lend themselves to assessment by two or all three methods.

Documentation. Checking documentation is a standard audit technique. It is particularly useful for understanding whether the tracking of hazards to correct is effective. It can also be used to determine the quality of certain activities, such as self-inspections or routine hazard analysis.

Inspection records can tell the evaluator whether serious hazards are being found or whether the same hazards are being found repeatedly. If serious hazards are not being found and accidents keep occurring there may be a need to train inspectors to look for different hazards. If the same hazards are being found repeatedly the problem may be more complicated. Perhaps the hazards are not being corrected. If so, this would suggest a tracking problem or a problem in accountability for hazard correction.

If certain hazards recur repeatedly after being corrected, someone is not taking responsibility for keeping those hazards under control. Either the responsibility is not clear or those who are responsible are not being held accountable.

Employee Interviews. Talking to randomly selected employees at all levels will provide a good indication of the quality of employee training and of employee perceptions of the program. If safety and health training is effective employees will be able to tell you about the hazards they work with and how they protect themselves and others by keeping those hazards controlled. Every employee should also be able to say precisely what he or she is expected to do as part of the program. And all employees should know where to go in an emergency.

Employee perceptions can provide other useful information. An employee's opinion of how easy it is to report a hazard and get a response will tell you a lot about how well your hazard reporting system is working. If employees indicate that your system for enforcing safety and health rules and safe work practices is inconsistent or confusing you will know that the system needs improvement.

Interviews should not be limited to hourly employees. Much can be learned from talking with first-line supervisors. It is also helpful to query line managers about their understanding of their safety and health responsibilities.

Site Conditions and Root Causes of Hazards. Examining the conditions of the workplace can reveal existing hazards. But is can also provide information about the breakdown of those management systems meant to prevent or control these hazards.

Looking at conditions and practices is a well established technique for assessing the effectiveness of safety and health programs. For example, let's say that in areas where PPE is required you see large and understandable signs communicating this requirement and all employees -- with no exceptions -- wearing equipment properly. You have obtained valuable visual evidence that the PPE program is working.

Another way to obtain information about safety and health program management is through root analysis of observed hazards. This approach to hazards is much like the most sophisticated accident investigation techniques in which many contributing factors are located and corrected or controlled.

For example, let's say that during a review of conditions, you find a machine being operated without a guard on a pinch point. You should not limit your response to ensuring that a guard is installed. Asking a few questions can reveal a lot about the safety program's management systems. Why was the guard missing? If the operator says he did not know a guard was supposed to be on the machine, follow up with questions about the existence of safe work procedures and/or training.

If he says that the guard slows him down and that the supervisor knows he takes it off, ask questions about the enforcement of rules, accountability and the clarity of the worksite objective of putting safety and health first.

Let's say, however, that your insurance inspector or an OSHA inspector is the first person to notice the need for the guard. Or you first notice it when someone is hurt. A different lead-off question is appropriate. Has a comprehensive survey of the worksite been done by someone with enough expertise to recognize all potential and existing hazards?

Analyzing the root causes of hazards, while very helpful during a formal assessment, is a technique that also lends itself to everyday use. Attempt to analyze causes whenever hazards are spotted.

When evaluating each part of your worksite's safety and health program, use one or more of the above methods, as appropriate.

The remainder of this appendix will identify the components found in each element of a quality safety and health program and will describe useful ways to asses these components.

1. Assessing the Key Components of Management Leadership and Employee Involvement.

Worksite Policy on Safe and Healthful Working Conditions
-Documentation. If there is a written policy, does it clearly declare the priority of worker safety and health over other organizational values, such as production.

-Interviews. When asked, can employees at all levels express the worksite policy on worker safety and health?

If the policy is written, can hourly employees tell you where they have seen it? Can employees at all levels explain the priority of worker safety and health over other organizational values, as the policy intends?

-Site Conditions and Root Causes of Hazards. Have injuries occurred because employees at any level did not understand the importance of safety precautions in relation to other organizational values, such as production?

Goal and Objective for Worker Safety and Health
-Documentation. If there is a written goal for your safety and health program, is it updated annually?
If there are written objectives, such as an annual plan to reach that goal, are they clearly stated?
If managers and supervisors have written objectives, do these documents include objectives for the safety and health program?

-Interviews. Do managers and supervisors have a clear idea of their objectives for worker safety and health?
Do hourly employees understand the current objectives of the safety and health program?

-Site Conditions and Root Causes of Hazards. (Only helpful in a general sense.)

Visible Top Management Leadership
-Documentation. Are there one or more written programs which involve top-level management in safety and health activities? For example, top management can receive and sign off on inspection reports either after each inspection or in a quarterly summary. These reports can then be posted for employees to see. Top management can provide "open door" times each week or each month for employees to come in to discuss safety and health concerns. Top management can reward the best safety suggestions each month or at other specified intervals.

-Interviews. Can hourly employees describe how management officials are involved in safety and health activities?

Do hourly employees perceive that managers and supervisors follow safety and health rules and work practices, such as wearing appropriate personal protective equipment?

-Site Conditions and Root Causes of Hazards. When employees are found not wearing required personal protective equipment or not following safe work practices, have any of them said that managers or supervisors also did not follow these rules.

Employee Involvement

-Documentation. Are there one or more written programs that provide for employee involvement in decisions affecting their safety and health?

Is there documentation of these activities; for example, employee inspection reports, minutes of joint employee-management or employee committee meetings?

Is there written documentation of any management response to employee safety and health program activities?

Does the documentation indicate a genuine substance to employee activities?

-Interviews. Are employees aware of ways they can be involved in decisions affecting their safety and health.

Do employees appear to take pride in the achievements of the worksite safety and health program?

Are employees comfortable answering questions about safety and health programs and conditions at the site?

Do employees feel they have the support of management for their safety and health activities?

-Site Conditions and Root Causes of Hazards. (Not applicable.)

Assignment of Responsibility

-Documentation. Are responsibilities written out so that they can be clearly understood?

-Interviews. Do employees understand their own responsibilities and those of others?

-Site Conditions and Root Causes of Hazards. Was the hazard caused in part because no one was assigned the responsibility to control or prevent it?
Was the hazard allowed to exist in part because someone in management did not have the clear responsibility to hold a lower-level manager or supervisor accountable for carrying out assigned responsibilities?

Adequate Authority and Resources
-Documentation. (Only generally applicable.)

-Interviews. Do safety staff members or any other personnel with responsibilities for ensuring safe operation of production equipment have the authority to shut down that equipment or to order maintenance or parts?
Do employees talk about not being able to get safety and health improvements because of cost?
Do employees mention the need for more safety or health personnel or expert consultants?
Do hazards go unrecognized because greater expertise is needed to diagnose them?

-Site Conditions and Root Causes of Hazards Do recognized hazards go uncorrected because of lack of authority or resources?
Do hazards go unrecognized because greater expertise is needed to diagnose them?

Accountability of Managers, Supervisors and Hourly Employees

-Documentation. Do performance evaluations for all line managers and supervisors include specific criteria relating to safety and health protection?

Is there documented evidence of employees at all levels being held accountable for safety and health responsibilities, including safe work practices? Is accountability accomplished through either performance evaluations affecting pay and/or promotions or disciplinary actions?

-Interviews. When you ask employees what happens to people who violate safety and health rules or safe work practices, do they indicate that rule breakers are clearly and consistently held accountable?

Do hourly employees indicate that supervisors and managers genuinely care about meeting safety and health responsibilities?

When asked what happens when rules are broken, do hourly employees complain that supervisors and managers do not follow rules and never are disciplined for infractions?

-Site Conditions and Root Causes of Hazards. Are hazards occurring because employees, supervisors and/or managers are not being held accountable for their safety and health responsibilities?

Are identified hazards not being corrected because those persons assigned the responsibility are not being held accountable?

Evaluation of Program Operations

Documentation. Is there a written evaluation of each major part of the program, as identified in the OSHA Safety and Health Program Management Guidelines (54 CFR 3908, January 26, 1989)? Does this written evaluation list what is being done, assess the effectiveness of each program element against the goal and objectives and recommend changes as needed to make the program more effective or to try alternatives?

Interviews. Can employees, supervisors and/or managers tell you how the program is evaluated and revised each year?

Site Conditions and Root Causes of Hazards. (Only generally applicable).

2. Assessing the Key Components of Worksite Analysis

Comprehensive Surveys, Change Analysis, Routine Hazard Analysis

-Documentation. Are there documents that provide comprehensive analysis of all potential safety and health hazards of the worksite?

Are there documents that provide both the analysis of potential safety and health hazards for each new facility, equipment, material or process and the means for such hazards' elimination or control?

Does documentation exist of the step-by-step analysis of the hazards in each part of each job, so that you can clearly discern the evolution of decisions on safe work procedures?

If complicated processes exist, with a potential for catastrophic impact from an accident but low probability of such accident (as in nuclear power or chemical production), are there documents analyzing the potential hazards in each part of the processes and the means to prevent or control them?

If there are processes with a potential for catastrophic impact from an accident but low probability of an accident, have analyses such as "fault tree" or "what if" been documented to ensure enough back-up systems for worker protection in the event of multiple control failure?

Interviews. Do employees complain that new facilities, equipment, materials or processes are hazardous?

Do any employees say they have been involved in job safety analysis or process review and are satisfied with the results?

Does the safety and health staff indicate ignorance of existing or potential hazards at the worksite?

Does the occupational nurse/doctor or other health care provider understand the potential occupational diseases and health effects in this worksite?

Site Conditions and Root Causes of Hazards. Have hazards appeared where no one in management realized there was potential for their development?

Where workers have faithfully followed job procedures, have accidents or near misses occurred because of hidden hazards?

Have hazards been discovered in the design of new facilities, equipment, materials and processes after use has begun?

Have accidents or near misses occurred when two or more failures in the hazard control system occurred at the same time, surprising everyone?

Regular Site Safety and Health Inspections

-Documentation. If inspection reports are written, do they show that inspections are done on a regular basis?

Do the hazards found indicate good ability to recognize those hazards typical of this industry?

Are hazards found during inspections tracked to complete correction?

What is the relationship between hazards uncovered during inspections and those implicated in injuries or illness?

-Interviews. Do employees indicate that they see inspections being conducted and that these inspections appear thorough?

-Site Conditions and Root Causes of Hazards. Are the hazards discovered during accident investigations ones that should have been recognized and corrected by the regular inspection process?

Employee Reports of Hazards

-Documentation. Is the system for written reports being used frequently? Are valid hazards that have been reported by employees tracked to complete correction?

Are the responses timely and adequate?

-

Interviews. Do employees know whom to contact and what to do if they see something they believe to be hazards to themselves or co-workers?

Do employees think that responses to their reports of hazards are timely and adequate?

Do employees say that sometimes when they report a hazard they hear nothing further about it?

Do any employees say that they or other workers are being harassed, officially or otherwise, for reporting hazards?

-Site Conditions and Root Causes of Hazards. When hazards are found, do employees ever say they have complained repeatedly but to no avail?

Are hazards ever found where employees could reasonable be expected to have previously recognized and reported them?

Accident and Near Miss Investigations

-Documentation. Do accident investigation reports show a thorough analysis of causes, rather than a tendency automatically to blame the injured employees?

Are near misses, (property damage or close calls) investigated using the same techniques as accident investigations?

Are hazards that are identified as contributing to accidents or near misses tracked to correction?

-Interviews. Do employees understand and accept the results of accident and near misses investigations?

Do employees mention a tendency on management's part to blame the injured employee?

Do employees believe that all hazards contributing to accidents are corrected or controlled?

-Site Conditions and Root Causes of Hazards. Are accidents sometimes caused at least partly by factors that might also have contributed to previous near misses that were not investigated or accidents that were too superficially investigated?

Injury and Illness Pattern Analysis

-Documentation. In addition to the required OSHA log, are careful records kept of first aid injuries and/or illnesses that might not immediately appear to be work-related?

Is there any periodic, written analysis of the patterns of near misses, injuries and/or illnesses over time, seeking previously unrecognized connections between them that indicate unrecognized hazards needing correction or control?

Looking at the OSHA 200 log, and where applicable, first aid logs, are there patterns of illness or injury that should have been analyzed for previously undetected hazards?

If there is an occupational nurse/doctor on the worksite, or if employees suffering from ordinary illness are encouraged to see a nearby health care provider, are the lists of those visits analyzed for clusters of illness that might be work-related?

-Interviews. Do employees mention illnesses or injuries that seem work-related to them but that have not been analyzed for previously undetected hazards?

-Site Conditions and Root Causes of Hazards. (Not generally applicable.)

3. Assessing the Key Components of Hazard Prevention and Control.

Appropriate Use of Engineering Controls, Work Practices, Personal Protective Equipment and Administrative Controls

- Documentation. If there are documented comprehensive surveys are they accompanied by a plan for systematic prevention or control of hazards found?

If there is a written plan does it show that the best method of hazard protection was chosen?
Are there written safe work procedures?
If respirators are used, is there a written respirator program?

-Interviews. Do employees say they have been trained in and have ready access to reliable, safe work procedures?

Do employees say they have difficulty accomplishing their work because of unwieldy controls meant to protect them?

Do employees ever mention personal protective equipment, work procedures or engineering controls as interfering with their ability to work safely?

Do employees who use PPE understand why they use it and how to maintain it?

Do employees who use PPE indicate that the rules for PPE use are consistently and fairly enforced?

Do employees indicate that safe work procedures are fairly and consistently enforced?

-Site Conditions and Root Causes of Hazards. Do you ever find that controls meant to protect workers are actually putting them at risk or not providing enough protection?

Are employees engaging in unsafe practices or creating unsafe conditions because rules and work practices are not fairly and consistently enforced?

Are employees in areas designated for PPE wearing it properly, with no exceptions?

Are hazards that could feasibly be controlled through improved design being inadequately controlled by other means?

Facility and Equipment Preventive Maintenance

-Documentation. Is there a preventive maintenance schedule that provides for timely maintenance of the facilities and equipment?
Is there a written or computerized record of performed maintenance that shows the schedule has been followed?
Do maintenance request records show a pattern of certain facilities or equipment needing repair or breaking down before a maintenance was scheduled or actually performed?
Do any accident/incident investigations list facility or equipment breakdown as a major cause?

-Interviews. Do employees mention difficulty with improperly functioning equipment or facilities in poor repair?

Do maintenance employees believe that the preventive maintenance system is working well?
Do employees believe that hazard controls needing maintenance are properly cared for?

-Site Conditions and Root Causes of Hazards. Is poor maintenance a frequent source of hazards?

Are hazard controls in good working order?

Does equipment appear to be in good working order?

Emergency Planning and Preparation

-Documentation. Are there clearly written procedures for every likely emergency, with clear evacuation routes? Can employees tell you exactly what they are supposed to do?

-Interviews. When asked about any kind of likely emergency, can employees tell you exactly what they are supposed to do?

-Site Conditions and Root Causes of Hazards. Have hazards occurred during actual or simulated emergencies due to confusion about what to do?

In larger worksites, are emergency evacuation routes clearly marked?
Are emergency telephone numbers and fire alarms easy to find?

Establishing a Medical Program

-Documentation. Are good clear records kept of medical testing and assistance?

- Interviews. Do employees say that test results were explained to them?
Do employees feel that more first aid or CPR-trained personnel should be available?

Are employees satisfied with the medical arrangements provided at the site or elsewhere?

Does the occupational health care provider understand the potential hazards of the worksite, so that occupational illness symptoms can be recognized?

-Site Conditions and Root Causes of Hazards. Have further injuries or worsening of injuries occurred because proper medical assistance (including trained first aid and CPR providers) was not readily available?
Have occupational illnesses possibly gone undetected because no one with occupational health specialty training reviewed employee symptoms as part of the medical program?

4. Assessing the Key Components of Safety and Health Training.

Ensuring that all Employees Understand Hazards

-Documentation. Does the written training program include complete training for every employee in emergency procedures and in all potential hazards to which employees may be exposed?

Do training records show that every employee received the planned training?

Do the written evaluations of training indicate that the training was successful and that the employees learned what was intended?

-Interviews. Can employees tell you what hazards they are exposed to, why those hazards are a threat and how they can help protect themselves and others?

If PPE is used, can employees explain why they use it and how to use and maintain it properly?

Can employees tell you precisely what they are supposed to do and where they are supposed to go in every kind of emergency likely to occur at your worksite?

Do employees feel that health and safety training is adequate?

-Site Conditions and Root Causes of Hazards. Have employees been hurt or made ill by hazards of which they were completely unaware, or whose dangers they did not understand or from which they did not know how to protect themselves?
Have employees or rescue workers ever been endangered by employees not knowing what to do or where to go in a given emergency situation?

Are there hazards in the workplace that exist, at least in part, because one or more employees have not received adequate hazard control training?

Are there any instances of employees not wearing required PPE properly because they have not received proper training?

Ensuring that Supervisors Understand Their Responsibilities

-Documentation. Do training records indicate that all supervisors have been trained in their responsibilities to analyze work under their supervision for unrecognized hazards; to maintain physical protections; to reinforce employee training through performance feedback; and, where necessary, to enforce safe work procedures and safety and health rules?

-Interviews. Are supervisors aware of their responsibilities?

Do employees say that supervisors are carrying out these duties?

-Site Conditions and Root Causes. Has a supervisor's lack of understand of safety and health responsibilities played a part in creating hazardous activities or conditions?

Ensuring that Managers Understand Their Safety and Health Responsibilities

-Documentation. Do training plans for managers include training in safety and health responsibilities?
Do records indicate that all line managers have received this training?

-Interviews. Do employees indicate that managers know and carry out their safety and health responsibilities?

-Site Conditions and Root Causes of Hazards. Has an incomplete or inaccurate understanding by management of its safety and health responsibilities played a part in the creation of hazardous activities or conditions?

APPENDIX 12-3

SAMPLE FROM AN EVALUATION INSTRUCTION

The questions asked in this sample instruction relate to the section of the evaluation dealing with routine inspections. They are similar, however, to those that should be asked for every element in your safety and health program.

Write an evaluation report by responding to the following questions and instructions. Use complete sentences. Where appropriate, one sentence can cover more than one question. Your answers should reflect completed judgments. Avoid using terms that suggest incomplete judgments, such as "appears," "apparently" and "seems."

Section 2. Routine Inspections

1.List any established objectives that involve routine inspections.

2.If any specific objectives were set for inspections were they effectively met? Describe.

3.Are inspections following set procedures? If not, describe how they differ and how frequently this difference occurs.

4.If procedures are not being followed, discuss the reasons and whether the alternative activity meets the objective seen for inspections.

5.Has inspection activity contributed to identification and control of hazards and potential hazards? Describe.

6.If not otherwise covered above, please answer the following and provide examples.

a.Are inspections revealing hazards escaping their controls? Examples: guards removed from machines, housekeeping problems, employees failing to follow established safety procedures. If so, be sure to deal with this topic in the evaluation of hazard prevention and control.

b.Are inspections revealing new hazards that either did not exist or were not identified previously? If so, be sure to deal with this topic under the evaluation comprehensive surveys, change analysis or routine hazard analysis.

c.Are inspections revealing repeated instances of the same problem? If so, is the problem going uncorrected or is it recurring after being corrected? If the former, be sure to deal with the problem under the evaluation of tracking of hazard correction, accountability or both. If the latter, be sure to address the problem under hazard prevention and control.

7.List any recommendations for changes in routine inspection activity needed for next year. Make sure that the recommended logically reflect the conclusions made above.

Previous Section: Safety and Health Training