Event evaluation is an activity that seeks to understand and measure the extent to which an event has succeeded in achieving its purpose. The purpose of an event will differ with respect to the category and variation of event. However, to provide reach and interaction would be a generic purpose that events satisfy.
There can be two attitudes with which event evaluation can be put in its proper perspective. The concept of event evaluation stated above was a critical examination digging out what went wrong. A more constructive focus for evaluation is to make recommendations about how an event might be improved to achieve its aims more effectively.
Event Evaluation is concerned with assessment, which usually involves measuring a set of key variables, as well as monitoring those variables to determine positive and negative outcomes. It is a subjective determination that can utilize objective quantitative measures (Getz, 1997)
To carry out an event evaluation and measurement exercise it is essential that the predefined objectives of the events have been properly understood. The brief should contain all the data to be communicated since if an event has been organized without a clearly defined purpose, any evaluation would be rather pointless.
The Basic Event Evaluation Process
The basic event evaluation process involves three steps:
- Establishing tangible objectives and incorporating sensitivity in evaluation
- Measuring the performance before, during and after the event
- Correcting deviations from plans
These steps are discussed below:
1. Establishing Tangible Objectives and Sensitivity in Evaluation
Setting objectives for an event is easier said than done. It is more difficult to set standards and declaring an event successful after it meets them. To provide tangibility to the problem, the best approach is to begin with definition of the target audience for whom the event has been organized. In the case of commercial events, the audience could be end users who use the company’s products. An event might be conceptualized to achieve different things for different audience. Once the audience has been defined, the next step is to identify and put on paper what each of the audience is expected to think, feel and do having been to the event, that it did not think, feel or do beforehand. This adds an element of tangibility to the evaluation and measurement proceedings.
The number of mega-events has increased dramatically in the past few years and the costs of organizing events have also increased exponentially. The costs of production in major events can be enormous and therefore, in the near future one can expect companies to start asking questions about the effectiveness of their events to see whether their money is being spent prudently.
Creativity is derived from the Greek word enthousiasm which literally translates into ‘God, within’. Setting out to evaluate such an effort that is considered to be the work of Gods themselves demands a certain amount of sensitivity during evaluation. Objective evaluation should also take into consideration the nature of the concept and the process of execution of the event in their entirety. However professional the evaluation, there is scope for error and misjudgment if sensitivity is not adhered to. This is because it takes a creative and sensitive mind to spot wrong questions or situations where asking questions might be the wrong method and observation might be more appropriate. One of the ways of nurturing and encouraging this sensitivity is to place evaluation within the context of a team approach all the way from conceptualization to carrying out of the event.
From experience, it is known that people involved in an event are more open. minded and less committed to any particular course of action before the event occurs. Yet another learning is that, if things are shown to be wrong after a decision has been taken, the majority of people involved in the decision-making process may try to wash their hands of the fault. Thus, adding sensitivity to the evaluation process is very important.
2. Measuring Performance
Although perfect measurement is not always practicable, the measurement of performance against the objectives should ideally be done on a forward looking basis so that deviations may be detected in advance of their occurrence and avoided by appropriate actions. The concept research is used to anticipate the viability of a concept during the conceptualization process. Formative and objective evaluations are carried out during the customization phase of an event. Summative evaluation can be carried out to measure performance during the event.
- Concept Research: At the conceptualization stage, if a concept team does not have a sound basis upon which to make a decision ‘between various options, a commissioning of audience research to help in defining the strategic approach to be adopted in the event is appropriate. It essentially involves presenting the various options to a representative sample of the target audience in a story form and inviting their reactions. This provides enough material for understanding the pros and cons of the various available alternatives. The downside to this method is that it is speculative in nature since it deals with plans that nobody has as yet tried to implement. This method is called concept research.
- Formative Evaluation: Evaluation at this stage focuses on things that are actually happening. After the conceptualization team makes an attempt to customize and implement an agreed strategy, steps can be taken to evaluate the success with which customization is proceeding. These evaluations are aimed at shaping the form of the final event. Mock-up displays and presentations of the event are used to carry out research to check whether they are achieving the desired reactions from the audience. These evaluations are conducted among small sample representative of the target audience in an open-ended and qualitative fashion since the main emphasis is on discovering how the concept might be better represented. The outcome of these formative evaluations lead to a discussion among the team in which proposals for rectifying any weak points in the communications can be put forward. A point, which should be safeguarded against whilst using this technique, is to interpret consumer reactions with considerable sensitivity to stimulate the creative process further and also to ensure that good ideas are not killed simply because they were not properly presented in mock-up form.
- Objective Evaluation: This is the stage when approval from the client is sought before starting the execution related activities of an event. The evaluation team has to provide the objective evidence that has been collected which justifies the proposed concept solutions. The team also provides reassurance on how and why the particular event will work among its intended audience. Since taking the client into confidence requires certain amount of objectivity and professionalism, this technique is called objective evaluation.
- Summative Evaluation: After the event has started, the evaluation team should be concerned with measuring the impact of the event upon its audience. Among other things, they should establish the extent to which the objectives or aims of the event have been met and whether the event can be improved in any way and if so, how This will not apply for short term events though. A major purpose of evaluating an event after it has opened to the public is that it provides the team with the opportunity of learning from their mistakes. The team should assimilate the information thus collected so that they can avoid making similar mistakes in the future.
3. Correcting Deviations
The fundamental reason why event evaluation is carried out is to navigate the event so as to ensure that the event objectives are achieved in total. And since deviations may occur during any stage in the event designing phase, it is important that measurement is carried out at all possible stages.
Critical Evaluation Points of Events
Events can be evaluated based on the critical success factors listed below; from both the clients’ and event organizer’s viewpoints.
Critical Evaluation Points from Event Organizer’s Point of View
There are multiple criteria for evaluating the success of an event from the event organizer’s point of view. These are over and above ensuring perfect reach and interaction for the client by networking on-time & at lowest cost. The client event-target audience fit should match the clients’ brand/product/company image and personality perfectly, keeping the target audience as the focal point. This is a very critical evaluation point. Ensuring the profitability of an event such that there is maximum profitability with minimum mark ups is another critical evaluation point. Since resources are also a major constraint for event organizers, the resource management efficiency i.e., resources committed and span of time for which it stays committed – financial, human, equipment and infrastructure should be a minimum. The number of staff and volunteers involved should be appropriate to offer quality service.
Logistics and efficiency of event execution for ensuring smooth proceedings without unnecessary delays and damages is another critical success factor. Creating avenues for lead generation & its proper management during the event is a critical factor. Each and every completed event should generate more inquiries and these should be responded to immediately. Opportunities for explanation of available synergies and expansion of services offered to client to keep strategic integration and diversification options open is also an important factor. Since an event is essentially a one-off affair and any last moment problem can convert an exceptionally well-planned event into a disaster, all care needs to be taken during the event execution. Yet, another important critical success factor is the degree of localization or customization accommodated in the concept to suit the demographic and other variables of various places where the event is to be carried out.
Critical Evaluation Points from Clients’ Point of View
We have discussed earlier that the impact an event has on its target audience is equivalent to the measure of reach and interaction that occur during the event. Whereas reach is tangible, interaction to a certain extent is intangible as well as not always quantifiable. Immediate and long-term benefits that accrue from an event are important when evaluating an event from the clients’ point of view. A cost-benefit analysis concerning the effectiveness of reach and interaction is a must as a pre-event activity. Post-event stock taking activity should be done to confirm whether the event has occurred as per plans. This analysis should consider the actual cost of the event that includes the non-budgeted expenditure as well as the actual benefits that accrued to the client from the event. The accrual of benefits can be judged by measuring the tangible parts of the objectives that have been achieved.
Measuring Reach
Reach is of two types — external and actual event reach. Since events require massive external publicity-press, radio, television and other media are needed to ensure that the event is noticed and the benefit of reach is provided to the client. Measurement of external reach is possible by using the circulation figures of newspapers and promotions on television and the radio. The DART and TRP ratings that rate the popularity of programmes on air and around which the promotion is slotted, is a very tangible though approximate method for measuring the external reach of a promotion campaign on television. Measurement of external reach should be tempered with the timing of the promotions as effectiveness of recall and action initiated amongst the target audience is highly dependent on this important variable. For example, releasing ads and promos one month ill advance should be considered more as an awareness exercise for propagating the event concept, time, date and venue of these owe to the audience. The entry criteria – free, invited or ticketed show should be clearly mentioned here. The measurement of the actual reach of an event is relatively simple. The capacity of the venue is a figure that provides the upper limit for the actual reach. Ticket sales or numbers of invitees are also direct measurement tools. Registration of participants and requests for filling in questionnaires are also common methods of measuring the actual reach of an event.
A ratio of the external reach to the actual event reach is a very tangible and useful measurement criteria.
Ideally, External Reach / Actual Reach = 1
The ideal situation in real life is very rare since the external reach gets drastically reduced in terms of reaching out to the target audience and is therefore impractical in most cases. This is because the target audience is derived from the target population which is invariably very large. It is impractical to assume that all the constituents of the target population can make it to the event. The above ratio is usually found to be greater than 1 in practice.
Measuring Interaction
In most event categories, compared to reach, it is much more difficult to access the interaction between the audience and the event and the benefit that accrues to the client. A certain amount of quantifiable data can be of help in measuring interaction for an event from the clients’ point of view.
These are as follows:
- No. of interaction points: The no. of direct and indirect interaction points that have been planned and arranged for an event provide the first important measurement tool. The greater the no. of interaction points the better for the client.
- No. of interactions: The opportunity for interaction between the client and the audience before, during and after the event is also a very tangible measurement criterion. The greater the opportunity for increasing the no. of interaction, the better for the client.
- Quality of interactions: One-way or two-way communication during interaction has a profound impact on the quality of interaction that takes place. The quality of interaction is perceived as good when there is an avenue for two-way interaction
- Time duration of interaction: Every event has a limited time period within which both benefits the other issues such as controversies are effective. The amount of time that is available for interaction is very important in that the greater the duration of the interaction, more are the chances that there are some meaningful and decisive interaction between the client and the audience.
Concept of Event Quality and Measuring Quality of Event
Exactly on the lines of the evaluation of effectiveness of an event comes the concept of event quality. In essence, quality of an event exists in the clients’ perspective and thus varies from client to client. By aiming for quality by maintaining standards, preventing mistakes, never cutting corners and using only top quality infrastructure is looking at quality from a skewed angle.
Unless the target audience and the clients perceive the quality of the job in the same way as the event organizers, the big picture of quality is not complete. Therefore, it is critical to match the clients’ expectations and experiences by including even the minutest details to arrive at the perceived quality of event. In matters of dispute, it is value to the client that finally matters.
For the client, quality of an event is a bundle of attributes. A few of these critical attributes are quality and reliability of equipment used, aesthetic appeal, appropriate cost and timely completion of the project.
Each client will care more about some attribute than others. Thus, it is important to find out how clients would define quality event service. Competence in project management from conceptualization to carryout, reliability and integrity as in the past performances of events that have been executed by the event organizer is a very important quality criterion. Responsiveness to the clients’ requirements i.e., empathy, mutual confidence and trust are also criteria used by clients to size up the quality of event organizers. In addition, an easy-to. Work with manner, personal involvement and caring that the event organizer exudes also helps. Delivery of promises and deals should be ensured.
Every client expects the event to provide the ideal audience to associate with; impress and entice. Thus, the quality of an event can also be defined in terms of the audience quality. Clients should focus on three major statistics that define audience quality:
- Net buying influences which can be defined as the ratio of the number of audience that can recommend, specify or approve purchase to the total population at the event.
- Total buying plans imply the percentage of the audience planning to buy a product/service from the sponsors’ stables within the next 12 months after the show.
- Average audience interest is the percentage of audience that shows an interest in the sponsors’ products or services during the event itself and immediately after. This may be measured by keeping track of the number of visitors to the sponsors’ stall or exhibit area during the event.