engagement evaluation

Engagement is about achieving change – in people’s awareness, attitudes, understandings and behaviours.

These changes in turn translate into impacts in the broader political and biophysical landscape. When developing an engagement project’s objectives – its outputs, practises, desired outcomes and efficiencies – firstly translate them into performance indicators which can express the engagement’s intended change impacts.

Indicators are both qualitative and quantitative in nature and express a change in the state or condition of things or people. So collect original baseline measures of these indicators where you can. By setting an ‘ideal’ – a target or performance indicator -and measuring it against the original ‘baseline’ – you can compare the before and after condition. In this way indicators effectively signpost your engagement’s impact or success.

OUTPUTS – the products or activity associated with the engagement.
Success indicators numbers of things done or produced through the engagement …information products produced, events held or contacts made.

PROCESS – those best practise, ‘critical success’ factors which are within your project management control. The how, with whom, when and where of the engagement process. Reach, accessibility, representative inclusion, understanding, value, timeliness and responsiveness of the engagement.
Success indicators – quality of the engagement e.g. …inclusion (e.g. all impacted where informed), reach (e.g. 80% of all targeted stakeholders participated) …value (perception of stakeholders of the worth of the process).

INTENDED OUTCOMES – occur in the short, mid or long term and have impact at varying scales in a change system…e.g. better understanding of issues and stakeholder needs, long term partnership-building, policy influence and change, improved relationship quality (reputation), better service delivery or cost-effective and equitable resource usage.
Success indicators – what happened as a result of the engagement …e.g. 16 new  policy recommendations, 5 new inter-sectoral partnerships created, improved reputation with corporate stakeholders, demonstrable improvement in knowledge amongst participants.

EFFICIENCIES – that the financial, human and other resource inputs delivered the outputs at a reasonable cost, reached the desired audience in an appropriate way, and achieved the desired outcomes.
Success indicators – the financial, human and other resource use effectiveness of delivering a product, activity or outcome e.g. for the time the professionally-paid people spent to promote, organise, cater for and deliver a seminar at an expensive venue, did you get a cost-effective ‘bang for your buck’ in terms of targeted reach, stakeholder attendance, change in participant understanding and awareness … and the desired strategic engagement outcomes for the mid and long terms?

What makes a good indicator?
It is…

  • Accurately and reliably representative of what it indicates.
  • Understandable…what it means is demonstrable and clear.
  • Measurable …there is a before and after quantity or quality.
  • Collectable, collate-able and interpretable …in a timely  manner.

And most  importantly,

  • Changes in it’s value will express trends which trigger real world political or practical adaptations in response to its change !

 Adapt Strategic Communications…strategic engagement planning and facilitation, www.adaptstrategic.com.au

Engagement evaluation …a reputational strategy

Engagement brings benefit to your organisation and the communities it serves and evaluation helps to ensure that those benefits accrue mutually. Stakeholder engagement adds value – it shapes the responsiveness, quality and effectiveness of organisational programs and services. In turn, your stakeholders capacity and investment in these outcomes is built through collaboration in planning, decision-making and getting things done.

In order to continue to get things done and accrue the reputational benefits of engagement- evaluation is essential !

Engagement evaluation can…

Gather information …about the effectiveness of its engagements outcomes and processes.
Add adaptive value…informing and improving future engagement planning, practise and activities.
Explain… the relationship between the critical elements of action, context and outcomes.

and it answer questions…

  • Was it successful?
    Evaluation is accountable to the engagement project’s performance in terms of its plans and actual versus desired outcomes.
  • What did we learn?
    Evaluation obtains new knowledge about the context in which the engagement occurred, including problems and opportunities that were unforeseen and which affected the project’s success.
  • How does this influence future engagement practise?
    Evaluation shares the knowledge and evidence from this experience and supports future adaptations in engagement practise

Engagement evaluation includes …

  • Engagement project plans which include scheduled and budgeted data collection, analysis and reporting.
  • Informal and formal consultation with participants to assess engagement strategies.
  • Steering committees to establish and interpret a project evaluation framework
  • Briefs to undertake evaluation through external contractors or consultants.
  • Reporting systems which contribute to knowledge data bases.
  • Systems of ongoing review for engagement projects, programs and policies

Engagement evaluation follows 5 key principles…

1. It is integral to an engagement plan

Ideally it is part of the ongoing, adaptive management of your engagement project or program.

2.It is a process which has structure and strategy

Whilst informal evaluation is a natural part of your ongoing engagement delivery, you must have measurable performance indicators which reflect the engagement’s objectives and anticipated outcomes.
The evidence you collect needs validity in terms of the engagement’s impact.

3. It is best developed with stakeholder participation

Maximise the engagements involvement of representatives from key stakeholder groups both expert, internal and external and in all or any of the evaluation stages- design, data collection and analysis, interpretation and reporting recommendations. Stakeholders are great informants, they contribute richer and more meaningful interpretations of an engagement’s success, and because they participated, they want to know the outcome. Stakeholder input lends greater legitimacy to recommendations and they benefit from their involvement in evaluating the engagement.

4. It considers the context and associated social risks and modifies its methods accordingly

Are the performance criteria appropriate to the context – politically, socially and culturally?
Is the engagement around controversial issues or contested stakes?
Are there tensions between government and community or between sectors?
Can the expectations set by the engagement be met?
Do stakeholders have both the time and willingness to participate in evaluation?

5. It’s scale and scope reflects the engagement’s purpose, audience and impact intensity

Corporate engagement and government, public engagement require different approaches.
The former would look at informational, attitudinal and behavioural impacts and effects on reputational parameters. The latter is more likely to involve ongoing, systematic reporting, steering committee oversight and external evaluation. Matching the scope and methods of evaluation to the engagements original objectives and desired impacts is important. It’s scale should match that of the engagement undertaken – as a rule of thumb allocate no more than 10 % of a project budget to its evaluation.

Be sure to get the thumbs up from your engagement by building evaluation in to your original engagement planning…Adapt Strategic Communications…www.adaptstrategic.com.au