Redefining Aboriginal Evaluation

Graphics for Evaluation Frameworks and Reports

Program Evaluation Models

For the Tending the Fire Program offered through Prairie Spirit Connections in Regina, Saskatchewan, an ordinary evaluation logic model was simply inappropriate. The program, designed for multi-barrier Aboriginal men who have substance abuse patterns and involvement with the ‘criminal justice’ system, is grounded in tradition because all of the staff are. The program staff each participate in the programs’ traditional ceremonies as participants on the same level as the men. The comprehensive program addresses multiple needs in providing financial assistance, housing, and assistance with equipment to gain employment, in addition to spiritual guidance and emotional healing. The ten month program expects the men to be job ready, however this is not always realistic so aftercare is provided through counselling and support services.

The challenge was to provide an evaluation framework that could be embraced by the traditional staff and satisfy the needs of funders to be assured that the program had all the elements needed for evaluation. For the staff, the framework needed to be visual and for the funders, the framework needed a logic model that included needs, resources, activities, outputs and outcomes. Representing the complex traditional program visually was the greater challenge and so a computer graphic was created in anticipation of an animated version for presentation to prospective participants and their families as well as to the community.

A visual representation of the program was developed in 3D to convey the notions of iterative recovery cycles (represented by the concentric circles) and that participation focused on a balance of the spiritual, cognitive, emotional, and physical aspects of the self (the overlapping circles in the center), and at the same time, the visual could be translated as aspects of a logic model.

Using the concepts of the logic model a program participant would follow a sequence of actions in each cycle; 1) Recognizing/acknowledging NEEDS, 2) willingness to move forward and PLANNING, 3) Putting plans into ACTION, and 4) OUTCOMES and transition to the next cycle. This is referred to as a cycle of change whereby both the participant and the program are ever-evolving. The participants’ recognition/acknowledgment of NEEDS at the beginning of the program might include: repair of nutritional deficiencies, spiritual grounding, and stable housing. At each iteration there would be a new assessment of needs as presumably some healing has occurred and some needs are being met, at least partially. Each year of the program evolves from the previous year’s lessons learned and this can be read into the graphic as well. For example, the programs’ recognition/acknowledgment of NEEDS might include information about comparable programs for Aboriginal males, professional development for staff, and core funding.

The traditional healing approach of the program assumes that each participant, including staff are on a healing journey described previously as repeating the cycle of change at progressive stages of healing. The program recognizes this by regularly changing emphases as the cohort of men progresses through the ten months. The Tending the Fire healing journey stages include: 1) Crisis intervention, 2) Awareness of broader issues, 3) Ownership of personal responsibility, 4) Releasing/letting go of anger/resentment, 5)Building on strengths, and 6) Standing tall. The healing journey is represented as a series of concentric circles with many iterations of the cycle of change categories.

As with most multiple-barrier Aboriginal people, there are profound needs for healing in many ways. All are needed simultaneously but the primary emphasis at the beginning is with spiritual and physical recovery. Without either, there is little possibility of progress. Emotional and mental (cognitive) will follow shortly and all will proceed simultaneously and synergistically. The visual model illustrates that in the first iteration, the mind and emotion as severely hurt through Crisis — the two spheres in the middle that are side-by-side. The top sphere is the spirit and the bottom the body. The program would build on the strengths of the spirit and body by addressing these immediately, in order to address the crisis needs of the more unstable effects of damaged mental and emotional pathways over the term of the program.

Presenting Visual Evaluation Reports

To make evaluation reports more accessible to community members, especially if one wants to include those without post-secondary education, it is necessary to minimize several typical aspects of research reports and maximize the visual messages. Reports for more experienced program planners and committees can be custom designed for their decision-making needs. Word processing programs that allow the inclusion of charts and graphics have made these options much more feasible than ever.

For reports to the community, we regularly use 13 and 14 point font sizes with wide margins to mimimize the impact of the volume of words. Ideally, there is a table, chart or illustration that accompanies each verbal presentation of data so that several types of readers have choices. We have also found that bullet points make the information more accessible to the visual reader. We use them even when the concepts are complex as we have found that several complex concepts in a dense paragraph are almost inaccessible to people not used to academic styles.

For program planners who have time to study a visual chart before making decisions, we have provided information in a complex visual way that would have taken thousands of words to convey. We were able to provide very variable information about six communities who had different combinations of programs, different numbers of staff for each program, with varying responsibilities and levels of training for each staff member for each of those responsibilities, and we were able to provide it all on one page. Once familiarized with the symbol use in this customized multivariate chart, the committee was able to see the forest and the trees all at once. The staff and community needs were all visible on the one page. We couldn’t even conceive of a way to communicate this amount of multivariate information in words or tables and make it effective and usable.


For a healing program, a before and after question about a participants’ position on the healing circle, was reported using an image of a person for each respondent spread around the circle of healing. This could also be completed using a person for percentage point for large samples. In the before measure many persons were at a point of illness and imbalance. In that after measure one can easily see that considerably fewer were ill or imbalanced any longer and instead most persons were at a point of healing and letting go.