Service Evaluation Report 2005-06
1 Introduction and Aims
It was decided that an evaluation of the University Counselling Service would be carried out, as it was for the previous 3 academic years, with some minor alterations to some of the questions asked in the questionnaire.
The questionnaire was used as per the previous year with minor alterations in place to ensure questions were asked clearly and concisely. Part way through the academic year the system changed from paper to on-line forms. (See Appendix 1: Questionnaire). There was also a change in how and when questionnaires were distributed. At the end of each course of counselling every client who had indicated on their initial registration form that they would be willing to receive a Service Evaluation Questionnaire received an email. Individual counsellors were responsible for adding their clients’ names and email addresses to the list when they had finished working with them and for deciding the optimum time between finishing counselling and the questionnaire being sent out – up to 2 weeks. An email was sent out explaining the purpose of the questionnaire. Initially the email included a Word attachment that was to be completed; latterly the email contained a web link on which the recipient had to click and complete the e-form to submit. (See Appendix 2: covering email with weblink to questionnaire). Counsellors were encouraged to include all clients who had indicated they would be willing to receive a questionnaire, whether they had been seen only once or for a longer term of counselling and regardless of how well they appeared to engage with the process.
Of all the clients seen over the academic year, approximately 78% agreed to being sent an evaluation questionnaire. Consequently, 232 questionnaires were sent out. Participants were assured of confidentiality in that they were invited to print out a hard copy and anonymously return the completed questionnaire by post or (latterly) submit the form on-line with no reply address visible or traceable. No time limit was stated by which the questionnaires should be returned.
2.3 Data Collection
7 of the 232 sent questionnaires were returned in error (ie the email address was invalid). Over the academic year, 68 were returned completed, 5 were void. The response rate, therefore, is calculated to be 29% which is the same response rate as the previous year, although the sample range increased slightly (by 8%). Of the 154 sent as a Word attachment, 21 were returned (14%); of the 78 sent to be completed on-line, 42 were returned (54%). This indicates that online questionnaires invite a much higher response rate than Word attached forms. Of the Word attached forms, the data was collated manually including all verbatim responses which were typed out individually. With regard to the on-line forms, data was collated using the Formsbuilder software and all verbatim responses were pasted into the report document – a much faster and efficient system. Both Word and on-line data was added together to give totals. All qualitative responses were recorded verbatim without selecting out, to give a complete and unbiased view. (See Appendix 3: full results and verbatim responses). NB the on-line forms’ verbatim responses are pasted as written, and therefore include typographical errors as they appear on the submitted responses.
2.4 Evaluation of Methodology
The increased response rate of on-line forms and the efficiency in which data can be collated and calculated, indicate that the complete change to on-line forms will be beneficial. It could be useful to consider modernising the procedures to activate sending out questionnaires to increase efficiency and reduce time spent on manually completing activation forms.
See Appendix 3 for a full copy of results and verbatim responses.
4 Analysis of Findings: Discussion, Comparison to Previous Year and Recommendations
Q1: First contact with the UCS
Q1.1 How did you first make contact with the UCS?
The increase in first contact via email (from 28% in 0405 to 53% in 0506) is significant. Both the ‘in person’ and telephone contact has reduced by 12% and 13% respectively. However, the comments indicate that pleasant and efficient first contact in person is important. Generally, first contact needs to be reassuring, straightforward and prompt.
- Ensure reception staff are e-literate
- Continue reassuring, straightforward, prompt and pleasant first contact
Q1.2 What do you think to the UCS website?
(This question had not been asked previously)
75% indicated that the website was good or ok; 25% had not seen it. The comments suggest that although the information is generally useful, the site itself could be clearer to navigate and maintained more conscientiously.
- Consider revamping the website
- Ensure the site is maintained regularly
Q1.3 Do you think the UCS is marketed well enough (ie is enough information about the UCS easy to find and obtain)?
Only 60% felt that the UCS is marketed well enough. Staff still seem unclear about whether the service is available to them for counselling. Generally most comments suggested more visibility around campus. Despite the logo being mentioned as and ‘important way of attracting attention’, the UCS was instructed to remove it from all material as it had not been sufficiently ratified. The term ‘marketing’ in this context was objected to. GPs seem to ‘ market’ the service widely to their patients. Once people are using the website for information there seems satisfaction about the information available but initial awareness is low.
- Ensure information about the UCS is distributed appropriately to all departments for dissemination through the Departmental Consultancy Initiative
- Ensure new staff still receive information about the UCS in their introductory pack
- Re-word the question to remove the term ‘marketing’< /span>
- Discuss strategic marketing with senior management (holding in mind the natural marketing tension between resources and demand)
Q1.4 Does your experience of the UCS match (or even exceed) your expectation of it (ie does it fit with what you heard/read/understood about the Service)? First time asked:
85% of respondents felt that their experience matched or exceeded their expectation of the UCS. The main areas that did not match included the long time to get an appointment and a few where a long-term in-depth approach was anticipated.
- Ensure clear information about what to expect is available – perhaps including an explanation of what the service cannot provide
- Include (perhaps on the website) options of what might help if counselling at the UCS does not match need
Q1.5 On your first contact with the UCS, was the response efficient and appropriate?
The positive response to this question had dropped by 7% this year and the key concern is the wait-time for an appointment. However, 85% of respondents felt the initial (first contact) response was indeed efficient and appropriate
- Consider sending an email to those on the waiting list if their wait is protracted
- Ensure the web counter is as accurate as possible
- Ensure the ‘wait’ is presented positively
- Continue to prioritise responding promptly to first-time contacters
Q2.1 What do you think of the location of the UCS?
This year saw a 10% increase in the percentage of respondents who considered the location of the UCS as ‘poor’. The percentage of those considering the location ‘good’ remains similar. As with previous years, respondents are still concerned as to the position of the UCS within the open-plan layout.
- Continue to engage management to consider the UCS location
- Ensure signage is clear and copious
- Position UCS information adjacent to UCS area (so people can get a feel for the place and ‘see what its like … before the first session’)
Q2.2 What did you think of the reception area (where the Departmental Secretary sits)?
47% felt this was good, an improvement of 9% from last year. The key issue of concern is the need for privacy around the secretary/receptionist area, for both those waiting and for the secretary to carry out [her] duties with appropriate respect for privacy.
- Consider how the reception area can afford adequate privacy
Q2.3 What did you think to the waiting area?
Privacy is the essential requirement for respondents with regard to the waiting areas. The current waiting area is generally considered too small yet the availability of water, bins for tissues and reading material is appreciated. There was confusion and discomfort around the waiting area also used by funding and disability staff and clientele.
- Consider how the waiting area can afford more privacy
- Consider a separate counselling service waiting area
Q2.4 What did you think to the counselling room? and
Q2.5 What is important for you in a counselling room?
Respondents continue this year to highly rate the counselling room (‘good’ 80%). Comments consistently year-on-year indicate the important elements of a counselling room as privacy, comfort (chairs, lighting, regulated temperature, air) quietness (no disturbances, etc) with a relaxed-yet-professional atmosphere.
- Ensure counselling rooms are as quiet as possible
- Continue to offer comfortable, private counselling rooms appropriately furnished
- Chase up the order of new chairs in line with Health and Safety requirements
Q3: The Counselling Experience
Q3.1 Do you feel you had to wait too long before you were offered a counselling appointment after you had registered?
59% of respondents felt that their wait had been too long, an increase of 17% from the previous year. The wait is perceived by some as useful time to prepare to get the most out of their counselling, by some as inappropriate to meet any immediate/urgent/desperate need for support/intervention. There is a sense of ‘being forgotten’ between registration and initial appointment as there is no contact from the UCS until an appointment is available. There is continuing misinformation about the procedures for engaging with the UCS, the perception being that, with intervention from a member of staff, people waiting are moved up the list.
- Ensure clear information is available to all (students and referring staff) about the remit of the UCS – particularly that it is not a mental health emergency resource able to respond to immediate or desperate need for support
- Endeavour to keep waiting lists to an expedient number of days
- Consider contacting those on the waiting list to keep them informed
Q3.2 Do you feel that the number of counselling sessions you contracted was appropriate to your needs? (first time asked)
79% of respondents felt the number of sessions was appropriate. It was considered useful to have a clear focus for counselling before commencing. Some respondents felt they had to stop before they felt ready – this may be due to the trial in term 3 to offer limited counselling sessions in order to attempt to meet demand. Counsellors ‘leaving’ is considered disruptive – this may refer to counsellors on short-term sessional contracts who are unable to offer continuing appointments yet may be re-employed the following term. Flexibility in contracting is valued.
- Promote the ‘preparing for counselling’ information more proactively
- Consider ways to meet demand in term 3 without reducing session numbers
- Ensure sessional counsellors are offered contracts that are confirmed for at least an academic year
- Continue offering flexible contracts
Q3.3 Are you satisfied with the confidentiality policy of the UCS?
A high satisfaction (97%) was recorded again this year. The theme throughout the comments continues to be an appreciation of confidentiality as a central tenet of effective counselling.
- Continue to ensure the professional management of confidentiality as far as is ethically and practicably possible
Q3.4 Would you recommend others to use the University Counselling Service, if appropriate?
93% of respondents answered affirmatively. The reasons for offering a negative response included concerns about the waiting process and the experience of having the number of sessions limited (term 3).
- Ensure there is clear information about the process of undertaking counselling (maybe on the website)
Q3.5 Generally, do you find counselling useful?
90% of respondents replied in the affirmative. Negative concerns included having too little time available (probably term 3). It appears to becoming apparent that, as one astute respondent recorded, ‘you get out what you put in’.
- Continue to provide a high standard, effective counselling service which, as far as can be expected, meets the needs of its users
Q3.61 and 3.62. Respondents were asked to complete either 3.61(students) or 3.62 (staff) as applicable.
Students who responded indicated that they thought that counselling can help to manage anxiety (90%), in life as a whole (89%), improved their ability to do their course (69%), develop ‘ emotional literacy and intelligence’ (65%) and to stay on at University. The related comments attest to the range of impact counselling can have, from offering coping strategies to significant personal development and life re-evaluation. The development of ‘emotional literacy and intelligence’ was significant in the light of the UCS bid in response to the VC’s Teaching Innovation funding request.
The responses may indicate that counselling has a significant part to play in staff retention and well-being.
- Continue to explore the effect of counselling, particularly in terms of the development of emotional literacy and general well-being
Q4.1 Overall level of satisfaction with the UCS
80% of respondents reported feeling satisfied or very satisfied with the UCS. No-one was very dissatisfied.
Q4.2 What could be improved to increase overall satisfaction?
Most comments pointed directly to improvements being necessary in the waiting time for an initial appointment (including contact between registration and appointment) and an increase in the number of appointments available. The privacy of the location was also a factor that was considered unsatisfactory.
- Continue to consider the pros and cons of an initial ‘ assessment/signpost’ interview/appointment
- Continue to bid for resources to ensure the service is staffed in line with the rise in numbers available to use the service
- Continue to pursue privacy as an imperative in terms of location
Q4.3 Completing statements about the experience of the UCS:
It may be better if…
A significant number of comments (approximately 75%) related to reducing waiting times and to the number of appointments available. Other points referred to the location of the service and the style of counselling offered.
I did not like...
Nearly 50% or respondents referred to their dissatisfaction with the confidentiality of the location; nearly 40% did not like aspects of the length of wait time; 15% expressed dislike for their counsellor or counselling style.
It was good that...
Comments included expressions of appreciation for the work of their counsellors, the rooms, the logistics of the service and the helpful process of counselling, to varying degrees.
What surprised me was…
Comments were various but significantly referred to the effectiveness of good counselling.
- Continue to consider ways to manage appointment availability
- Ensure the location of the UCS upholds the presumption of confidentiality
- Continue to offer a high standard of counselling and service to all clients, whilst being flexible to meet need, as far as is practicable
- Continue to ensure clear information about counselling and its potential is disseminated, perhaps by updating verbatim quotes from respondents on the web site
Q4.4 Are there any other comments you wish to make?
Respondents generally used this question to endorse their positive experience of counselling. (NB it is recognised that those who choose to respond to evaluation questionnaires of this nature may provide skewed responses). Other comments included suggestions regarding the questionnaire (which has, during the year, been modified).
- Continue to monitor feedback of the UCS to ensure high standards of service are maintained