Primary
Phase - Final Report
The Mental Health Performance Measures Pilot Project is a
joint effort of the Alaska Mental Health Board and the Division of Mental
Health and Development Disabilities (DMHDD) that began in February 2001. The goal of this project is to develop
standardized outcome measures to be used at all state-funded mental health
treatment facilities in Alaska. To
finalize these questionnaires, DMHDD contracted with the Alaska
Comprehensive and Specialized Evaluation Services (ACSES) to serve as
independent consultants to conduct a pilot project on draft
questionnaires. The pilot project has
been approached in two phases, a preliminary phase and a primary phase. The work of the Preliminary Phase, to
collect clinician and consumer feedback about the ease of use, structure, and
utility of the instruments, has been completed and results were submitted to
the Oversight Committee. The goal of the Primary Phase, to evaluate the
sensitivity of the revised versions of the instruments to consumer change
across time, has been achieved and results are presented in this report.
Reports
Prepared by ACSES and Submitted to the Oversight Committee:
Instrumentation
Three instruments were used in the Preliminary Phase,
namely, the Client Assessment Worksheet, Mental Health Consumer Satisfaction
Survey, and Mental Health Statistical Improvement Program Consumer Survey
(MHSIP). Based on feedback from
clinicians and consumers, the Oversight Committee made additional revisions to
these instruments after the Preliminary Phase. The resultant instruments are
the Client Status Review, Demographics Questionnaire, and the MHSIP.
Client Status Review
For the Primary Phase, the revised Client Assessment
Worksheet was renamed the Client Status Review. Appendix Seven provides the
original Client Assessment Worksheet; Appendix Eight provides the Client Status
Review used in the primary phase of this pilot project.
Demographic Questionnaire
For the Primary Phase, the revised Mental Health Consumer
Satisfaction Survey was renamed the Demographic Questionnaire. Two versions of the Demographic
Questionnaire were developed, an Adult Services version and a Child and Family
Services version. Appendix Nine
provides the original Mental Health Consumer Satisfaction Survey; Appendix Ten
provides the Adult Demographic Questionnaire; Appendix Eleven provides the
Youth Demographics Questionnaire.
Mental Health Statistical Improvement Program Consumer
Surveys
Three versions of this instrument were used, namely, Adult
Survey, Youth Services Survey, and Youth Services Survey for Families. Appendix Twelve provides the original Adult
MHSIP. The original youth surveys
consisted of 26 items; subsequent revisions (by the national group that is
developing and refining this instrument) have resulted in 21-item
questionnaires. Appendix Thirteen
provides the original Youth Services Survey; Appendix Fourteen provides the
Youth Services Survey for Families.
Global Assessment of Functioning
In addition to administering these questionnaires,
clinicians were asked to provide a current Global Assessment of Functioning
(GAF) for each client. GAFs were
collected to provide an established measure of client functioning against which
to compare the overall CSR results. It
should be noted that GAFs are not precise measures; however, clinicians are
trained in their use and they do provide some data for comparison purposes.
Procedures
Prior to data collection, an ACSES staff member visited each
of the participating agencies to provide training regarding timing and
procedures, including information on research design, data submission
procedures, and confidentiality issues.
Prior to administering the instruments, clinicians were to ask consumers
if they would be willing to help with the project by answering the questionnaires,
providing opinions about the questions, and making recommendations about
possible changes.
Time One Data Collection
At Time One, after receiving consumer consent, clinicians
administered the CSR in a structured interview format to as many consumers as
possible over a one-month time period (later extended to last six weeks), and
asked the same consumers to complete and return the Demographic Questionnaire
and MHSIP independently. The one-month
time period was selected to allow for missed appointments, to allow inclusion
of consumers who are on a once-a-month schedule, and similar
circumstances. To match Time One and
Time Two responses, questionnaires were coded with consumers’ ARORA or agency
identification number (no names were used anywhere on the research
protocols). Due to two agencies getting
started late, the data collection period was extended for an additional two
weeks to maximize data collection opportunities; all sites were notified of and
invited to participate in this change of timeframes. Actual Time One dates differed from administration site to
administration site and are provided in the individual agency descriptions
below.
Subsequent to completing the CSR with the consumer,
clinicians handed the consumer a packet containing the Demographic
Questionnaire and the MHSIP to complete independently and gave them a brief
overview of the tasks involved. A cover
letter from ACSES to the
consumer
was included in the packet to explain the purpose of the project and the
procedures used to insure confidentiality and anonymity. On behalf of ACSES, clinicians also provided
stamped, self-addressed envelopes to the consumers, who had the option either
to mail the questionnaires directly to ACSES or to seal the envelope and give
it to an agency staff member (e.g., a receptionist) who would mail it for
them. Consumers were also given the
option of asking clinicians to help them complete the questionnaires if they
needed assistance. Due to the low rate of return by mail from consumers during
the Preliminary Phase, clinicians were asked to encourage their consumers to
exercise their right to give feedback about the services they receive and, if
possible, to complete the questionnaires while at the agencies and return them
in sealed envelopes to a staff person to be mailed for them.
Time Two Data Collection
Time Two administrations were conducted four to six weeks
after Time One, following the same procedures outlined for Time One and using
the same questionnaires. During Time
Two, clinicians administered the questionnaires to the same consumers
who responded during Time One. Any
consumers who completed the questionnaires during Time One and were discharged
before Time Two began, were to be administered the questionnaires the second
time at their time of discharge.
Ideally, consumers were to complete the questionnaires during a
termination session. However, if the
consumers did not attend a termination session, the clinicians were asked to
mail all three instruments to the consumers to complete and return in a
pre-paid, self-addressed envelope to ACSES.
Time Two data collection began on July 16 and was completed for all
agencies on August 13, 2001.
Due to the low participation rate at
several of the sites, as reported in the Interim Report, the Oversight
Committee requested that more Time One data be collected. This was particularly important for
gathering rural consumer and clinician feedback, as well as additional input
from child and youth consumers.
Arrangements for additional Time One data collection were made; agencies
did not collect Time Two data for those consumers.
Based on community size, region, willingness to participate,
number of consumers serviced, and informed consent procedures in place, six
pilot sites were selected by DMHDD to participate in this pilot project. These agencies were as follows: Southcentral Counseling Center, 4Rivers
Counseling Service, Alternatives Community Mental Health Center, Bristol Bay
Mental Health Center, LifeQuest, and Norton Sound Behavioral Health Service. A brief description of their level of
participation during the Primary Phase follows.
Staff training for the Time Two administration
phase was conducted with the clinical associates on July 5. Time Two data was collected from July 16
through August 13. It is notable that throughout this pilot
project participation at this agency was higher than at any of the other
agencies. This may be attributable to
the Executive Director’s strategy to solicit volunteers, set a minimum level of
participation, and offer monetary and compensatory
incentives to each staff participant who achieved
the requisite participation level. SCC staff members are to be commended for their
efforts in this project.
4Rivers Counseling Service, McGrath
Staff training for Time One of the primary phase was
conducted on May 4 with the Executive Director. The primary clinician at this site continued to have strong
personal objections to the overall project and called in sick on the day of the
training. However, on the same day as
the training, an ACSES staff member observed this clinician to be in good
health on the flight from Anchorage to McGrath. The Director prepared detailed written instructions for the
clinician outlining the administration process and a directive that she was to
participate by administering the questionnaires to her consumers according to
the research design procedures.
The clinician called an ACSES staff person stating that,
because of to 4Rivers policies and procedures, the clients’ bill of rights, and
her ethical practice, she could not administer the surveys to clients aged
13-18 without written informed consent of the parents. She reported that this was consistent with
her need to obtain written informed consent from parents each time she traveled
to her assigned villages to provide mental health services. She expressed a great deal of concern that
asking her clients to complete the questionnaires was not part of their
treatment plans and, therefore, outside the realm of professional
practice. She also reported being
unable to administer the questionnaires to children and youth because she saw
these consumers in their school environment and therefore did not have contact
with the parent/primary caregiver to obtain informed consent. The ACSES staff person notified the
Executive Director of the clinician’s request for an informed consent form and
the need to coordinate this request with DMHDD (as other agencies had
successfully done). The informed
consent form developed for this project was provided to the Executive
Director. Anne Henry, DMHDD, also
talked with the clinician to impress upon her the importance of gathering information
from her client population. The
clinician stated that she was participating; however, she did not schedule
further trips to the villages during Time One administration and was on
vacation for the month of June.
The Director prepared a short introduction
letter from his agency to explain the project to consumers and to emphasize the
importance of rural input into the project.
Data collection was scheduled to begin on May 7 and to be completed on
June 15. Because the Director was out
of McGrath for most of the Time One administration period and the clinician did
not administer any questionnaires, no data was received from this site. The lack of data from Time One precluded the
agency’s participation during Time Two administration.
Alternatives CMHC, Anchorage
Staff training for Time One of the primary phase was
conducted on May 10 with clinicians and case managers, with much discussion
about the administration process. Staff
wanted to administer the questionnaires to their consumers aged 18-21 who were
involved with the Division of Juvenile Justice. For this group of consumers, the adult questionnaires were used,
with the clinicians noting the consumer’s age and custody status on the forms.
The Utilization Review Manager expressed the need for an
informed consent prior to administering the questionnaires. Although it was explained that the consent
form developed for the Preliminary Phase was sufficient, the manager remained
firm in the need for a different consent form.
This issue was referred to Executive Director to resolve with
DMHDD. A staff person noted that the
Division of Family and Youth Services (DFYS) has its own regulations about
consumers participating in this type of project. The Director decided that they would
contact
DFYS about giving them a general consent for their consumers to participate in
the project, which might eliminate the need for youths to complete the consent
forms. Developing and coordinating
another consent form with DMHDD and other projects with which the agency was
involved led to delay in data collection.
Data collection started the week of May 21 and was completed on June
15.
Time Two staff training was
conducted on July 10, and data were collected from July 16 through August
13. Staff also collected additional
Time One data during this phase from consumers who had not yet participated in
the project. During the Primary Phase,
the agency experienced significant staff turnover that negatively affected
staff’s ability to collect an adequate sample size of consumers. The Executive Director and the Utilization
Review Coordinator remained interested, cooperative, and enthusiastic about the
project, and did their best to collect data.
Bristol Bay Mental Health Center, Dillingham
Staff training for Time One of the primary phase was
conducted on May 1, with data collection for adult consumers starting on May 2
and scheduled to end May 30. The
Executive Director requested that staff administer the questionnaires to child
and youth clients as well. However, no
child or youth surveys were administered because the assigned clinician felt
the CSR was not age-appropriate. In
some of the villages, village-based family social workers were to be asked to
administer the questionnaires to their clients, instead of agency staff. A ‘village code’ number on the form would
identify this. Time One data collection
was conducted from May 8 through June 15.
Staff training for Time Two was held
on July 9 with data collection occurring from July 16 through August 13. Staff also collected additional Time One
data during this phase from consumers who had not yet participated in the
project.
LifeQuest, Wasilla
Staff training for Time One of the primary phase was
conducted on May 3 with case managers and the Medical Services Team; staff
training for outpatient clinicians was conducted on May 7. The Medical Services Team opted out of
administering the questionnaires due to the ‘non-billable’ nature of the
service and concerns that the data would not show improvement unless administered
at intake, since the majority of their consumers were already stabilized on
medications. Data collection by case
managers began on May 7; outpatient clinicians started data collection on May
8. Data collection was completed on
June 15.
Time Two staff training was
conducted on July 12 with case managers and outpatient clinicians, with data
collection scheduled to begin on July 16 and to be completed on August 13. When no data had been received from the agency
by August 27, an ACSES staff member contacted one of the managers coordinating
the project at LifeQuest. This
individual reported being unsure what had happened, stating that the departure
of the administrative person coordinating the project and staff’s busy
schedules were the most likely contributing factors to the lack of data. The manager reported that staff was ‘so
pushed’ with other demands that she had not asked them to do more. The Executive Director, however, stressed
the importance of gathering the Time Two data to the managers and the data
collection period was extended until September 14 to permit staff to
re-administer as many surveys as possible.
This deadline was extended again until September 21, when data had still
not been received. Due to the need to
establish a cut-off date that would permit adequate time to evaluate the data,
the deadline was
changed
to September 19, and an ACSES staff person traveled to Wasilla to pick up any
data that had been collected.
Norton Sound Behavioral Health Services (BHS), Nome
Staff training for Time One of the primary phase was
conducted on May 15. For the first week
during the administration period, no surveys were administered because most of
the staff was attending a local training conference. In some of the villages, staff planned to ask the village-based
counselors to administer the questionnaires.
Data was collected from May 21 through June 15.
Time Two staff training was
conducted on July 13, with data collection occurring from July 16 through
August 13. Staff also gathered
additional Time One data during this phase from consumers who had not yet
participated in the project. Despite
the Executive Director’s continued interest and enthusiasm about the project,
the agency’s participation rate continued to be lower than expected. This significantly limited the rural input
that was so important to the project.
Findings are presented in three sections: 1) staff
observations, which outline observations made by ACSES staff regarding the
level of cooperation and enthusiasm about the pilot project among participating
agency staff; 2) quantitative results, which provide data derived from the
questionnaires; and 3) qualitative results, which summarize written comments
made by consumers and clinicians about the questionnaires. Individualized reports regarding consumer
satisfaction at Time One were prepared for three agencies for which adequate
data were available. These reports were
distributed directly to these agencies and are not part of this report.
Clinicians’ responses to participating in the pilot project
were obtained from direct observation by an ACSES staff member during staff
training and from direct comments by clinicians. As in the Preliminary Phase, responses ranged from enthusiastic
acceptance of the project to significant resistance. Staff at most of the agencies were professional and
cooperative. However, at one agency,
some staff did not even look at the questionnaires during the training; at
another agency, one clinician avoided the training by calling in sick. All agency directors continued to be very
cooperative and interested in participating.
Specific levels of staff interest and cooperation at the various
agencies can be inferred from the response rates contained in Table One.
Several agencies
reported difficulties obtaining consumer participation for Time Two
administrations. Reasons cited included
not being able to contact consumers by phone, consumers being unwilling to meet
with the clinician, and consumers being on extended vacations or absences from
home during late summer months or relocating out of state. At two agencies, staff turnover and
workloads were reported as obstacles to completing the Time Two
administrations. Another concern had to
do with
language, with some staff members reporting that, for many of their clients,
English was a second language, creating a significant barrier to data
collection. Another concern expressed
by some staff members was that administering the instruments could have a
negative impact on therapeutic relationships.
These individuals suggested that an independent party, rather than
agency staff, should administer these types of
questionnaires.
Finally, several expressed concerns that the State would cut funding if
consumers did not show improvement over the two administrations.
Response and Match Rates
Table 1 provides the number of instruments completed at
Time One and Time Two, broken down by agency.
Across all agencies, 150 completed CSRs were submitted to ACSES for Time
One and 69 CSRs were submitted for Time Two.
In addition to the 150 CSRs returned at Time One, 21 CSRs were collected
out of phase with no plans for a Time Two administration; these 21 CSRs were included
in Time One analyses. Of the total 171 Time
One CSR protocols, 37.4% (n=64) were submitted by SCC, 33.9% (n=58) by
LifeQuest, 16.4% (n=28) by Bristol Bay, 7% (n=12) by Norton Sound, and 5.3%
(n=9) by Alternatives.
The length of time between Time One and Time Two
administrations ranged from 37 to 135 days, with a mean of 72.6 days
(SD=26.9). Based on ARORA numbers
provided on the CSR, it was possible to match 67 Time One (n=150) and Time Two
(n=69) CSRs, representing an overall match rate of 44.7% (67/150). Two of the Time Two CSRs included ARORA
numbers that did not match with any Time One ARORA numbers. It is unknown
whether this means that two consumers were administered Time Two protocols
without having received Time One protocols, or whether ARORA coding was
incorrect. The five participating
agencies varied in the overall match rate between Time One and Time Two as
follows: Southcentral Counseling Center, 75% (48/64); Bristol Bay, 23.5%
(4/17); Life Quest, 22.4% (13/58); Norton Sound, 20% (1/5); and Alternatives,
16.7% (1/6).
For Time One, 106 Demographic Questionnaires and 106 MHSIPs
were received from consumers. For Time Two, 54 Demographic Questionnaires and
54 MHSIPs were received. An additional
15 Demographic Questionnaires and MHSIPs were received for and used in
subsequent Time One analyses. These
additional protocols had no matching Time Two data. Of these received Time One forms, 112 were adult versions and 19
were youth versions; of the 54 Time Two forms, 53 were from adults and 1 from a
youth. On the consumer-completed
questionnaires, to insure confidentiality of responses, no means to match Time
One and Time Two information was requested.
However, by comparing responses to demographic items, it was possible to
match 42 Time One and Time Two Demographic Questionnaires and MHSIPs, for an
overall match rate of 39.6% (42/106).
Consumer Characteristics
Based on the place of administration of the 171 Time One
CSRs, it was possible to determine that 76.6% of the participants were from
urban areas and 23.4% were from rural areas.
Unfortunately, of the 67 matched CSRs, only 7.5% (n=5) were received
from rural areas. Table 2 provides Time
One and Time Two consumer and clinician responses for the adult version of the
Demographic Questionnaires. Based on
Time One data, 51 (45.5%) were female, 46 (41.1%) were male, and 15 (13.4%)
gave no response. The majority of adult
respondents were in the 23-59 years old category (83.9%, n=94). Of these consumers, 58 (51.8%) were
Caucasian, 40 (35.7%) Alaska Native/American Indian, five (4.5%) African
American, three (2.7%) Hispanic/Latino, two (1.8%) Asian/Pacific Islander, one
(0.9%) did not know, and three (2.7%) provided no response. Of the consumers, 67% indicated receiving
case management services, 76.8% psychiatric services and medications, 33.9%
transportation services, and 19.6% housing.
These figures suggest that the typical consumers who participated in
this pilot study were persons with persistent and severe mental illness.
Table 3 provides Time One and Time
Two consumer responses for 19 youth and child version Demographic
Questionnaires. Based on Time One data,
the majority of minor respondents were in the 6-12 years old category (63.2%,
n=12). Of the respondents, 13 (68.4%)
were Caucasian, three (15.8%) Alaska Native/American Indian, one (5.3%)
African-American, and two (10.5%) Other.
Client Status Review
On average, clinicians reported that
it took 10-30 minutes to administer the CSR, with the most commonly reported
timeframe being 10-15 minutes. A few
clinicians reported taking longer, depending on the client’s intellectual level
or diagnosis. One clinician noted that
‘scheduling time to meet with the clients was more of a problem’ than
administration time.
Of the 171 CSRs received during Time One administration, five
questionnaires were missing the consumer’s ARORA or agency number. Most of the missing ARORA (or agency)
numbers were later obtained by an ACSES staff member; however, this took
additional staff time. Almost
three-quarters of the Time One instruments (70.8%, n=121) were returned without
the Sum of Ratings section completed.
Of the 50 CSRs returned with the Sum of Ratings scored, eight (16.0%)
were scored incorrectly. Of the 69 CSRs
received during Time Two, all included an ARORA or agency number. Of these 69 Time Two CSRs, 18 (26.1%) did
not provide calculated Sum of Ratings.
Of the 41 returned with the Sum of Ratings scored, three (7.3%) were
scored incorrectly.
Initial review of the CSR data
revealed some missing responses, particularly on the five components of
Question #6. At Time One, 7.0% of
protocols had at least one of the 10 responses missing; at Time Two, 4.3% had
at least one item missing. Table Four
provides the specific number of missing responses for each CSR item. As the Sum of Ratings is calculated based on
responses to all 10 items, any missing data will yield a Sum of Rating that is
not indicative of the consumer’s actual level of functioning. An alternative would be to calculate a Sum
of Rating based on the average score to the responses provided; this would then
yield a score that is comparable to other Sum of Ratings received from the same
or other individuals. However, scoring
the questionnaire in this matter is more complicated and prone to error.
Table Five provides Time One and
Time Two responses for the 67 matched CSRs.
Table Six provides means and standard deviations for each CSR item and
the Sum of Ratings for Times One and Two.
As a way of reminder, on the CSR lower scores indicate higher level of
functioning. Thus, the desired change
is one of lower scores at Time Two as compared to Time One. Thus, a difference score (typically
calculated as Time Two minus Time One) that is negative indicates
improvement, whereas a difference score that is positive indicates worsening. Table Six provides these mean difference
scores and results of t-tests comparing scores at Time One and Two. For the 10 primary items, four had negative
(improved) difference scores means (with difference scores ranging from -.03
to- .25), five had positive (worsened) difference score means (with difference
scores ranging from .01 to .15), and one had equal means (i.e., a difference
score of 0) at Time Two as compared to Time One. The Sum of Ratings was negligibly higher (worse) at Time Two than
Time One. Results of the t-tests
revealed that the differences between Time One and Time Two failed to reach
statistical significance on all individual items and the Sum of Ratings.
To look at change from Time One to
Time Two for each item and the Sum of Ratings in yet another way, Table Seven
provides the number of consumers who, at Time One versus Time Two, had either
the same, a lower, or a higher score.
Table Seven also provides results of McNemar’s change tests (Siegal,
1956), used to determine whether changes in responses to these items increased
or decreased in a consistent manner across consumers from Time One to
Time
Two. For all of the individual items,
the highest number of respondents had no change on the individual items. For those individuals for whom change was
reported, on a majority of the items the change was in the direction of
improvement (i.e., a decrease in item score).
However, when considering the Sum of Ratings, the proportion of consumers
who had the same scores was very low; the proportions who had an increase or
decrease in functioning was nearly identical.
Results of McNemar’s tests revealed
significant decreases in scores (improvements of functioning) on four items,
namely, “How often can you currently get the physical health care that you
need?”, “Have friends/relatives asked you to cut down on alcohol or other drugs
or to quit entirely?”; “Have you experienced guilt because of your drinking or
use?”; and “Do you need an eye opener in the morning to get started?” Note that for the three drinking related
items, decreases (improvements) meant that the individuals stated “yes” at Time
One and “no” at Time Two.
In terms of overall functioning, at Time One, the Sum of
Ratings ranged from 8 to 34, with a mean of 19.4 (SD=4.3) and with lower scores
indicating better functioning. GAFs
obtained at Time One ranged from 21 to 95, with a mean of 48.6 (SD=12.0) and
with higher scores indicating better functioning. The correlation between the Time One Sum of Ratings and GAF
scores was -.37. At Time Two, the Sum
of Ratings ranged from 14 to 42, with a mean of 19.5 (SD=5.0). GAFs obtained at Time Two ranged from 21 to
80, with a mean of 42.3 (SD=10.8). The
correlation between Time Two Sum of Ratings and GAF scores was -.22. These levels of correlations suggest that
the GAF and Sum of Ratings measure different aspects of consumer functioning.
MHSIP
Table Eight provides Time One and Time Two responses to the
adult version of the MHSIP. Table Nine
provides means and standard deviations for each MHSIP item and the Domain
Subscales for Times One and Two. Table
Nine also provides the mean difference scores and results of t-tests
comparing Times One and Two. Of the
means for the 28 items, 20 were lower at Time Two than Time One (indicating
higher levels of satisfaction), seven were higher (indicating lower level of
satisfaction), and one was the same. Of
these differences, t-tests revealed statistical significance for only
one item, namely, “Encouraged to use consumer programs”, with greater
satisfaction being expressed with this item at Time Two. Of the four subscales, all were somewhat
lower at Time Two than Time One, indicating slightly higher levels of
satisfaction at Time Two; however, the differences were not statistically
significant.
Table Ten provides the number of
consumers who had the same, lower, or higher scores at Time Two as compares to
Time One for each item and the domain subscales. Table Ten also provides results of McNemar’s change tests
(Siegal, 1956), used to determine whether changes in responses to these items
increased or decreased in a consistent manner from Time One to Time Two. For most of the items, the highest number of
respondents had no change on the individual items. For those individuals for whom change was reported, on 22 of the
items, the change was in the direction of greater satisfaction (decrease in
score); on four of the items, change was in the direction of lesser
satisfaction, and on two items the number of consumers whose satisfaction
decreased and increased were the same.
Results of the McNemar’s tests revealed significant changes in scores on
only four items, all in the direction of greater satisfaction: “Staff were
willing to see me as often as I felt necessary”; “Encouraged to use consumer-run
programs”; “I do better in school and/or work”; and “My housing situation has
improved”. When considering the four
domain subscales, the majority of consumers reported
slightly
higher levels of satisfaction at Time Two than Time One, although these differences
did not reach statistical significance.
Table Eleven provides Time One
responses to the Youth Services Survey for Families. No Time Two data were provided on this instrument. Table Twelve provides Time One and Time Two
responses to the Youth Services Survey.
Given that only one Time Two questionnaire was collected, no further
analyses were conducted.
At the request of the
Oversight Committee, additional analyses were conducted to assess whether rural
and urban respondents differed from one another in their responses to the
instruments. Additionally, analyses
were requested regarding potential differences between consumers of Alaska
Native heritage as compared to those of other ethnic backgrounds. The rural versus urban comparisons were
based on location of data collection and hence were possible for all
instruments. Ethnicity was not assessed
on the CSR, making such analyses impossible for this instrument. As ethnicity information is collected on the
Demographic Questionnaires, it was possible to conduct analyses on ethnicity
for the MHSIP and Demographic Questionnaire.
Rural versus Urban
Analyses: Initial review of the
Demographic Questionnaire revealed that consumers from urban and rural agencies
differed significantly on a number of critical variables related to type of
clientele served. Specifically, urban
consumers represented in the current sample were more likely than their rural
counterparts to report that they are:
·
receiving psychiatric services/medications (91% urban; 56% rural);
·
receiving case management (86% urban; 25% rural)
·
receiving long-term care (2 years or more; 72% urban; 34% rural)
·
non-Native versus Alaska Native (79% urban; 39% rural)
These findings strongly suggest that the urban and rural
samples included in this project were not equivalent, but rather were comprised
of significantly different consumers.
Specifically, it appears likely that the urban sample was comprised of a
relatively homogeneous group of consumers who received intensive long-term care
and who were likely to represent a persistently and severely mentally ill
population. The rural sample appeared
to be comprised of a much more heterogeneous group of consumers. Further, rural versus urban location of
assessment was highly correlated with ethnic background, making it impossible
to tease out the different effects of these two ways of grouping the
sample.
Given the dissimilarity of the urban versus rural sample
and the confounding of location with ethnicity, additional analyses are not
appropriate (thus, although they were calculated, they are not reported here,
lest they be misunderstood as usable and appropriate). For example, if differences were revealed in
urban versus rural responses, it would be impossible to ascertain whether these
differences were caused by urban versus rural factors, ethnic backgrounds, or
type of consumer. To address questions
regarding location in the state and ethnicity, future data collection must
incorporate careful stratified sampling procedures.
A complete listing of the qualitative comments provided by
clinicians and consumers is provided in Appendices One to Six, categorized by
instrument and site. Following is a
summary of the most commonly expressed concerns, provided separately for each
questionnaire. This summary should not
be used in place of reading the comments.
It is evident that the clinicians and consumers put a lot of thought
into their comments and a careful review of the comments is crucial to the
credibility of any additional instrument revision or replacement effort. Not surprisingly, given that all clinicians
and many consumers had encountered the questionnaires previously, most consumer and
clinician responses to the follow-up questions about the questionnaires
themselves were brief, and significantly fewer responses were obtained than in
the Preliminary Phase.
Overall Comments
According to clinicians, consumers were willing to complete
the questionnaires in the structured interview format and most who were asked
to participate did so voluntarily. One
consumer who declined to participate did so because of ‘too much asking and too
much paperwork’ (as recorded by the clinician). One rural agency reported that three consumers declined to
participate. One clinician offered this
general comment about all of the questionnaires: ‘Many of the participants requested help with the forms, which
were created with the intention for independent completion from the consumers. Keeping this in mind, I think it is important
to plan the wording carefully to encompass the wide range of education barriers
that many people struggle with.’ A
number of clinicians noted that consumers needed clarification on many
questions. At one rural agency,
clinicians noted that about one-half of their consumers needed help completing
the Demographic Questionnaire and MHSIP.
They also noted that most consumers did not want to complete the forms
while at the agency, preferring to take and mail them later. Also of note were concerns expressed by
consumers about inadequate staffing, overworked staff, and Medicaid funding
cuts that have decreased services and make filling out additional forms a
significant hardship.
Client Status Review:
Overall Comments
Appendix One provides clinician and consumer comments in
response to the CSR. Several consumers
expressed positive opinions about the process:
‘I enjoy the questions. I
support this questionnaire. Good
job.’ However, a number of consumers
also expressed negative comments about the CSR, such as ‘I didn’t like
intrusive questions; I don’t think these questions are necessary; don’t like
nosey people.’ One consumer
commented that ‘the questions assuming substance abuse and problems with the
law are offensive.’
A significant number of
clinicians and consumers commented that the CSR was not appropriate for
children and youth, and recommended developing a separate questionnaire that is
more applicable to a younger consumer population. Comments in
this regard included ‘doesn’t apply to consumer’s age’, ‘none applied’, ‘not
worded for parents to answer for a child’, and ‘should make a form for the
parent specifically to answer for the child.’
Several clinicians noted having to reword questions, using consumers’
names, since they were requesting the information from the consumers’
parents. The overall recommendation
from consumers and clinicians alike was to develop a separate questionnaire for
child and youth consumers structured in such a manner as to be completed by a
parent or primary caregiver.
One rural provider commented,
representatively so, in conversation that rural services differ significantly
from urban services in that they tend to be much more short-term (one or two
sessions), crisis-focused, or intensive in nature. Systemic interventions at the family and community level are more
common and will not be captured by the current CSR. This provider suggested alternative means for assessing the
success of rural mental health programs, not focusing on individual clients,
but focusing on community perceptions and use of the local mental health
agency.
The majority of consumers reported
that the CSR questions were neither hard to answer (64.2%), nor unclear
(68.9%). One consumer suggested adding
the question, ‘What do you
feel has
been helping the most?’ Another
suggested asking, ‘Is treatment confidential (private)?’, while noting the lack
of privacy in a case manager’s office.
Another consumer indicated a ‘Need to ask if [services are] adequate
after hospitalization.’ Clinician
comments included adding questions about client satisfaction and complaints
about medications and services, treatment results over time, and lack of
improvement and its reasons. One
clinician indicated that the questionnaire should have asked more about mental
health. Another clinician asked, ‘What
are you attempting to determine?’
Specific CSR Item Comments
Following is a summary of comments made regarding specific
CSR items. The reader again is urged to
read the detailed comments in the Appendix and not to rely only upon the
summaries.
Comments about CSR Question One: On Question 1 (regarding activity), one consumer commented ‘If
you weren’t doing one of those things, you’d be dead.’ Along the same line, one clinician noted
that the response ‘not active’ makes it appear ‘as if the consumer never does
any activity’, suggesting ‘It may be
better to state a choice such as rarely active, minimally active, etc.’ Another
asked ‘level of activity, how defined for a home-schooled child?’ Another consumer described the question as
‘nebulous.’ There were several
clinician comments that adding agency group activities should be part of
meaningful activity since this was ‘an active area’ for many consumers.
Comments about CSR Question Two: Several consumers commented that Question 2
(regarding physical health problems) was difficult to answer or unclear due to
the wording and being ‘unsure about physical.’
One consumer attributed weight problems that negatively affected her
mental health as the factor that kept her from doing normal activities.
Comments about CSR Question
Three: For Question 3 (regarding
physical health care access), a number of consumers noted services that they
needed, such as eye and/or dental care.
One consumer wrote ‘I need dental care and just filled out a grant
through the dental program.’
Comments about CSR Question Four: Numerous comments about Question 4
(regarding payment for health care) indicated that consumers had difficulty
choosing between ‘fully insured’ or ‘well-insured’ because Medicaid and
Medicare were not identified and do not cover all services. Other consumer comments included ‘confusing;
Medicaid & Medicare needed to be specified.’ One consumer asked ‘Is health care the same as mental health
care?’
Comments about CSR Question Five: There were several comments about including
crisis respite services on Question 5 (regarding hospitalization for mental
health care) since consumers had used this type of service with the past six
months.
Comments about CSR Question Six: Question 6 (regarding drug and alcohol use)
received the most criticism, with a recurrent theme from both clinicians and
consumers being that the question presumed a substance use problem. One consumer captured the predominant
feeling about this question: ‘I don’t
drink and you should have asked if I did before you implied that I needed to
answer these questions.’ Other consumer
comments included: ‘I felt it was intrusive; assumes I have drug/alcohol
problem; where to indicate no longer using? You assume I’m alcoholic; I have 3
years clean and sober and nowhere to note it; Don’t drink!’ Clinician comments
echoed those of consumers: ‘#6
continues to be a poorly worded question as many of those who participated do
not use drugs or alcohol and felt this question assumed that they did;
Consumers find it difficult to differentiate whether the question is pertaining
to past or present use; Need to ask whether client drinks/drugs.’ There also seemed to be some confusion on
the part of consumers about how to answer the questions. One clinician commented that a consumer had
answered ‘yes’ to the question about feeling guilty because of one relapse with
alcohol use.
Another
consumer answered ‘yes’ to ‘are you annoyed by friends or relatives who question
your use?’ and wrote in ‘annoyed by other people pressuring (me) to use.’ There were also a number of comments
wondering if caffeine and tobacco were considered drugs. One clinician noted
that response option one had ‘weird phrasing – double negative.’
Comments about CSR Question
Seven: On Question 7 (regarding
financial situation), one parent indicated that the child has Medicaid, but the
parent herself or himself faces ‘extreme financial hardship.’ One clinician reported that a consumer had
problems with this question, stating ‘Kids questioned if this relates to their
finances or their parents’?’
Comments about CSR Question
Eight: On Question 8 (regarding
housing), several consumers wrote in ‘pay rent’ as a clarification to ‘live
with others.’ One consumer was unsure
about being independent or not, commenting ‘I’m very dependent on various
agencies.’ One clinician stated that
the question ‘continues to not address transitional housing’ and asked ‘Does this
fit into the homeless category?’ A clinician
commented that ‘the parents own; the kids live with their parents. Question
doesn’t make sense.’
Comments about CSR Question Nine: For Question 9 (regarding general safety),
consumers’ comments indicated that mental health issues, rather than environmental
factors, primarily determined their feelings of safety. One parent added an explanation to response
option three to indicate that the child is fearful of being teased or picked
on. Several consumers noted feeling
fearful or paranoid as a result of their symptoms or their own actions,
acknowledging that there were no identifiable external threats. Several clinicians noted that consumers felt
safe and were aware of suffering from paranoia. Only one consumer referred to environmental factors, stating
‘It’s hard to feel safe when hearing about community violence that may not
directly affect me.’
Comments about CSR Question Ten: One clinician had several
comments for Question 10 (regarding involvement with police, court, jail) about
the redundancy of ‘legal issues pending’ and ‘felony charges pending’, and the
need to clarify terms, such as ‘extreme impact’ and ‘non-lock-up
facility.’ Another clinician commented
that ‘non-criminal involvement’ was unclear terminology. One clinician added the comment ‘unknown, if
any’ regarding the consumer’s legal involvement.
Demographic
Questionnaire (Adult, Child and Families)
Of the consumers who completed a Demographic Questionnaire
and MHSIP, 95.5% returned them directly to ACSES via mail. The vast majority of consumers (60.1%)
reported that they understood the questions and that they were clear
(61.5%). Only two consumers stated that
the questionnaire was too long.
Appendix Two provides comments on the adult version of the
Demographics Questionnaire; Appendix Three provides comments on the youth
version. Many consumers took the opportunity to express appreciation for staff
and services they receive, as well as to identify some problem areas. Comments from a number of consumers
suggested the need to ask open-ended questions regarding how they felt about
the quality of the services.
Suggestions were as follows:
·
Do
you need more help?
·
Are
you happy with your counselor or doctor?
·
Are
you happy with the programs?
·
How
are the meds working?
·
If
you don’t like your counselor, what are your options?
·
Can
you choose providers?
·
·
Does
the system adequately help you completely control your mental illness?
One consumer wondered if “… this [is] a test of our
competence?’ Another consumer stated
‘the purpose isn’t clear…’ A few consumers noted that the questionnaire did not
ask everything that they thought was important about mental health services and
that questions should be added, although they did not elaborate.
Two respondents completed one
questionnaire each for two children receiving services (completed by biological
parent and adoptive parent for one child; completed by a biological parent and
a foster parent for the other child).
On the question asking about guardian, conservator, or payee, several
consumers wrote in the name of the person, instead of selecting the job title
of the person.
Several consumers selected or wrote
in more than one response for the question about ethnicity. When this occurred, the minority ethnicity
was recorded in the data tables. One
clinician recommended using the new United States Census Bureau categories for
the ethnic group.
For the question regarding services,
one clinician noted that ‘housing, transportation, employment and advocacy
services are not applicable’ in rural location. Several clinicians noted that consumers did not know what
‘advocacy’ meant. Some consumers were
unclear that group therapy or group counseling were included in the
‘therapy/counseling’ choice, as indicated by their addition of the name of
specific treatment groups in the ‘Other’ category. A couple of consumers selected ‘Other’ and wrote in comments,
such as ‘new client’ or ‘just started’.
The question concerning frequency of
services received the largest number of write-in consumer responses, suggesting
the need to more clearly define the answer options. One consumer selected two responses: ‘daily’ and wrote
‘occasionally’ in the ‘Other’ category.
Consumers wrote a variety of responses in the ‘Other’ category (e.g.,
‘for years’, ‘varies’, ‘3-5 days/week’, ‘3x a week’, ‘4x a week’, ‘rarely’, ‘5
days/week’). One consumer selected
three responses, commenting that ‘some services are monthly, others may be
weekly – to lump them together makes it more difficult to answer.’
MHSIP (Adult)
Over three-fourths of the consumers found the questions
understandable and clear (77.4%) and inclusive of issues that they thought were
important to ask about mental health services (61.9%). Appendix Four provides the comments provided
by consumers in response to the adult version of the MHSIP. A few consumers responded that the
questionnaire did not ask everything that is important about mental health
services, but did not elaborate on what was missing. Of note were consumer comments regarding the need to ask about
other agency staff and about billing techniques. Only three respondents felt that the questionnaire was too
long. One consumer asked ‘Will I get
the outcome of this survey?’ and wondered ‘if this survey helps in
funding.’ Another consumer suggested
adding a question about ‘What programs or areas of your life would you like
more help in…?’ Another consumer
commented ‘This questionnaire is a good tool if your ears and hearts are
open and compassionate.’ A clinician
reported continuing to have to provide assistance in understanding the
questions and clarifying the wording to simplify questions to many of the
consumers. The clinician reported that
‘many of the participants requested assistance with the Demographic and MHSIP
survey questionnaires which were intended for independent completion from the
consumers. Keeping this in mind, I
think it is important to plan the wording carefully to encompass the wide range
of education barriers that many people struggle with so that, in the event this
form is completed independently, it will be easy to follow and answer.’ One clinician reported that many of the
consumers read the word
‘options’
in Question 2 as ‘opinion’ and needed assistance with the question. The
suggestion was to change the wording of the question to ‘choice’ or
‘preference.’ Several consumers noted
that Question 17 was difficult to understand and unclear, with one consumer
commenting that ‘both therapist and I decided treatment goals’ and another
‘difficult to know if you want the end results or the actual
therapy/participation.’ Question 19 was
also identified as problematic for several consumers, but no elaboration was
provided.
MHSIP (Youth and Families)
The majority of respondents (70.6%) thought the
questionnaires were understandable, clear, and inclusive of issues that were
important to ask about mental health services.
None reported that the questionnaires were too long. Appendix Five provides consumers’ comments
in response to the Youth Services Survey for Families; Appendix Six provides
consumers comments in response to the Youth Services Survey. However, few comments were provided by
consumers. One consumer suggested
adding the question ‘what kind of services do you feel were lacking?’ One consumer noted the ‘high turnover in
manpower’ and having been ‘treated disparately’ at an agency. One youth consumer selected N/A for 18 of
the 26 responses. Another youth
consumer selected two responses for the question asking if staff were sensitive
to cultural/ethnic background, writing in the comment ‘1 person’ for Agree and
‘all others’ for Disagree.
Two sets of recommendations will be offered. The first set focuses on recommendations for revising the instruments; the second set focuses on recommendations for the next phase of pilot testing.
Assuming that DMHDD and the AMHB wish to revise the CSR,
Demographic Questionnaires, and the MHSIPs to incorporate Primary Phase
feedback from consumers and clinicians, a number of recommendations follow for
such revisions. These recommendations were developed based upon consumers’ and
clinicians’ feedback.
Client Status Review
·
Develop
a separate questionnaire that is age-appropriate for child/youth consumers
·
Question
1: Answer 5: change to ‘Rarely active’
·
Question
2: Answer 1: change to ‘Rarely’
Answer 5: change to
‘Almost always’
·
Question
3: Reword parenthetical description to
read “Does not include dental and eye care, and is defined as care received
from your regular, local, or visiting health care provider”
·
Question
4: Specify in the question whether health care refers to mental or physical
health, or both
Answer 1: change ‘ Fully
insured’ to ‘free’
Answer 2: change ‘Well-insured’
to ‘Good insurance’; add ‘, such as Medicaid/Medicare/insurance co-pay’
to end of parenthetical sentence
·
Questions
6: Given considerable missing data on
this item, and clinician and consumer criticisms, consider a major rewrite of
this item. One possibility is to
rewrite question to be a more general query of substance use/abuse that
highlights the need for further evaluation, rather than using a standardized
assessment tool designed to access degree of SA
·
Question
8:
Answer 1: delete ‘or
I chose to live with others’
Answer 2: delete
‘family or others’
Answer 4: change
‘temporary’ to ‘transitional’ and add ‘temporarily’ to ‘staying with
family/friends’
·
Question
10:
Answer 3: change
‘legal issues now pending’ to ‘misdemeanor changes pending or conviction’
Answer 4: after
‘pending’, add ‘or conviction’; delete ‘or extreme impact’; change ‘contested
divorce, contested custody issues’ to ‘contested divorce and/or custody issues’
Answer 5: change
‘non-lock-up facility’ to ‘half-way house’; add ‘Title 12’ before ‘mandatory’
·
For
all versions, incorporate the questions at the end of the MHSIP
·
Question
2a: Change ‘If yes,’ to check boxes for
guardian, payee, conservator
·
Question
3:
Change stem to read:
‘Which one of …’
Use new US census ethnic categories
·
Question
5, answer 1 – add ‘(includes individual & group)’
·
Question
8, answer 1 – after ‘Daily’, add ‘(5-7 days/week)’
Demographic Questionnaire (Child
& Family Services)
·
Completed
by: change ‘biological parent’ to
‘biological/adoptive parent’
·
Question
2: change stem to read ‘Which one of …’
·
Question
6: after stem, add ‘(Please select only
one.)’
·
Question
7, answer 1 – add ‘(includes individual & group)’
·
Question
10, answer 1 – after ‘Daily’, add ‘(5-7 days/week)’
Adult MHSIP
·
Place
demographic questions at end of MHSIP questions, similar to that of Youth &
Family MHSIPs
·
Include
open-ended questions in the Adult version:
What has been the most helpful thing about the services you
have received over the last 6 months?
What would improve services here?
Youth MHSIP
·
Use
the revised 21-item versions of the Youth Services Survey and Youth Services
Survey for Families
Recommendations for the Next
Phase of Pilot Work
For any future investigations into the utility and
psychometric characteristics of these instruments, we make the following
recommendations:
1)
The
Oversight Committee may be interested in gathering feedback from staff members
at mental health agencies around the state via focus group and key informant
interview formats to identify problem areas and possible solutions that could
help further refine the performance measures instruments for youth and rural
consumers. It appears that rural
providers perceive the instruments as less applicable to their clientele than urban
providers, which has likely contributed to the resistance by rural providers to
participating in this pilot project and may also lead to resistance to using
the measures in future. The suggestion
of rural providers to develop alternate means of measuring outcomes in rural
areas in this regard appears particularly pertinent.
2)
Using
the same focus group and key informant interview formats suggested in item 1),
it would be helpful to gather feedback from child and youth mental health care
providers before beginning construction of a child/youth version of the
CSR. Merely adapting the current CSR
downward in age may not fully meet the needs and concerns of these providers,
as completely different topics may need to be assessed to measure outcomes
among a younger consumer population.
3)
To
gain further insight into the current questionnaires and any newly developed
questionnaires (such as a child or rural version), administration of
instruments should be repeated at a different set of agencies (or at different
programs within agencies that participated this time). This re-testing phase is particularly
important for determining the utility of new versions (youth, rural) in rural
areas, with ethnically diverse populations, and with youth and adolescent
populations. Stratified sampling (by
primary diagnosis, ethnicity, gender, and rural/urban residence) may be
necessary to receive definitive answers about the instruments’ utility across
geographic locations, ethnic backgrounds, and types of clientele. Such stratified sampling would require
extreme cooperation by agencies and may be difficult to realize.
4)
This
phase ideally would allow sufficient time to include a trial of reporting
collected data back to clinical staff to let them see what types of summary
reports can be expected based on the data collection. Such feedback would serve two purposes. First, it would demonstrate to clinicians that useful data comes
back to the agency and that data do not just disappear into a “black
hole.” Second, such reporting back
would allow for double-checking whether the data are indeed useful in meeting
their purpose of improving services.
Clinicians could look over the data report to ascertain if the data
would, in fact, help them improve the services they are currently providing to
their consumers.
For future investigations into the statewide
implementation of outcome measures, we make the following recommendations:
6) It appears necessary to collect more input from direct service provider about possible barriers, resistances, and solutions to the actual process of gathering and reporting outcome data (independent of what instruments are being used). It appears that rural versus urban staff members had different concerns and that different procedures (and perhaps even instruments) are needed to accommodate rural versus urban settings. To gather this input, agency staff and consumers could meet in (separate) focus groups to answer specific questions about the process of gathering and using outcome data. These focus groups could be combined with the focus groups mentioned in item 1) above.
7) Given concerns and the lack of clarity expressed by clinicians and consumers about the purpose of these instruments, it may help to use the terminology of “outcome” rather than “performance” measures.
|
AGENCY (# of packets provided) |
TIME ONE |
TIME TWO |
||||||||||||
|
Client
Status Review |
Demographic Questionnaire |
MHSIP |
Client Status Review |
Demographic Questionnaire |
MHSIP |
|||||||||
|
Adult |
Youth |
Family |
Adult |
Youth |
Family |
Adult |
Youth |
Family |
Adult |
Youth |
Family |
|||
|
Bristol Bay (105) 75 Adult 15 each Youth & Family |
17 (11)1,2 |
11 (7)1,2 |
-- |
-- |
11 (7)1,2 |
-- |
-- |
5 |
3 |
-- |
-- |
3 |
-- |
-- |
|
LifeQuest (575) 225 Adult 175 each Youth & Family |
58 |
23 |
0 |
15 |
23 |
6 |
9 |
14 |
3 |
1 |
0 |
3 |
1 |
0 |
|
Alternatives (225) 25 Adult (18-21 only) 100 each Youth & Family |
6 (3)1,2 |
0 (1)1,2 |
0 |
3 |
0 (1)1,2 |
0 |
3 |
1 |
0 |
0 |
0 |
0 |
0 |
0 |
|
Norton Sound (100) Adult |
5 (7)1,2 |
7 (6)1,2 |
0 (1)1,2 |
0 |
7 (6)1,2 |
0 (1)1,2 |
|
1 |
0 |
-- |
-- |
0 |
-- |
-- |
|
SCC (84) Adult |
64 |
57 |
|
-- |
57 |
-- |
-- |
48 |
47 |
-- |
-- |
47 |
-- |
-- |
|
TOTALS |
150 (21) 171 |
112 |
1 |
18 |
112 |
7 |
12 |
69 |
53 |
1 |
0 |
53 |
1 |
0 |
|
121 |
121 |
54 |
54 |
|||||||||||
Notes: 1)
Additional Time One administration only
2) Data
not included in Time Two
TABLE TWO
Demographic
Questionnaire (Adult)
|
1. How old are you? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Less than 18 |
18-22 |
23-59 |
60 or older |
No response |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Time
1 |
Time
2 |
Time 1 |
Time
2 |
Time
1 |
Time
2 |
Time
1 |
Time
2 |
Time
1 |
Time
2 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||
|
0 |
0 |
8 7.1% |
3 5.7% |
94 83.9 |
49 92.5% |
10 8.9 |
1 1.9% |
0 |
0 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||
|
2. Do you
have a guardian, conservator or payee? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Yes
|
No
|
No Response
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
30 26.8% |
22 41.5% |
82
73.2% |
30 56.6% |
0 |
1 1.9% |
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Indicate which one. |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Guardian
|
Conservator
|
Payee
|
Other
|
N/A or Don’t know |
No Response
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
3
2.7% |
0 |
4
3.6% |
3 5.7% |
14
12.5% |
14 26.4% |
4 3.6% |
1 1.9% |
3 2.7% |
30 (N/A) 56.6% |
2
1.8% |
5 9.4% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
3.
Which of the following best describes the race or ethnic group that
you consider yourself?
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
African American |
Asian/Pacific
Islander |
Hispanic/ Latino |
Alaska Native/ American Indian |
Caucasian |
Don’t know |
No Response |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
5 4.5% |
4 7.5% |
2 1.8% |
2 3.8% |
3 2.7% |
0 |
40 35.7% |
11 20.8% |
58 51.8% |
35 66% |
1 0.9% |
0 |
3 2.7% |
1 1.9% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
4.
Gender |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Female |
Male |
No Response |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
51 45.5% |
20 37.7% |
46 41.1% |
29 54.7% |
15 13.4% |
4 7.5% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
5.
Which of the following services have you received in the last year
from <agency>? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Therapy/
Counseling
|
Housing
|
Family Services |
Transportation Services |
Psych Svcs/Meds
|
Case Management |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
70 62.5% |
43 81.1% |
22 19.6% |
5 9.4% |
5 4.5% |
4 7.5% |
38 33.9% |
19 35.8% |
86 76.8% |
47 88.7% |
75 67.0% |
47 88.7% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||
|
Employment |
SA
Tx |
Advocacy |
Other |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3 2.7% |
7 13.2% |
13 11.6% |
8 15.1% |
14 12.5% |
10 18.9% |
12 10.7% |
4 7.5% |
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
6.
Are you still receiving services from <agency>? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yes |
No |
No Response |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
101 90.2% |
50 94.3% |
6 5.4% |
1 1.9% |
5 4.5% |
2 3.8% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
7.
How long have you been receiving, or did you receive, services from
<agency>? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Less than 6 mos. |
6 mos. – 2 years |
2-5 years |
More than 5 years |
No Response |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
22 19.6% |
2 3.8% |
21 18.8% |
11 20.8% |
24 21.4% |
13 24.5% |
43 38.4% |
26 49.1% |
2 1.8% |
1 1.9% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Note: Gray boxes provide Time One data; white
boxes provide Time Two data |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
8. About how often do (or did) you receive services from <agency>? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daily |
Once/week |
Twice/week |
Once/month |
Twice a month |
Every other month |
Every 3 months |
Other |
No Response |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
21 18.8% |
12 22.6% |
27 24.1% |
13 24.5% |
20 17.9% |
15 28.3% |
10 8.9% |
2 3.8% |
13 11.6% |
5 9.4% |
1 0.9% |
0 |
2 1.8% |
0 |
13 11.6% |
5 9.4% |
5 4.5% |
1 1.9% |
||||||||||||||||||||||||||||||||||||||||||||||
|
9.
Comments. Is there anything
else that you would like to tell us about yourself or the services that you
receive that might help us to better serve you? (See Appendix Two) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
No |
Written Comments |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
98 57.3% |
37 69.8% |
73 42.7% |
16 30.2% |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Note: Gray boxes provide Time One data; white boxes provide
Time Two data |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
TABLE THREE
Demographic
Questionnaire (Child & Family Services)
|
Completed by: |
|||||||||
|
Biological Parent |
Self |
Foster Parent |
Guardian |
Other |
|||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
|
7 36.8% |
0 |
5 26.3% |
1 100% |
2 10.5% |
0 |
2 10.5% |
0 |
3 15.8% |
0 |
|
1. How old are you (or the child receiving services)? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
6-12 |
13-17 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
12 63.2% |
0 |
7 36.8% |
1 100% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
2. Which of the following best describes the
race/ethnicity that you consider yourself (or the child receiving services) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
African
American |
Asian/Pacific Islander |
Hispanic/ Latino |
Alaska Native/ Amer. Indian |
Caucasian
|
Other |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
1 5.3% |
0 |
0 |
0 |
0 |
0 |
3 15.8% |
0 |
13 68.4% |
1 100% |
2 10.5% |
0 |
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3.
Gender |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Female |
Male |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3 15.8% |
0 |
16 84.2% |
1 100% |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
4. What is
your school situation (or that of the child receiving services)? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
In School
|
Not
in school
|
Working on GED |
Other |
Grade |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
1-6 |
7-8 |
9-12 |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
19 100% |
1 100% |
0 |
0 |
0 |
0 |
0 |
0 |
11 57.9% |
0 |
3 15.8% |
0 |
5 26.3% |
1 100% |
|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
5.
Are you (or the child receiving services) in protective custody? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yes |
No |
No Response |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3 15.8% |
0 |
15 78.9% |
1 100% |
1 5.3% |
0 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
If yes, please select one of
the following and indicate for how long. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
DFYS |
DJJ |
Parent-Guardian |
Other |
N/A or No Response |
Length of time |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
7 mos. |
No Response |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
2 10.5% |
0 |
1 5.3% |
0 |
0 |
0 |
0 |
0 |
16 84.2% |
0 |
2 10.5% |
0 |
2 10.5% |
0
|
||||||||||||||||||||||||||||||||||||||||||||||||||
|
6. Who
referred you (or the child receiving services) to <agency> for mental
health services? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Self-referred
|
Parent/Guardian |
DFYS |
DJJ |
School |
Other |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3 15.8% |
0 |
4 21.1% |
1 100% |
5 26.3% |
0 |
2 10.5% |
0 |
2 10.5% |
0 |
3 15.8% |
0 |
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Note: Gray boxes provide Time One data; white boxes
provide Time Two data |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
7. Which of the following services have you
(or the child receiving services) received in the last year from
<agency>? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Therapy/ Counseling |
Housing |
Family Services |
Transportation Services |
Psych Svcs/Meds |
Case Management |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
15 78.9% |
1 100% |
0 |
0 |
6 31.6% |
0 |
8 42.1% |
0 |
5 26.3% |
0 |
14 73.7% |
1 100% |
||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Employment |
SA Tx |
Advocacy Service |
TX Foster Care |
Other |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
0 |
0 |
0 |
0 |
3 15.8% |
0 |
1 5.3% |
1 100% |
4 21.2% |
0 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
8.
Are you (or the child receiving services) still receiving services
from <agency>? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Yes |
No |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
19 100% |
1 100% |
0 |
0 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
9. How long have you (or the child receiving
services) been receiving, or did you receive, services from <agency>? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Less than 6 mos. |
6 mos. – 2 years |
2-5 years |
More than 5
years
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
3 15.8% |
0 |
11 57.9% |
0 |
3 15.8% |
1 100% |
2 10.5% |
0 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
10. About how often do (or did) you (or the child receiving services) receive services from <agency>? |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Daily |
Once/week |
Twice a week |
Once a month |
Twice a month |
Every
other month |
Every 3 months |
Other |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
0 |
1 100% |
6 31.6% |
0 |
9 47.4% |
0 |
1 5.3% |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
3 15.8% |
0 |
||||||||||||||||||||||||||||||||||||||||||||||||
|
11. Comments. Is there
anything else that you would like to tell us about yourself (or the child
receiving services), or the services that you receive that might help us to
better serve you? (See Appendix Three) |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
No |
Written Comments |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
13 68.4% |
1 100% |
6 31.6% |
0 |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Note: Gray boxes provide Time One data; white boxes provide Time Two
data
TABLE FOUR
|
Item Number |
Time 1 N=171 |
Time 2 N=69 |
|
1 |
0 |
1 |
|
2 |
1 |
1 |
|
3 |
0 |
1 |
|
4 |
1 |
1 |
|
5 |
0 |
1 |
|
6a |
11 |
5 |
|
6b |
13 |
5 |
|
6c |
13 |
6 |
|
6d |
15 |
5 |
|
6e |
13 |
3 |
|
7 |
2 |
1 |
|
8 |
1 |
1 |
|
9 |
1 |
1 |
|
10 |
1 |
1 |
TABLE FIVE
(Based
on 171 protocols obtained at Time One and 69 protocols at Time Two)
|
1. How often
are you currently involved in activities that are important to you? |
||||||||||||||||||||||||||||||||||||||
|
Usually everyday |
2-5 days/week |
5-10 days/month |
1-4 days/month |
Not Active |
No Response |
|||||||||||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
|||||||||||||||||||||||||||
|
49 28.8% |
20 29% |
64 37.6% |
19 27.5% |
17 10% |
7 10.1% |
22 12.9% |
13 18.8% |
18 10.6% |
9 13% |
1 0.6% |
1 1.4% |
|||||||||||||||||||||||||||
|
2. How
often do physical health problems currently keep you from doing your normal
activities? |
||||||||||||||||||||||||||||||||||||||
|
Never |
Occasionally |
Often |
Regularly |
Always |
No Response |
|||||||||||||||||||||||||||||||||
|
55 32.2% |
18 26.1% |
56 32.7% |
24 34.8% |
25 14.6% |
10 14.5% |
23 13.5% |
12 17.4% |
12 7% |
4 5.8% |
0 |
1 1.4% |
|||||||||||||||||||||||||||
|
3.
How often can you currently get the physical health care that
you need? |
||||||||||||||||||||||||||||||||||||||
|
All of the time |
Most of time |
Sometimes |
Limited |
Almost Never |
No Response |
|||||||||||||||||||||||||||||||||
|
85 49.7% |
34 49.3% |
45 26.3% |
21 30.4% |
20 11.7% |
10 14.5% |
14 8.2% |
2 2.9% |
6 3.5% |
1 1.4% |
1 0.6% |
1 1.4% |
|||||||||||||||||||||||||||
|
4.
How is your health care paid for? |
||||||||||||||||||||||||||||||||||||||
|
Fully insured |
Well-insured |
Some ins. |
Very little ins. |
No insurance |
No Response |
|||||||||||||||||||||||||||||||||
|
90 52.9% |
26 38.2% |
61 35.9% |
39 57.4% |
6 3.5% |
0 |
3 1.8% |
1 1.5% |
10 5.9% |
2 2.9% |
1 0.6% |
1 1.4% |
|||||||||||||||||||||||||||
|
5.
During the last 6 months, how many times
have you been hospitalized for mental health treatment? (Indicate length of
stay for each hospitalization, if available.) |
||||||||||||||||||||||||||||||||||||||
|
None |
1 time |
2 times |
3 times |
4 + times |
No Response |
|||||||||||||||||||||||||||||||||
|
145 84.8% |
55 80.9% |
20 11.7% |
10 14.7% |
4 2.3% |
0 |
0 |
2 2.9% |
2 1.2% |
1 1.5% |
0 |
1 1.4% |
|||||||||||||||||||||||||||
|
6a. Have friends or relatives asked you to Cut
down on alcohol, tobacco or other drugs, or to quit entirely? |
||||||||||||||||||||||||||||||||||||||
|
Yes |
No |
No Response |
N/A (added by
consumer) |
|||||||||||||||||||||||||||||||||||
|
40 25.2% |
14 20.6% |
119 74.8% |
51 75% |
11 6.4% |
4 5.8% |
1 0.6% |
|
|||||||||||||||||||||||||||||||
|
6b. Are
you Annoyed by friends or relatives who question your use? |
||||||||||||||||||||||||||||||||||||||
|
Yes |
No |
No Response |
N/A (added by
consumer) |
|||||||||||||||||||||||||||||||||||
|
27 17.2% |
10 14.7% |
130 82.8% |
53 77.9% |
13 7.6% |
6 8.3% |
1 0.6% |
0 |
|
||||||||||||||||||||||||||||||
|
6c. Have
you experienced Guilt because of your drinking or use? |
||||||||||||||||||||||||||||||||||||||
|
Yes |
No |
No Response |
N/A (added by
consumers) |
|||||||||||||||||||||||||||||||||||
|
31 20.1% |
11 16.2% |
123 79.9% |
52 76.5% |
16 19.4% |
6 8.7% |
1 0.6% |
0 |
|
||||||||||||||||||||||||||||||
Note: Gray boxes provide Time One data; white boxes
provide Time Two data
|
6d. Do you need
an Eye opener in the morning (alcohol or other drugs) to get started? |
||||||||||||||||||||||||||
|
Yes |
No |
No Response |
N/A (added by
consumers) |
|||||||||||||||||||||||
|
9 5.9% |
1 1.5% |
143 94.1% |
62 91.2% |
18 10.5% |
6 8.7% |
1 0.6% |
0 |
|
||||||||||||||||||
Number of ‘Yes’ answers
|
||||||||||||||||||||||||||
|
Never used/ none |
Yes to one |
Yes to two |
Yes to three |
Yes to four |
No Response |
N/A |
||||||||||||||||||||
|
110 69.2% |
50 73.5% |
21 13.2% |
5 7.4% |
13 8.2% |
8 11.8% |
9 5.7% |
3 4.4% |
6 3.8% |
1 1.5% |
11 6.4% |
1 1.5% |
1 0.6% |
0 |
|||||||||||||
|
7.
Which of the following best describes your
current financial situation? |
||||||||||||||||||||||||||
|
Very
secure financially |
Financially secure most of the time |
Somewhat financially secure |
Many financial
problems |
Extreme financial hardship |
No Response |
|||||||||||||||||||||
|
36 21.3% |
8 11.8% |
37 21.9% |
21 30.9% |
42 24.9% |
28 41.2% |
38 22.5% |
10 14.7% |
16 9.5% |
1 1.4% |
2 1.2% |
1 1.4% |
|||||||||||||||
|
8.
Which of the following categories best describes your current housing
situation? |
||||||||||||||||||||||||||
|
Independent |
Live with others
|
Residential/
Sheltered care |
Homeless |
Institutional |
No Response |
|||||||||||||||||||||
|
115 68% |
50 73.5% |
37 21.9% |
10 14.7% |
12 7.1% |
3 4.4% |
5 3% |
5 7.4% |
0 |
0 |
2 1.2 % |
1 1.4% |
|||||||||||||||
|
9.
How would you best describe your current feeling of general safety? |
||||||||||||||||||||||||||
|
Safe all the time |
Safe most of the
time |
Safe sometimes; unsafe other times |
Unsafe most of the time |
Unsafe all the time |
No Response |
|||||||||||||||||||||
|
50 29.6% |
14 20.6% |
74 43.8% |
32 47.1% |
37 21.9% |
19 27.9% |
0 |
0 |
8 4.7% |
3 4.4% |
2 1.2% |
1 1.4% |
|||||||||||||||
|
10.
Which of the following best describes your current involvement with
police, court or jail? |
||||||||||||||||||||||||||
|
None |
Limited |
Moderate |
Significant |
Extreme |
No Response |
|||||||||||||||||||||
|
119 70.4% |
54 79.4% |
18 10.7% |
6 8.8% |
19 11.2% |
1 1.5% |
12 7.1% |
7 10.3% |
1 0.6% |
0 |
2 1.2% |
1 1.4% |
|||||||||||||||
Note: Gray boxes provide Time
One data; white boxes provide Time Two data
TABLE SIX
Mean and Standard Deviation
Changes - Client Status Review
(Based on 67 pairs matched by
ARORA number)
|
Question
|
Time One
|
Time Two
|
Difference |
||||
|
Mean |
SD
|
Mean |
SD |
Mean |
SD |
t |
|
|
1.
How often are you currently
involved in activities that are important to you? |
2.46 |
1.25 |
2.61 |
1.42 |
0.15 |
1.28 |
0.95 |
|
2.
How often do physical health problems keep you from doing normal
activities? |
2.43 |
1.21 |
2.43 |
1.22 |
0 |
0.90 |
0 |
|
3. How often can you currently get the
physical health care that you need? |
2.01 |
1.05 |
1.76 |
0.92 |
-0.25 |
1.22 |
-1.70 |
|
4. How is your health care paid for? |
1.69 |
0.82 |
1.75 |
0.80 |
0.06 |
0.55 |
0.89 |
|
5. During last 6 months, how many times have
you been hospitalized for mental health Tx? |
1.25 |
0.66 |
1.30 |
0.76 |
0.04 |
0.59 |
0.62 |
|
6a. Have friends/relatives asked you to cut
down on alcohol/other drugs, to quit entirely? |
0.30 |
0.46 |
0.22 |
0.42 |
-0.11 |
0.36 |
-2.42 |
|
6b. Are you annoyed by friends/relatives who
question your use? |
0.21 |
0.41 |
0.16 |
0.37 |
-0.05 |
0.38 |
-1.00 |
|
6c. Have you experienced guilt because of your
drinking or use? |
0.22 |
0.41 |
0.18 |
0.39 |
-0.07 |
0.36 |
-1.43 |
|
6d. Do you need an eye opener in the morning
to get started? |
0.09 |
0.29 |
0.02 |
0.13 |
-0.08 |
0.33 |
-1.93 |
|
6e.
Never used or # of ‘yes’ answers |
1.80 |
1.21 |
1.52 |
0.98 |
-0.25 |
1.08 |
-1.84 |
|
7. Which of the following best describes your
current financial situation? |
2.54 |
1.12 |
2.61 |
0.92 |
0.07 |
0.93 |
-0.66 |
|
8. Which of the following best describes your
current housing situation? |
1.43 |
0.82 |
1.45 |
0.89 |
0.01 |
0.71 |
0.17 |
|
9. How would you best describe your current
feeling of general safety? |
2.22 |
0.92 |
2.19 |
0.84 |
-0.03 |
0.87 |
-0.28 |
|
10. Which of following best describes your
current involvement with police, court or jail? |
1.57 |
1.08 |
1.43 |
0.96 |
-0.13 |
0.92 |
-1.20 |
|
Sum of Ratings
|
19.40 |
4.32 |
19.48 |
5.00 |
0.07 |
4.92 |
-0.12 |
Notes: 1) Lower scores indicate higher levels of functioning;
2) Positive Difference Scores indicate worse
functioning; negative differences
indicate better functioning
TABLE SEVEN
Changes in Client Status Review
(based on 67 pairs matched by ARORA number)
|
Question |
No Change |
Decrease |
Increase |
McNemar
c4 |
|
1.
How
often are you currently involved in activities that are important to you? |
35 |
16 |
16 |
0 |
|
2.
How
often do physical health problems currently keep you from doing your normal
activities? |
35 |
18 |
14 |
0.28 |
|
3. How often
can you currently get the physical health care that you need? |
29 |
26 |
12 |
4.45* |
|
4. How is your
health care paid for? |
47 |
8 |
12 |
0.45 |
|
5. During the
last 6 months, how many times have you been hospitalized for mental health
treatment? |
58 |
4 |
5 |
0 |
|
6a. Have
friends/relatives asked you to cut down on alcohol or other drugs or to quit
entirely? |
54 |
12 |
1 |
7.69* |
|
6b. Are you
annoyed by friends/relatives who question your use? |
53 |
11 |
3 |
3.50 |
|
6c. Have you
experienced guilt because of your drinking or use? |
52 |
13 |
2 |
6.67* |
|
6d. Do you need
an eye opener in the morning to get started? |
54 |
12 |
1 |
7.69* |
|
6e. Never used or # of ‘yes’ answers |
44 |
15 |
8 |
1.57 |
|
7. Which of the
following best describes your current financial situation? |
44 |
15 |
8 |
0 |
|
8. Which of the
following best describes your current housing situation? |
56 |
6 |
5 |
0 |
|
9. How would
you best describe your current feeling of general safety? |
32 |
18 |
17 |
0 |
|
10. Which of
the following categories best describes your current involvement with police,
court or jail? |
55 |
8 |
4 |
0.75 |
|
Sum of Ratings |
4 |
32 |
31 |
0 |
Note: Corrected for continuity. * p
< .05.
TABLE EIGHT
MHSIP (Adult)
(Based on 112 surveys
returned at Time One (T1) and 53 at Time Two (T2))
|
1.
I like the services that I receive here. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
No Response |
Mean SD |
||||||||||||||||||||||||||||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
||||||||||||||||||||||||||||||||||||||||||
|
51 45.5% |
28 52.8% |
37 33.0% |
16 30.2% |
16 14.3% |
6 11.3% |
2 1.8% |
1 1.9% |
1 0.9% |
1 1.9% |
2 1.8% |
0 |
3 2.7% |
1 1.9% |
1.74 0.85 |
1.67 0.90 |
||||||||||||||||||||||||||||||||||||||||||
|
2.
If I had other options, I would still get services from this agency. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
42 37.5% |
21 39.6% |
40 35.7% |
14 26.4% |
17 15.2% |
11 20.8% |
7 6.3% |
4 7.5% |
1 0.9% |
1 1.9% |
2 1.8% |
0 |
3 2.7% |
2 3.8% |
1.93 0.95 |
2.02 1.07 |
||||||||||||||||||||||||||||||||||||||||||
|
3.
I would recommend this agency to a friend or family member. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
48 42.9% |
25 47.2% |
36 32.1% |
19 35.8% |
17 15.2% |
3 5.7% |
4 3.6% |
4 7.5% |
1 0.9% |
0 |
2 1.8% |
1 1.9% |
4 3.6% |
1 1.9% |
1.81 0.91 |
1.73 0.90 |
||||||||||||||||||||||||||||||||||||||||||
|
4.
The location of the services was convenient (parking, public
transportation, distance, etc.) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
39 34.8% |
24 45.3% |
48 42.9% |
16 30.2% |
7 6.3% |
8 15.1% |
5 4.5% |
1 1.9% |
7 6.3% |
3 5.7% |
2 1.8% |
0 |
4 3.6% |
1 1.9% |
1.99 1.11 |
1.90 1.11 |
||||||||||||||||||||||||||||||||||||||||||
|
5.
Staff was willing to see me as often as I felt it was necessary. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
45 40.2% |
25 47.2% |
38 33.9% |
19 35.8% |
11 9.8% |
6 11.3% |
9 8% |
2 3.8% |
2 1.8% |
0 |
2 1.8% |
0 |
5 4.5% |
1 1.9% |
1.90 1.02 |
1.71 0.82 |
||||||||||||||||||||||||||||||||||||||||||
|
6.
Staff returned my calls within 24 hours. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
47 42% |
17 32.1% |
38 33.9% |
23 43.4% |
9 8% |
6 11.3% |
5 4.5% |
4 7.5% |
2 1.8% |
1 1.9% |
7 6.3% |
1 1.9% |
4 3.6% |
1 1.9% |
1.78 0.94 |
2.00 0.98 |
||||||||||||||||||||||||||||||||||||||||||
|
7.
Services were available at times that were good for me. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
49 43.8% |
20 37.7% |
44 39.3% |
25 47.2% |
12 10.7% |
6 11.3% |
2 1.8% |
1 1.9% |
2 1.8% |
0 |
1 0.9% |
0 |
2 1.8% |
1 1.9% |
1.75 0.86 |
1.77 0.73 |
||||||||||||||||||||||||||||||||||||||||||
|
8.
I was able to get all the services I thought I needed. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
40 35.7% |
21 39.6% |
40 35.7% |
19 35.8% |
12 10.7% |
7 13.2% |
13 11.6% |
5 9.4% |
2 1.8% |
0 |
1 0.9% |
0 |
4 3.6% |
1 1.9% |
2.04 1.07 |
1.92 0.97 |
||||||||||||||||||||||||||||||||||||||||||
|
9.
I was able to see a psychiatrist when I wanted to. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
33 29.5% |
18 34% |
39 34.8% |
23 43.4% |
16 14.3% |
4 7.5% |
9 8% |
4 7.5% |
4 3.6% |
1 1.9% |
6 5.4% |
1 1.9% |
5 4.5% |
2 3.8% |
2.13 1.09 |
1.94 0.98 |
||||||||||||||||||||||||||||||||||||||||||
|
10.
Staff here believe that I can grow, change, and recover. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
50 44.6% |
26 49.1% |
38 33.9% |
16 30.2% |
15 13.4% |
7 13.2% |
2 1.8% |
1 1.9% |
1 0.9% |
1 1.9% |
3 2.7% |
1 1.9% |
3 2.7% |
1 1.9% |
1.74 0.84 |
1.73 0.92 |
||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
11.
I felt comfortable asking questions about my treatment and medication. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
No Response |
Mean SD |
||||||||||||||||||||||||||||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
||||||||||||||||||||||||||||||||||||||||||
|
40 35.7% |
20 37.7% |
51 45.5% |
26 49.1% |
10 8.9% |
5 9.4% |
4 3.6% |
0 |
1 0.9% |
1 1.9% |
3 2.7% |
0 |
3 2.7% |
1 1.9% |
1.82 0.83 |
1.77 0.78 |
||||||||||||||||||||||||||||||||||||||||||
|
12.
I felt free to complain. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
30 26.8% |
18 34% |
51 45.5% |
24 45.3% |
11 9.8% |
6 11.3% |
8 7.1% |
2 3.8% |
3 2.7% |
1 1.9% |
3 2.7% |
1 1.9% |
6 5.4% |
1 1.9% |
2.06 0.99 |
1.90 0.90 |
||||||||||||||||||||||||||||||||||||||||||
|
13.
I was given information about my rights. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
38 33.9% |
18 34% |
46 41.1% |
21 39.6% |
14 12.5% |
9 17% |
7 6.3% |
2 3.8% |
3 2.7% |
0 |
2 1.8% |
1 1.9% |
2 1.8% |
2 3.8% |
1.99 1.00 |
1.90 0.84 |
||||||||||||||||||||||||||||||||||||||||||
|
14.
Staff encouraged me to take responsibility for how I live my life. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
42 37.5% |
17 32.1% |
48 42.9% |
26 49.1% |
9 8% |
7 13.2% |
4 3.6% |
2 3.8% |
1 0.9% |
0 |
3 2.7% |
0 |
5 4.5% |
1 1.9% |
1.79 0.83 |
1.88 0.78 |
||||||||||||||||||||||||||||||||||||||||||
|
15.
Staff told me what side effects to watch out for. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
28 25% |
18 34% |
48 42.9% |
23 43.4% |
15 13.4% |
7 13.2% |
9 8% |
3 5.7% |
2 1.8% |
1 1.9% |
4 3.6% |
0 |
6 5.4% |
1 1.9% |
2.11 0.97 |
1.96 0.95 |
||||||||||||||||||||||||||||||||||||||||||
|
16.
Staff respected my wishes about who is, and who is not, to be given
information about my treatment. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
46 41.1% |
23 43.4% |
42 37.5% |
20 37.7% |
12 10.7% |
7 13.2% |
3 2.7% |
1 1.9% |
2 1.8% |
0 |
4 3.6% |
0 |
3 2.7% |
2 3.8% |
1.79 0.90 |
1.73 0.78 |
||||||||||||||||||||||||||||||||||||||||||
|
17.
I, not staff, decided my treatment goals. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
24 21.4% |
13 24.5% |
45 40.2% |
18 34% |
18 16.1% |
6 11.3% |
11 9.8% |
10 18.9% |
5 4.5% |
3 5.7% |
6 5.4% |
1 1.9% |
3 2.7% |
2 3.8% |
2.30 1.09 |
2.44 1.25 |
||||||||||||||||||||||||||||||||||||||||||
|
18.
Staff were sensitive to my cultural/ethnic background (race, religion,
language, etc.). |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
33 29.5% |
21 39.6% |
47 42% |
17 32.1% |
15 13.4% |
8 15.1% |
2 1.8% |
3 5.7% |
0 |
0 |
10 8.9% |
2 3.8% |
5 4.5% |
2 3.8% |
1.86 0.75 |
1.86 0.91 |
||||||||||||||||||||||||||||||||||||||||||
|
19.
Staff helped me obtain the information I needed so I could take charge
of managing my illness. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
36 32.1% |
19 35.8% |
42 37.5% |
26 49.1% |
20 17.9% |
4 7.5% |
5 4.5% |
2 3.8% |
2 1.8% |
0 |
4 3.6% |
0 |
3 2.7% |
2 3.8% |
2.00 0.95 |
1.78 0.76 |
||||||||||||||||||||||||||||||||||||||||||
|
20.
I was encouraged to use consumer-run programs (support groups, drop-in
centers, crisis phone line, etc.). |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
28 25% |
22 41.5% |
40 35.7% |
22 41.5% |
22 19.6% |
3 5.7% |
7 6.3% |
3 5.7% |
3 2.7% |
1 1.9% |
9 8% |
1 1.9% |
3 2.7% |
1 1.9% |
2.17 1.02 |
1.80 0.94 |
||||||||||||||||||||||||||||||||||||||||||
Note: Gray
boxes provide Time One data; white boxes provide Time Two data
|
As a direct result of the services I received
here: |
|||||||||||||||||
|
21.
I deal more effectively with daily problems. |
|||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
No Response |
Mean SD |
||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
||
|
27 24.1% |
12 22.6% |
54 48.2% |
24 45.3% |
21 18.8% |
10 18.9% |
5 4.5% |
4 7.5% |
1 0.9% |
0 |
2 1.8% |
0 |
2 1.8% |
3 5.7% |
2.06 0.85 |
2.12 0.87 |
||
|
22.
I am better able to control my life. |
|||||||||||||||||
|
31 27.7% |
12 22.6% |
53 47.3% |
26 49.1% |
15 13.4% |
10 18.9% |
5 4.5% |
2 3.8% |
3 2.7% |
0 |
1 0.9% |
0 |
4 3.6% |
3 5.7% |
2.03 0.94 |
2.04 0.78 |
||
|
23.
I am better able to deal with crisis. |
|||||||||||||||||
|
24 21.4% |
12 22.6% |
53 47.3% |
28 52.8% |
22 19.6% |
3 5.7% |
6 5.4% |
6 11.3% |
1 0.9% |
0 |
1 0.9% |
0 |
5 4.5% |
4 7.5% |
2.12 0.86 |
2.06 0.90 |
||
|
24.
I am getting along better with my family. |
|||||||||||||||||
|
25 22.3% |
9 17% |
45 40.2% |
24 45.3% |
20 17.9% |
11 20.8% |
9 8% |
2 3.8% |
2 1.8% |
1 1.9% |
8 7.1% |
2 3.8% |
3 2.7% |
4 7.5% |
2.19 0.98 |
2.19 0.88 |
||
|
25.
I do better in social situations. |
|||||||||||||||||
|
16 14.3% |
10 18.9% |
51 45.5% |
19 35.8% |
25 22.3% |
8 15.1% |
10 8.9% |
11 20.8% |
2 1.8% |
1 1.9% |
3 2.7% |
1 1.9% |
5 4.5% |
3 5.7% |
2.34 0.92 |
2.47 1.12 |
||
|
26.
I do better in school and/or work. |
|||||||||||||||||
|
9 8% |
5 9.4% |
35 31.3% |
15 28.3% |
25 22.3% |
8 15.1% |
6 5.4% |
3 5.7% |
3 2.7% |
0 |
30 26.8% |
19 35.8% |
4 3.6% |
3 5.7% |
2.47 0.94 |
2.29 0.86 |
||
|
27.
My housing situation has improved. |
|||||||||||||||||
|
18 16.1% |
11 20.8% |
46 41.1% |
25 47.2% |
20 17.9% |
9 17% |
9 8% |
1 1.9% |
2 1.8% |
1 1.9% |
11 9.8% |
3 5.7% |
6 5.4% |
3 5.7% |
2.27 0.95 |
2.06 0.84 |
||
|
28.
My symptoms are not bothering me as much. |
|||||||||||||||||
|
22 19.6% |
12 22.6% |
44 39.3% |
23 43.4% |
24 21.4% |
8 15.1% |
12 10.7% |
5 9.4% |
5 4.5% |
2 3.8% |
1 0.9% |
0 |
4 3.6% |
3 5.7% |
2.38 1.08 |
2.24 1.06 |
||
Note: Gray boxes provide Time One data; white boxes provide
Time Two data
TABLE NINE
Changes in MHSIP – Adult: By question and domain subscales
(based on 42 pairs matched by demographics)
|
Question |
Time One |
Time Two |
Difference |
||||
|
Mean |
SD |
Mean |
SD |
Mean |
SD |
t |
|
|
1. Like
services here |
1.76 |
0.92 |
1.60 |
0.77 |
-0.15 |
0.96 |
-0.97 |
|
2. If had
other options, still get services here |
1.93 |
1.03 |
2.05 |
1.05 |
0.13 |
1.22 |
0.65 |
|
3.
Recommend this agency |
1.75 |
0.93 |
1.68 |
0.82 |
-0.03 |
0.74 |
-0.22 |
|
4.
Location was convenient |
2.18 |
1.26 |
1.98 |
1.18 |
-0.15 |
0.89 |
-1.06 |
|
5. Staff
willing to see me as often as I felt was necessary |
2.03 |
1.12 |
1.71 |
0.86 |
-0.33 |
0.94 |
-2.18* |
|
6. Staff
returned calls w/in 24 hours |
1.87 |
1.10 |
1.93 |
0.93 |
0.11 |
1.09 |
0.60 |
|
7.
Services available at good times |
1.78 |
0.91 |
1.76 |
0.76 |
0 |
0.97 |
0 |
|
8. Got all
services I thought I needed |
2.12 |
1.22 |
1.98 |
1.02 |
-0.12 |
1.05 |
-0.74 |
|
9. Able to
see psychiatrist when I wanted to |
2.18 |
1.25 |
2.02 |
1.04 |
-0.10 |
1.05 |
-0.61 |
|
10. Staff
believe I can grow, change, recover |
1.88 |
0.88 |
1.63 |
0.80 |
-0.18 |
1.05 |
-1.07 |
|
11. Felt
comfortable asking questions about Tx & meds |
1.71 |
0.72 |
1.67 |
0.65 |
-0.02 |
0.85 |
-0.18 |
|
12. I felt
free to complain |
2.10 |
1.01 |
1.90 |
0.89 |
-0.21 |
1.06 |
-1.21 |
|
13. I was
given information about my rights |
1.60 |
0.77 |
1.78 |
0.82 |
-0.18 |
0.76 |
-1.48 |
|
14. Staff
encouraged me to take responsibility |
1.73 |
0.78 |
1.88 |
0.77 |
0.1 |
0.93 |
0.68 |
|
15. Staff
told me about side effects |
1.95 |
0.86 |
0.93 |
0.97 |
-0.05 |
0.95 |
-0.33 |
|
16. Staff
respected wishes about who is, is
not, to be given info |
1.75 |
0.87 |
1.67 |
0.79 |
-0.08 |
0.66 |
-0.72 |
|
17. I, not
staff, decided my Tx goals |
2.53 |
1.18 |
2.40 |
1.33 |
-0.13 |
1.40 |
-0.56 |
|
18. Staff
sensitive to cultural/ethnic background |
1.96 |
0.78 |
1.83 |
0.87 |
-0.11 |
0.89 |
-0.75 |
|
19. Staff
helped me get info to manage illness |
1.95 |
0.89 |
1.76 |
0.80 |
-0.2 |
0.97 |
-1.31 |
|
20.
Encouraged to use consumer programs |
2.15 |
1.01 |
1.73 |
0.84 |
-0.47 |
1.01 |
-2.90** |
|
21. Deal
more effectively with daily problems |
1.98 |
0.72 |
2.15 |
0.91 |
0.23 |
0.90 |
1.60 |
|
22. I am
better able to control my life |
2.05 |
1.00 |
2.07 |
0.82 |
0.08 |
0.92 |
0.52 |
|
23. I am
better able to deal with crisis |
2.05 |
1.05 |
2.15 |
0.92 |
0.22 |
1.00 |
1.31 |
Notes: 1) Lower scores indicate
higher levels of satisfaction
2) Positive differences indicate lower levels
of satisfaction; negative differences indicate higher levels of satisfaction
3) *p < .05; **p < .01
|
Question |
Time One |
Time Two |
Difference |
||||
Mean
|
SD |
Mean |
SD |
Mean |
SD |
t |
|
|
24. I am
getting along better with family |
2.21 |
0.95 |
2.15 |
0.93 |
-0.03 |
0.99 |
-0.17 |
|
25. I do
better in social situations |
2.33 |
0.88 |
2.44 |
1.18 |
0.13 |
1.06 |
0.76 |
|
26. I do
better in school and/or work |
2.80 |
1.03 |
2.28 |
0.94 |
-0.36 |
0.85 |
-2.01 |
|
27. My
housing situation has improved |
2.36 |
1.01 |
2.08 |
0.90 |
-0.31 |
1.01 |
-1.82 |
|
28.
Symptoms are not bothering me as much |
2.44 |
1.23 |
2.24 |
1.11 |
-0.15 |
1.12 |
-0.85 |
|
Domain Subscales |
|
|
|
|
|
|
|
|
Access |
2.01 |
0.84 |
1.89 |
0.72 |
-0.10 |
0.67 |
-0.99 |
|
Quality of Treatment |
1.97 |
0.61 |
1.84 |
0.60 |
-0.14 |
0.53 |
-1.62 |
|
Outcomes |
2.25 |
0.70 |
2.20 |
0.75 |
-0.001 |
0.62 |
-0.01 |
|
General Satisfaction |
1.81 |
0.83 |
1.78 |
0.78 |
-0.02 |
0.80 |
-0.13 |
Notes: 1) Lower scores indicate higher levels of satisfaction
2)
Positive differences indicate lower levels of satisfaction; negative
differences indicate higher levels of
satisfaction
3) *p < .0; **p < .01
TABLE TEN
Changes in MHSIP – Adult: By question and domain subscales
(based on 42 pairs matched by
demographics)
|
Question |
No Change1 |
Higher Satisfaction |
Lower Satisfaction |
McNemar c4 |
|
1.
Like services here |
23 |
13 |
6 |
1.89 |
|
2. If had
other options, still get services here |
20 |
11 |
11 |
0 |
|
3.
Recommend this agency |
24 |
11 |
7 |
0.5 |
|
4.
Location was convenient |
20 |
14 |
8 |
1.14 |
|
5. Staff
willing to see me as often as I felt necessary |
29 |
11 |
2 |
4.92* |
|
6. Staff
returned calls w/in 24 hours |
23 |
8 |
11 |
0.21 |
|
7.
Services available at good times |
24 |
7 |
11 |
0.5 |
|
8. Got all
services I thought I needed |
26 |
10 |
6 |
0.56 |
|
9. Able to
see psychiatrist when I wanted to |
18 |
13 |
11 |
0.04 |
|
10. Staff
believe I can grow, change & recover |
21 |
14 |
7 |
1.71 |
|
11. Felt
comfortable asking questions about Tx & meds |
21 |
11 |
10 |
0 |
|
12. I felt
free to complain |
17 |
17 |
8 |
2.56 |
|
13. I was
given information about my rights |
22 |
14 |
6 |
2.45 |
|
14. Staff
encouraged me to take responsibility |
18 |
10 |
14 |
0.38 |
|
15. Staff
told me what side effects to watch out for |
19 |
13 |
10 |
0.17 |
|
16. Staff
respected wishes about who is, not given info |
23 |
12 |
7 |
0.84 |
|
17. I, not
staff, decided my Tx goals |
19 |
15 |
8 |
1.56 |
|
18. Staff
sensitive to my cultural/ethnic background |
17 |
17 |
8 |
2.56 |
|
19. Staff
helped me obtain info to manage my illness |
19 |
15 |
8 |
1.56 |
|
20.
Encouraged to use consumer-run programs |
19 |
19 |
4 |
8.52** |
|
21. I deal
more effectively with daily problems |
21 |
9 |
12 |
0.19 |
|
22. I am
better able to control my life |
18 |
12 |
12 |
0 |
|
23. I am
better able to deal with crisis |
19 |
12 |
11 |
0 |
|
24. I am
getting along better with my family |
17 |
15 |
10 |
0.64 |
|
25. I do
better in social situations |
16 |
15 |
11 |
0.35 |
|
26. I do
better in school and/or work |
7 |
31 |
4 |
19.31*** |
|
27. My
housing situation has improved |
14 |
21 |
7 |
6.04* |
|
28. My
symptoms are not bothering me as much |
18 |
15 |
9 |
1.04 |
|
Access |
7 |
19 |
16 |
0.11 |
|
Quality |
1 |
22 |
19 |
0.10 |
|
Outcome |
3 |
22 |
17 |
0.41 |
|
Satisfaction |
13 |
17 |
12 |
0.55 |
Notes: Corrected
for continuity. * p <
.05. ** p < .01. *** p <.001
TABLE ELEVEN
MHSIP - Families (YSS-F)
(Based on 12 surveys
returned at Time One and 0 at Time Two)
|
1.
Overall, I am satisfied with the services that my child received. |
||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
Mean SD |
||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
|
|
6 50% |
|
5 41.7% |
|
1 8.3% |
|
0 |
|
0 |
|
0 |
|
1.58 0.67 |
|
|
|
As a result of the services my child
and/or family received: |
||||||||||||||
|
2.
My child is better at handling daily life. |
||||||||||||||
|
3 25% |
|
7 58.3% |
|
2 16.7% |
|
0 |
|
0 |
|
0 |
|
1.92 0.69 |
|
|
|
3. My child gets along better with family members. |
||||||||||||||
|
2 16.7% |
|
8 66.7% |
|
2 16.7% |
|
0 |
|
0 |
|
0 |
|
2.00 0.60 |
|
|
|
4.
My child gets along better with friends and other people. |
||||||||||||||
|
1 8.3% |
|
9 75% |
|
2 16.7% |
|
0 |
|
0 |
|
0 |
|
2.08 0.51 |
|
|
|
5.
My child is doing better in school and/or work. |
||||||||||||||
|
2 16.7% |
|
7 58.3% |
|
1 8.3% |
|
2 16.7% |
|
0 |
|
0 |
|
2.25 0.97 |
|
|
|
6.
My child is better able to cope when things go wrong. |
||||||||||||||
|
1 8.3% |
|
9 75% |
|
2 16.7% |
|
0 |
|
0 |
|
0 |
|
2.08 0.51 |
|
|
|
7.
I am satisfied with our family life right now. |
||||||||||||||
|
2 16.7% |
|
2 16.7% |
|
4 33.3% |
|
3 25% |
|
1 8.3% |
|
0 |
|
2.92 1.24 |
|
|
Note: Lower scores indicate higher levels of satisfaction
|
Feedback about the services my child
and/or family received: |
|||||||||||||||||||||||||||||||||||||||
|
8.
I helped to choose my child’s services. |
|||||||||||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
Mean SD |
|||||||||||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
||||||||||||||||||||||||||
|
4 33.3% |
|
7 58.3% |
|
0 |
|
0 |
|
0 |
|
1 8.3% |
|
1.64 0.50 |
|
||||||||||||||||||||||||||
|
9.
I helped to choose my child’s treatment goals. |
|||||||||||||||||||||||||||||||||||||||
|
7 58.3% |
|
4 33.3% |
|
0 |
|
0 |
|
0 |
|
1 8.3% |
|
1.36 0.50 |
|
||||||||||||||||||||||||||
|
10.
The people helping my child stuck with us no matter what. |
|||||||||||||||||||||||||||||||||||||||
|
6 50% |
|
4 33.3% |
|
2 16.7% |
|
0 |
|
0 |
|
0 |
|
1.67 0.78 |
|
||||||||||||||||||||||||||
|
11.
I felt my child had someone to talk to when he/she was troubled. |
|||||||||||||||||||||||||||||||||||||||
|
5 41.7% |
6 50% |
|
0 |
|
1 8.3% |
|
0 |
|
0 |
|
|
1.67 0.65 |
|
||||||||||||||||||||||||||
|
12.
The people helping my child listened to what he/she had to say. |
|||||||||||||||||||||||||||||||||||||||
|
4 33.3% |
|
7 58.3% |
|
1 8.3% |
|
0 |
|
0 |
|
0 |
|
1.75 0.62 |
|
||||||||||||||||||||||||||
|
13.
I was frequently involved in my child’s treatment. |
|||||||||||||||||||||||||||||||||||||||
|
4 33.3% |
|
7 58.3% |
|
1 8.3% |
|
0 |
|
0 |
|
0 |
|
1.75 0.62 |
|
||||||||||||||||||||||||||
14.
The services my child and/or family received were right for us. |
|||||||||||||||||||||||||||||||||||||||
|
4 33.3% |
|
7 58.3% |
|
1 8.3% |
|
0 |
|
0 |
|
0 |
|
1.75 0.62 |
|
||||||||||||||||||||||||||
|
15.
The location of services was convenient for us. |
|||||||||||||||||||||||||||||||||||||||
|
3 25% |
|
7 58.3% |
|
1 8.3% |
|
1 8.3% |
|
0 |
|
0 |
|
2.00 0.85 |
|
||||||||||||||||||||||||||
|
16.
Services were available at times that were convenient for us. |
|||||||||||||||||||||||||||||||||||||||
|
3 25% |
|
6 50% |
|
2 16.7% |
|
1 8.3% |
|
0 |
|
0 |
|
2.08 0.90 |
|
||||||||||||||||||||||||||
|
17.
If I need services for my child in the future, I would use these
services again. |
|||||||||||||||||||||||||||||||||||||||
|
5 41.7% |
|
4 33.3% |
|
2 16.7% |
|
0 |
|
0 |
|
1 8.3% |
|
1.73 0.79 |
|
||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
Note: Lower scores
indicate higher levels of satisfaction |
|||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||||||||||
|
18.
My family got the help we wanted for my child. |
|||||||||||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
Mean SD |
|||||||||||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
||||||||||||||||||||||||||
|
3 25% |
|
6 50% |
|
2 16.7% |
|
0 |
|
0 |
|
1 8.3% |
|
1.91 0.70 |
|
||||||||||||||||||||||||||
|
19.
My family got as much help as we needed for my child. |
|||||||||||||||||||||||||||||||||||||||
|
5 41.7% |
|
4 33.3% |
|
3 25% |
|
0 |
|
0 |
|
0 |
|
1.83 0.83 |
|
||||||||||||||||||||||||||
|
20. My child and family’s needs
determined my child’s treatment goals. |
|||||||||||||||||||||||||||||||||||||||
|
4 33.3% |
|
7 58.3% |
|
1 8.3% |
|
0 |
|
0 |
|
0 |
|
1.75 0.62 |
|
||||||||||||||||||||||||||
|
21.
Staff treated me with respect. |
|||||||||||||||||||||||||||||||||||||||
|
7 58.3% |
|
5 41.7% |
|
0 |
|
0 |
|
0 |
|
0 |
|
1.42 0.51 |
|
||||||||||||||||||||||||||
|
22.
Staff understood my family’s cultural traditions. |
|||||||||||||||||||||||||||||||||||||||
|
2 16.7% |
|
4 33.3% |
|
3 25% |
|
0 |
|
0 |
|
3 25% |
|
2.11 0.78 |
|
||||||||||||||||||||||||||
|
23.
Staff respected my family’s religious/spiritual beliefs. |
|||||||||||||||||||||||||||||||||||||||
|
1 8.3% |
|
6 50% |
|
2 16.7% |
|
0 |
0 |
|
3 25% |
|
2.11 0.60 |
|
|||||||||||||||||||||||||||
|
24. Staff spoke with me in a way that I understood. |
|||||||||||||||||||||||||||||||||||||||
|
3 25% |
|
9 75% |
|
0 |
|
0 |
|
0 |
|
0 |
|
1.75 0.45 |
|
||||||||||||||||||||||||||
|
25.
Staff were sensitive to my cultural/ethnic background. |
|||||||||||||||||||||||||||||||||||||||
|
1 8.3% |
|
4 33.3% |
|
2 16.7% |
|
0 |
|
0 |
|
5 41.7% |
|
2.14 0.69 |
|
||||||||||||||||||||||||||
|
26.
I felt we were discriminated against while trying to get services
here. |
|||||||||||||||||||||||||||||||||||||||
|
0 |
|
0 |
|
1 8.3% |
|
1 8.3% |
|
9 75% |
|
1 8.3% |
|
1.27 0.65 |
|
||||||||||||||||||||||||||
Note: Lower scores indicate higher levels of satisfaction
TABLE TWELVE
MHSIP – Youth (YSS)
(Based on 6 surveys
returned at Time One and 1 at Time Two)
|
1.
Overall, I am satisfied with the services that I received. |
|
|||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
Mean SD |
|
|||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
|
||||||||||||||||||
|
2 33.3% |
1 100% |
3 50% |
0 |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
0 |
0 |
2.00 1.09 |
0 |
|
||||||||||||||||||
|
As a result of the services I received: |
|
|||||||||||||||||||||||||||||||
|
2.
I am better at handling daily life. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
2 33.3% |
1 100% |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
2.00 0.82 |
0 |
|
||||||||||||||||||
|
3.
I get along better with family members. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
3 50% |
0 |
1 16.7% |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
2.20 1.10 |
0 |
|
||||||||||||||||||
|
4.
I get along better with friends and other people. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
3 50% |
0 |
0 |
1 100% |
0 |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
1.75 0.50 |
0 |
|
||||||||||||||||||
|
5.
I am doing better in school and/or work. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
1 16.7% |
0 |
1 16.7% |
1 100% |
2 33.3% |
0 |
1 16.7% |
0 |
0 |
0 |
2.80 1.20 |
0 |
|
||||||||||||||||||
|
6.
I am better able to cope when things go wrong. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
4 66.7% |
1 100% |
0 |
0 |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1.80 0.45 |
0 |
|
||||||||||||||||||
|
7.
I am satisfied with my family life right now. |
|
|||||||||||||||||||||||||||||||
|
2 33.3% |
0 |
3 50% |
1 100% |
0 |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1.60 0.55 |
0 |
|
||||||||||||||||||
|
Feedback about the services I received: |
|
|||||||||||||||||||||||||||||||
|
8.
I helped to choose my services. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
3 50% |
1 100% |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
0 |
0 |
0 |
2.67 1.51 |
0 |
|
||||||||||||||||||
|
9.
I helped to choose my treatment goals. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
4 66.7% |
1 100% |
0 |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
0 |
2.33 1.37 |
0 |
|
||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
|
Notes: 1) Lower scores indicate higher levels of
satisfaction |
||||||||||||||||||||||||||||||||
|
2) Gray boxes
provide Time One data; white boxes provide Time Two data |
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
10.
The people helping me stuck with me no matter what. |
|
|||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
Mean SD |
|
|||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
|
||||||||||||||||||
|
1 16.7% |
1 100% |
3 50% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
0 |
2.40 1.52 |
0 |
|
||||||||||||||||||
|
11.
I felt I had someone to talk to when I was troubled. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
4 66.7% |
1 100% |
0 |
0 |
0 |
0 |
0 |
1 16.7% |
0 |
0 |
1.80 0.45 |
0 |
|
||||||||||||||||||
|
12.
The people helping me listened to what I had to say. |
|
|||||||||||||||||||||||||||||||
|
2 33.3% |
0 |
3 50% |
1 100% |
0 |
0 |
0 |
0 |
0 |
1 16.7% |
0 |
0 |
1.60 0.55 |
0 |
|
||||||||||||||||||
|
13.
I was actively involved in my own treatment. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
3 50% |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
|
2.20 1.11 |
0 |
|
||||||||||||||||||
|
14.
I received the services that were right for me. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
1 16.7% |
1 100% |
2 33.3% |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
2.60 1.14 |
0 |
|
||||||||||||||||||
|
15.
The location of services was convenient. |
|
|||||||||||||||||||||||||||||||
|
0 |
0 |
4 66.7% |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
2.40 0.89 |
0 |
|
||||||||||||||||||
|
16.
Services were available at times that were convenient for me. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
1 16.7% |
1 100% |
0 |
0 |
2 33.3% |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
3.20 1.64 |
0 |
|
||||||||||||||||||
|
17.
If I need services in the future, I would use these services again. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
3 50% |
1 100% |
0 |
0 |
0 |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
2.40 1.52 |
0 |
|
||||||||||||||||||
|
18.
I got the help I wanted. |
|
|||||||||||||||||||||||||||||||
|
0 |
0 |
4 66.7% |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
2.40 0.89 |
0 |
|
||||||||||||||||||
|
19.
I got as much help as I needed. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
2 33.3% |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
2.25 1.26 |
0 |
|
||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
Notes: 1) Lower scores indicate higher levels of
satisfaction |
|
|||||||||||||||||||||||||||||||
|
2) Gray boxes provide
Time One data; white boxes provide Time Two data |
|
|||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
|
|
|||||||||||||||||||||||||||||||
|
20.
I, not staff, determined my treatment goals. |
|
|||||||||||||||||||||||||||||||
|
Strongly Agree |
Agree |
Neutral |
Disagree |
Strongly Disagree |
N/A |
Mean SD |
|
|||||||||||||||||||||||||
|
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
T1 |
T2 |
|
||||||||||||||||||
|
0 |
0 |
1 16.7% |
1 100% |
2 33.3% |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
1 16.7% |
0 |
3.40 1.14 |
0 |
|
||||||||||||||||||
|
21.
Staff treated me with respect. |
|
|||||||||||||||||||||||||||||||
|
3 50% |
0 |
2 33.3% |
1 100% |
1 16.7% |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
1.67 0.82 |
0 |
|
||||||||||||||||||
|
22.
Staff understood my family’s cultural traditions. |
|
|||||||||||||||||||||||||||||||
|
2 33.3% |
0 |
2 33.3% |
1 100% |
0 |
0 |
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
2.00 1.22 |
0 |
|
||||||||||||||||||
|
23.
Staff respected my family’s religious/spiritual beliefs. |
|
|||||||||||||||||||||||||||||||
|
3 50% |
0 |
2 33.3% |
1 100% |
1 16.7% |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
1.67 0.82 |
0 |
|
||||||||||||||||||
|
24.
Staff spoke with me in a way that I understood. |
|
|||||||||||||||||||||||||||||||
|
3 50% |
0 |
3 50% |
1 100% |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
1.50 0.55 |
0 |
|
||||||||||||||||||
|
25.
Staff were sensitive to my cultural/ethnic background. |
|
|||||||||||||||||||||||||||||||
|
2 33.3% |
0 |
2 33.3% |
1 100% |
2 33.3% |
0 |
0 |
0 |
0 |
0 |
0 |
0 |
2.00 0.89 |
0 |
|
||||||||||||||||||
|
26.
I felt discriminated against while trying to get services here. |
|
|||||||||||||||||||||||||||||||
|
1 16.7% |
0 |
0 |
0 |
1 16.7% |
0 |
1 16.7% |
1 100% |
3 50% |
0 |
| ||||||||||||||||||||||