Cool California City Challenge

The Cool California Challenge is a state-wide competition between California cities engaging their residents in climate action. Participants earn points for tracking and reducing their electricity, natural gas and motor vehicle emissions and for simple one-time actions, like uploading photos and stories, inviting friends or taking a survey. Results from a quasi-experimental design suggest a 14% reduction in electricity use among program participants.

Background

Note: To minimize site maintenance costs, all case studies on this site are written in the past tense, even if they are ongoing as is the case with this particular program

Getting Informed

The original research proposal submitted by UC Berkeley to the California Air Resources Board in 2010 was to involve three leading California cities in a grassroots, community-based social marketing (CBSM) campaign to reduce household carbon footprints. Participants would be required to complete a research survey that asked about adoption, barriers and benefits of low carbon behaviors, as well as their motivations for participating, attitudes, values and demographic information. This information would be used to iteratively design and test programs in those communities over several years.

There were several problems with this approach. First, the University of California, Berkeley’s Office of Protection of Human Subjects did not allow the research survey to be used as a requirement for program participation. Second, there was recognition that each household has a different mix of greenhouse gas reduction opportunities from home energy, transportation, food, waste and consumption, so targeting only specific behaviors would limit engagement. Finally, early on in the planning stages the organizers recognized that researching and designing strategies that work well in three highly motivated cities might have limited transferability to other less engaged and economically disadvantaged communities.

In an attempt to learn from a more diverse range of participants, the program was transformed into a state-wide program, open to any California city. The eight cities that joined the first year represented a broad cross-section of California ranging in size, demographics and political ideology. Given this diversity and the desire to keep the scope of interventions broad, rather than focus on barriers and benefits of specific end-use behaviors for targeted audiences, the program shifted to effectively employing a broad range of CBSM strategies in the software and marketing materials and training local program managers in CBSM theory and practice.

While organizers initially thought participants would be highly motivated by recognition for their cities and personal rewards, these ranked last on a list of motivations collected through survey responses. The top motivations were improving their communities, protecting the environment and helping organizations they care about. This was true across all demographic characteristics and political orientations. The messaging now focuses on creating more vibrant and sustainable communities rather than personal rewards.

Delivering the Program

• Cities signed up to participate, or designated a local community-based organization
• Participating cities received training on the program
• Cities used existing email lists (most effective), outreach events and tabling to enrol households 
• Participants tracked monthly energy bills and odometer readings using online software (Feedback; Goal Setting)
• Participants earned Kudo Points for inviting other participants, sharing photos and stories and other small actions (Building Motivation and Engagement Over Time; Competitions; Norm Appeals)
• Participants could create or join teams and a scoreboard ranked households, teams and cities. In addition, participants received weekly progress reports on their household, team and city rank and points (Competitions; comparative Feedback and Recognition; Norm Appeals)
• Participants receive persuasive messaging from local program managers via email, in-person and/or phone
• There were five levels, based on points: Minions, Warriors, Magicians, Champions and Gurus. Different messages were targeted to each level. (Personalized Communications)
• Cities received seed money based on the number of participants who signed up by an early program deadline ($30k was distributed between cities in year 1 and $50k in year 2)(Financial Incentives for participating municipalities)
• During the first year, 4 “Finalist” cities received in-person CBSM workshops and follow-up community strategy meetings. During year two, webinars were online for all cities.
• At the end of the program (which lasted 13 months in year 1 and 6 months in year 2) UC Berkeley conducted quality control of data, including contacting participants and correcting any obvious anomalous data.
• The California Air Resources Board conducted awards ceremony with top 3 cities. In Year 2 all cities shared $50,000 in prize money (on top of $50,000 in seed money). It was anticipated that in subsequent years all money would be distributed as prize money at the end, with no seed money at the beginning; however, community-based organizations would still receive seed money for being part of the Energy Upgrade California Ambassadors program. (Feedback and Recognition; Financial Incentives for participating municipal governments)

Overcoming Barriers

The program targeted two main barriers: low visibility and low self-efficacy.

Visibility: Energy use is usually fairly invisible. Participants’ total energy usage was made visible to other participants, as well as the number of points achieved. Participants were also encouraged to share stories and upload photos of actions they had taken. This feature was extremely popular even though it was hard to find in the software.

Self-efficacy: When faced with the immensity of climate change individuals tend to think that their individual actions do not matter. In other words, they do not believe in their ability to make a meaningful contribution to addressing the problem (low self-efficacy). The CoolCalifornia Challenge turned individual action into collective community action with a focus on improving communities and the state of California as a whole. This can lead to a greater sense of empowerment that individual action does add up to something meaningful. The program also sought to educate participants through regular email communication (addressing information barriers), build capacity and a sense of community around climate change, and enhance pro-environmental social norms in the community. 

Measuring Achievements

The program used a control group that was not randomly selected (quasi-experimental design), in addition to modeled results. 

Quasi-experimental design (year 1 only): the energy usage of participants who joined later was compared with usage of participants who joined earlier over the period for which the control group had not yet joined. Corrections were made based on location of city. Other variables such as income and education were evaluated, but deemed not to impact results. The mean electricity and natural gas usage of participants in both the control and treatment groups were closely aligned for two months prior to the competition, but diverged during the first five months of the program (May through September), after which the control group was not large enough to evaluate results. 

Modeled results (years 1 and 2): household energy usage was compared to a monthly benchmark for each household based on the number and age of people in homes and the location of homes (Jones and Kammen 2014). Each household’s energy usage in relationship to the benchmark was used to predict consumption in the subsequent month. If the household used less than their previous energy performance we assumed these were savings attributed to the program. Using this methodology for all months for electricity, natural gas and motor vehicles we calculated 500,000 lbs of CO2 in the first one-year program and 800,000 pounds in the second six-month long program.

Feedback

• Participants tracked their monthly energy bills and odometer readings, which provided them with individualized feedback on usage
• Participants earned Kudo Points for inviting other participants, sharing photos and stories and other small actions 
• Participants received weekly progress reports on their household, team and city rank and points 
• The California Air Resources Board conducted awards ceremony with top 3 cities. In Year 2 all cities shared $50,000 in prize money (on top of $50,000 in seed money).
• The CoolCalifornia Challenge provided feedback on the impacts of collective community action.

Results

Impacts - Individuals

Organizers had little understanding of the degree to which specific behaviors were changed since it did not track this in any rigorous way. In a post-program evaluation survey it did ask survey respondents what actions they had changed and many reported new behaviors or technologies purchased, but there are no data on the persistence of those changes or the extent to which behaviors change as a fraction of total participants.

Quasi-Experimental Design: Those participants who reported their energy use over the five months for which organizers could measure this effect using an experimental design. Extrapolating these savings over the entire 12 months of data, participants saved an average of about 1,000 kWh per year. It is likely that there were additional savings in natural gas in winter and gasoline from vehicles, but these could not be measured with the program's experimental design. On the other hand, there may have been some slippage after the competition ended, and those who reported their energy use may have reduced their energy use more than those who didn’t report their energy use.

Impacts - Overall

In the program's first two years, 17 cities participated in the challenge, eight in the first year and ten in the second, with one city participating both years.

Quasi-Experimental Design: The program had overall savings of between 1,000,000 and 3,8000,000 kWh per year. Participants reduced electricity consumption by 14% over the five months for which this effect could be measured using an experimental design. Extrapolating these savings, participants saved an average of about 1,000 kWh per household per year not including savings from natural gas and gasoline. If all of the participating households saved that much, the total would be about 2,700, 000 kwH in the first year and 3,800,000 kwH in the second year. Acknowledging that those who reported their energy use may have reduced their energy use more than those who didn’t report their energy use, one can estimate a minimum based on only the 1,000 participants who reported their energy use. 1,000 participants x 1,000 kWh / year = 1,000,000 kWh / year, not including savings from natural gas and gasoline.

Modelling: Modeled results (which DO include savings from natural gas and gasoline) indicated total energy savings of about 2,000,000 kWh and total GHG savings of about 1,330,000 lbs of CO2 equivalent, over two years. This included savings of 748,000 kWh and 500,000 lbs of CO2 equivalent during the first year of the program (13 months) and 1,242,000 kWh and 830,000 lbs of CO2 during the second round of the program (which lasted only 6 months.) This corroborates the results found using the Quasi-Experimental design.

Cost Effectiveness: The total cost to deliver the program was over $400,000 during the first year (pilot program) and $200,000 in the second year (including $100,000 in prize money). This did not include considerable volunteer time and in-kind contributions of time by city staff and community-based organizations. In the first year, the cost-effectiveness was therefore $0.53 per kWh saved ($400,000 / 748,000 kWh.) In the second year it was ($200,000 / 1,242,000.)

Improvement: During year two the program achieved 40% more participants and 60% more energy use reductions / GHG reductions in half the time and less than half the cost. It increased cost effectiveness by 69%, from $0.53 per kWh saved to $0.16 per kWh saved. Based on projections of increased participation in future years the program could easily become cost-effective compared with other utility energy efficiency programs.

Contacts

Christopher M. Jones, Ph.D.
Program Director, Cool Climate Network - RAEL
University of California at Berkeley
Tel: 510-643-5048  Email: cmjones@berkeley.edu

Notes

• The California Air Resources Board lent credibility to the awards and the program overall.
• The program ran with roughly 1/10 the budget of several other state-wide programs (e.g., Kansas Take Charge Challenge and Connecticut Neighbour-to-Neighbour Energy Challenge) during the first year and 1/20 the cost during the second year. Costs were kept low since expensive software had already been developed and by reducing the number of staff and volunteers needed to run the program: it is better to have one or two highly trained part-time staff than lots of volunteers and junior staff that need to be managed.
• The program is ongoing and several cities returned for more than one year, providing ongoing capacity building within communities.
• An important goal was to increase the benefits to all communities, not just those at the top of the leaderboard. One way in which the program did this was by providing prize money to all cities based on their performance (number of participants and points).
• Points were earned for being below a benchmark that was specific to households in each location. The benchmarking model provided estimates of electricity, natural gas, vehicle fuel consumption, food, waste, good and services for every zip code in the United States (Jones and Kammen 2014). Households earned five times as many points for reducing emissions over their own personal benchmark based on previous months (compared to the benchmark)
• There was no statistical correlation between size of cities and performance. Medium-sized cities seemed to perform best. It is crucial not to normalize points based on population.
• Although participants were highly altruistic they were also highly motivated by competition. Participation spiked only during intense moments of competition before deadlines; there was one deadline to enrol participants and another at the end of the program.
• Several cities also experienced either staffing changes or reductions during the program, which very much limited their capacity to participate. This is a key barrier to successful implementation of similar programs over multiple years.

This case study was written in 2016 by Chris Jones and Jay Kassirer.