Do mobile applications foster sustainable mobility? Evidence from a field experiment

Abstract

Mobile applications hold promise to foster sustainable mobility behavior, but evaluations of their effectiveness are subject to a number of empirical challenges. We conduct a randomized controlled trial with three distinctive features: unobtrusive tracking of the control group, limited sample attrition, and a representative sample. In our study, 410 participants track their mobility behavior over a 5 week period. After 1 week, the treatment group engages with the user interface of the “Swiss Climate Challenge App”. The user interface combines information on individual $\hbox {CO}_2$ emissions with gamification features. We find a treatment effect that implies a $9.8\%$ reduction in emissions caused by access to the mobile application. While we lack the statistical power to exclude a zero average effect, we find statistically significant emission reductions in the second half of the intervention period, among subjects in medium population density areas, and among men. Our findings suggest that mobile applications could generate considerable net benefits, but larger studies will be needed for validation.

1 Introduction

The transport sector poses a major challenge to the decarbonization of the global economy. In 2019, transportation was responsible for 24% of global $\hbox {CO}_2$ emissions from burning fossil fuels (International Energy Agency, 2020). Unless behavioral change, technology, and changes in the built environment can decouple transport emissions from economic activity and population growth, emissions in the transport sector will keep growing (Creutzig et al., 2014). Monetary incentives have been shown to reduce externalities associated with mobility (Hintermann et al., 2024; Tarduno, 2021; Kreindler, 2023) and to promote biking (Máca et al., 2020; Ciccone et al., 2021) as well as public transport use (Gravert and Olsson Collentine, 2021). Standard behavioral interventions like information provision, however, seem to be largely ineffective in this domain (Kristal and Whillans, 2020; Rosenfield et al., 2020; Hintermann et al., 2024).

Mobile applications offer new possibilities in the endeavor to foster sustainable mobility. A range of mobile applications in the spirit of “persuasive technology” (Fogg, 2002) are currently available (Froehlich et al., 2009; Jylhä et al., 2013; Jariyasunant et al., 2015; Cellina et al. 2019). Whether they indeed change mobility behavior is unclear. The meta-analysis of Sunio and Schmöcker (2017, p. 553) finds that “methodologically robust studies are largely missing”. Cellina et al. (2019) recently published the first randomized controlled trial on this question, but the authors point out three limitations of their study: obtrusive tracking, severe sample attrition, and a potentially unrepresentative sample.

In our study, we evaluate the “Swiss Climate Challenge App” (henceforth: SCC App) in a randomized controlled trial that addresses the aforementioned limitations. Study participants were recruited to participate in a study on mobility and to continuously use a tracking app for a period of 5 weeks. The tracking app comes with a plain user interface that shows only whether all necessary permissions are granted. After a 1-week pre-intervention period, the user interface of the treatment group switches to the SCC App for the rest of the study period. The SCC App provides graphical feedback on users’ personal mobility as well as additional features that rely on moral appeal, social comparison, and goal setting.¹ The control group remains in the plain user interface, without any mention of sustainability aspects.

Our randomized controlled trial provides three main contributions. First, we utilize technology that allows unobtrusive tracking, limiting potential experimenter demand effects. In contrast to subjects in Cellina et al. (2019), participants in our study do not have to manually validate their trips. Second, we incentivize full study participation to limit sample attrition. A large share of the participation fee was conditional on compliance with tracking criteria and our final sample size of 410 compares favorably to the 52 reported by Cellina et al. (2019). Third, we use a representative population sample. Our study sample was recruited to represent the Swiss population in terms of age, gender, and language region.

Participants in the treatment group engage with the SCC App. $78\%$ of the treatment group open the SCC App at some point during the intervention period, $58\%$ open the SCC App on at least five different days, and $31\%$ use it on at least 14 different days (out of 28 days in the intervention period).

We use a standard difference-in-differences approach to evaluate the effect of the SCC App. Our results suggest that the SCC App reduces emissions by $9.8\%$, but this average effect is not statistically significant at conventional significance levels. We find substantial heterogeneity in this effect, with statistically significant reductions in emissions in the second half of the intervention period, in medium population density areas, and among male participants. This study is, to the best of our knowledge, the largest randomized controlled trial to evaluate the effects of mobile applications that aim to foster sustainable mobility behavior in the spirit of “persuasive technology”. Notwithstanding, larger studies will be needed to validate our results.

2 Experimental design

We evaluate the SCC App in a randomized controlled trial. This section describes the functionalities of the SCC App, our recruiting procedure, and the study protocol.

The SCC App is a mobile application that automatically tracks the user’s mobility behavior with location and motion sensors. The underlying technology, provided by the company MotionTag, automatically detects the mode of transport with 92% accuracy (Molloy et al., 2020). It correctly detects more than 90% of airplane, car, subway, tram, and walk trips, but has some difficulty detecting bike, bus, train, and regional train trips (see Table 7 in Molloy et al., 2023) and may miss some trips due to gaps in GPS data (Mesaric et al., 2022). The SCC App uses the detected mode of transport in combination with the distance traveled to calculate the user’s $\hbox {CO}_2$ emissions.

The SCC App provides the user with graphical feedback on the environmental impact of her behavior and uses moral appeal, social comparison, and goal setting to motivate the user to reduce emissions. The home screen of the SCC App combines feedback with a moral appeal: it illustrates graphically how the user’s mobility behavior—if adopted by the entire world population—would affect global temperature. The respective temperature increase (e.g. $+0.6$  °C) is shown along with a happy-face earth for low values (see Fig. 1a) or a knocked-out earth for high values (see Fig. 1b). Users can also consult graphical feedback on the amount of CO₂ they emit with their mobility behavior on any given day, week, or month, as well as a breakdown of their mobility behavior by means of transport and details for every trip they took. They can also look at a social comparison feature to compare their emissions with the regional (cantonal) average or with invited friends. In addition, users can accept personal challenges like “Take the bike every day of the week” and win symbolic badges.²

A market research company recruited the participants for this study. The company provided us with a representative sample of the Swiss population (based on gender, age, and language region) and was responsible for all communication with the participants. Participants were informed that they were going to take part in a study on mobility. To avoid experimenter demand effects, the notion of sustainability was not imparted to the control group until the end of the study period. Participants needed to have either an iOS or Android smartphone with internet access. They further had to be willing to allow us to track their location for the entire study duration. Participants received 10 Swiss Francs³ in exchange for taking part in the pre-intervention period of the study. In order to minimize sample attrition, participants received an additional 40 Swiss Francs if they complied with our instructions until the end of the intervention period. Both in the pre-intervention period and in the intervention period, remuneration was conditional on granting all necessary permissions for location tracking.

One thousand seven hundred and eleven study participants gave informed consent, filled out a survey at the beginning of the study, and successfully downloaded the mobile application. Among other variables, we collected data on the participants’ characteristics and various self-reported environmental behaviors. After filling out the survey, we provided the participants with the instructions for downloading the tracking app (described below) and for granting the necessary location tracking permissions on the mobile phone. The instructions also provided a number, with which the study participants could register in the tracking app.

We recruited participants from April 6, 2021 until April 18, 2021. After this onboarding period, participants tracked their mobility behavior in the pre-intervention period, which lasted from April 18 until April 26, 2021.⁴ Participants were instructed to use the “ETH Research App”, a specifically developed mobile application, to track their mobility behavior. This application uses the same location tracking technology as the SCC App, but it has a plain user interface that does not refer to environmental aspects of mobility. Users of the ETH Research App only see whether all necessary permissions for mobility tracking are granted, as illustrated in Fig. 1c. 1146 participants ($67\%$ of 1711) successfully used the ETH Research App during the entire pre-intervention period. Our budget constraint allowed us to invite 570 of these participants to take part in the intervention period. Before the start of the intervention period, we randomly assigned these 570 participants to the control and treatment group. Randomization was stratified on recruitment date and $\hbox {CO}_2$ emissions in the pre-intervention period. This randomization procedure ensured that participants in the control and treatment group had a comparable recruitment date and comparable emissions in the pre-intervention period.

Fig. 1

Home screens of the SCC App (a, b) and the ETH Research App (c)

In the 4-week intervention period, the control group continued to use the ETH Research App, while the treatment group was given access to the SCC App. On April 27, participants in the treatment group received an email encouraging them to use the SCC App with its various features. They only needed to tap a button in the ETH Research App, which then turned into the full SCC App. At the same time, participants in the control group received an email which asked them to continue tracking with the ETH Research App for the rest of the study period.⁵ 410 ($72\%$ of 570) participants successfully tracked their mobility behavior in the intervention period. We observe no statistically significant difference in tracking success between the control and treatment group.⁶ Finally, participants were asked to fill out a final survey.

To summarize, $67\%$ of 1711 participants completed the pre-intervention period and $72\%$ of 570 participants invited for the intervention period completed the intervention period. The resulting sample consists of 410 participants.⁷

3 Data and descriptive statistics

We have detailed information about each trip taken by each user. For the purpose of this study, we use data on the date of a trip, the mode of transport, and the associated $\hbox {CO}_2$ emissions.⁸

Table 1 reports the mobility behavior of our sample during the pre-intervention period as well as participants’ characteristics. Column (1) shows the means and standard deviations (in parentheses) of the sample. Columns (2) and (3) present the same statistics for the control and treatment group. Column (4) reports whether the differences between control and treatment group are statistically significant (p-values). According to column (1), participants emit on average 6.3 kg of $\hbox {CO}_2$ per day during the pre-intervention period. The participants take an average of 6.9 trips (44.2 km). 2.1 of these trips (32%, 30.2 km) are by car, 0.9 (12%, 10.3 km) by public transport, 0.4 (6%, 1.4 km) by bike, and 3.4 (51%, 2.1 km) on foot. Compared to the 2021 Mobility and Transport Microcensus (Bundesamt für Statistik, 2023), our sample has similar modal shares, but it is more mobile (both in terms of trips and distance). A potential explanation for this difference could be increased mobility following the partial lifting of COVID-19 restrictions in the pre-intervention period. Participants in the Mobility and Transport Microcensus take an average of 3.8 trips (30 km) across modes, 1.4 of those (37%, 20.8 km) by car, 0.5 (13%, 5.9 km) by public transport, 0.2 (5%, 0.7 km) by bike, and 1.6 (42%, 1.6 km) on foot. Study participants were recruited to be representative of the Swiss population in terms of age, gender, and language region. Our sample is on average 44 years old, has a similar number of women and men, and a German speaking majority. Column (4) indicates that the differences between the treatment group and the control group are statistically insignificant.⁹ Hence, Table 1 suggests that the randomization procedure achieved a balance on observable characteristics.

Table 1 Pre-intervention data

We further analyze data on interactions with the SCC App. So-called screen events are recorded every time a user views the app’s user interface. An event is annotated with the timestamp and the page that was viewed. These data allow us to gain insights into participants’ engagement with the SCC App and to identify the most frequently used features.

The majority of participants in the treatment group uses the SCC App repeatedly. 166 of the 212 participants in the treatment group ($78\%$) use the SCC App at some point during the intervention period. Figure 2 shows a histogram of the total number of SCC App screen events (i.e. interactions with the user interface) among these participants. 31 participants use the SCC App sporadically, with up to 10 screen events, but many use it more extensively. 25 participants have more than 100 SCC App screen events. Most of these screen events ($53\%$) pertain to the SCC App home screen as shown in Fig. 1a and b, but users also view the social comparisons ($8\%$), individual trips ($7\%$), mobility challenges ($6\%$), among other pages.

Fig. 2

Histogram of screen events by participant

We further analyze the distribution of screen events by hour of the day and over the course of the intervention period. Figure 3 shows that participants in the treatment group use the SCC App throughout the day, but most screen events occur in the evening. This pattern is consistent with the idea that participants use the SCC App to reflect on their mobility behavior at the end of the day. Figure 4 shows that the number of screen events is very high at the beginning of the intervention period. After a few days, the number of screen events stabilizes at around 4 per participant and day. This pattern suggests that participants use the SCC App intensively at the beginning of the intervention period, but then use it more sporadically.

Fig. 3

Screen events by hour of the day

Fig. 4

Screen events by date

4 Estimation and results

We use a difference-in-differences regression framework to estimate the effect of the SCC App on the the log of $\hbox {CO}_2$ emissions. To deal with zero values, we calculate $log(CO_{2i,t}+1)$,¹⁰ where i denotes individual participants and t denotes time periods (i.e. pre-intervention period or intervention period):

$$\begin{aligned} log(CO2_{i,t}+1) =\, \alpha + \beta Treatment_i + \gamma Time_{t} + \delta (Treatment_i\times Time_t) + \phi X_i + \epsilon _{i,t} \end{aligned}$$

(1)

We regress $log(CO2_{i,t}+1)$ on a treatment dummy variable ($Treatment_i$ is equal to 1 if participant i is allocated to the treatment group, 0 otherwise), a dummy variable for observations after the start of the intervention ($Time_t$ is equal to 1 if the observation is from the intervention period, 0 otherwise), and the interaction of both dummy variables ($Treatment_i\times Time_t$). The coefficient for the latter, $\delta$, is our parameter of interest, as it measures the effect of the SCC App on $\hbox {CO}_2$ emissions. We estimate Eq. (1) with OLS and calculate heteroskedasticity robust standard errors. Our basic specification does not include additional control variables $X_i$. In addition, we estimate specifications controlling for demographic characteristics (age, gender, income, education, and language), location (population density and distance to public transport), and pre-intervention mobility behavior ($\hbox {CO}_2$ emissions, total trips, car trips, public transport trips, bike trips, and walk trips).

Table 2 shows the results of our analysis. Column (1) shows the OLS regression coefficient (and standard error in parenthesis) for the specification that regresses $log({{CO}_2}+1)$ on Treatment, Time, and $Treatment~\times ~Time$. We find no statistically significant coefficient for Treatment. The coefficient for Time indicates that emissions are $28\%$ higher during the intervention period compared to the pre-intervention period.¹¹ This time pattern may be related to the partial lifting of COVID-19 regulations during the pre-intervention period. Our coefficient of interest for the interaction term $Treatment \times Time$ implies a $9.8\%$ reduction in emissions, but this effect is statistically insignificant at conventional significance levels ($p>0.10$).

Table 2 Regression results

The specification in column (2) of Table 2 includes control variables for age, gender, income, education, and language. Compared to the specification in column (1), the coefficient of interest in column (2) is equal in size but remains statistically insignificant. The standard error in the second specification remains also equal in size, indicating that the set of control variables used for this specification cannot increase the precision of our main estimate. The specification in column (3) of Table 2 includes additional control variables for polulation density and the distance to public transport. The coefficient of interest in this specification is equal in size to the coefficients in columns (1) and (2). The standard error is only slightly lower than in the first two specifications.

The specification in column (4) of Table 2 further includes pre-intervention mobility variables. Although this specification does not change the coefficient of interest, it increases its precision compared to the specifications in columns (1), (2), and (3). Nevertheless, the coefficient remains statistically insignificant. The most important control variables appear to be pre-intervention emissions, pre-intervention car trips, and population density.

The results in Table 2 are not susceptible to outliers. Table 3 in Appendix 4 shows very similar results for the sample that excludes observations in the top $1\%$ of the outcome variable.

The effect of the SCC App may vary over time, as Fig. 4 suggests decreasing engagement over time. We test for this possibility by estimating Eq. (1) separately for the first and second half of the intervention period. Tables 4 and 5 in Appendix 4 show that the coefficient of interest is small and statistically insignificant in the first half of the intervention period, but larger and statistically significant ($p<0.10$) in the second half of the intervention period. This finding suggests that, despite decreasing engagement, the effect of the SCC App may increase over time.

Tables 6,  7, 8 and 9 in Appendix 4 show results for individual modes of transport. All coefficients of interest in these tables are statistically insignificant. If taken at face value, these coefficients suggest reduced car use (Table 6), limited effects on public transport (Table 7) and bike use (Table 8), and more trips on foot (Table 9). These tables yield other notable findings. Table 6 shows that the car is used more by high-income participants in medium and low density zip codes, who have more emissions, more car trips, and fewer public transport trips in the pre-intervention period. Table 7 shows that public transport is negatively associated with age and French speakers. Table 8 shows that bike use is associated with men, income, and high pre-intervention period emissions. In Table 9, walking is negatively associated with distance to public transport, emissions, car trips, public transport trips, and bike trips in the pre-intervention period.

Tables 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 and 20 in Appendix 4 show results for $\hbox {CO}_2$ emissions among different sub-samples. We split our sample by low/medium/high population density (Tables 10, 11 and 12), below/above median emissions in the pre-intervention period (Tables 13 and 14), below/above median age (Tables 15 and 16), male/female gender (Tables 17 and 18), and sub-samples with only those treatment participants who did not/did interact with the SCC App (Tables 19 and 20). These sub-sample regressions reveal two interesting dimensions of effect heterogeneity: population density and gender. We find an increase in $\hbox {CO}_2$ emissions in low density areas (specification 4 in Table 10, $p<0.10$), but a decrease in medium density areas (specification 4 in Table 11, $p<0.10$) and high density areas (Table 12, not statistically significant). This finding suggests that the SCC App may be more effective where public transport is more accessible, and potentially counterproductive in areas where car use is more prevalent. Such counteracting effects, sometimes referred to as “boomerang effects” (Schultz et al., 2007), have been found in related research on mobility behavior (Gessner et al., 2024). The coefficient of interest is also statistically significant in the male sub-sample (specification 4 in Table 17, $p<0.05$). This finding is consistent with Hintermann et al. (2024), who find a particularly strong effect of a mobility pricing intervention among men.

We also test whether the SCC App has an effect on environmental behaviors other than the target behavior (Tiefenbeck et al., 2013; Jessoe et al., 2021; Goetz et al., 2024). Two surveys at the beginning and end of our study provide data on self reported environmental behaviors. We do not find evidence for spillover effects. Appendix 5 depicts the details of this analysis.

On average, we do not find statistically significant effects of the SCC App on mobility behavior.¹² However, we find substantial heterogeneity in the effect of the SCC App, with reductions in emissions in the second half of the intervention period, in medium population density areas, and among male participants.

5 Conclusions

We evaluate the effect of mobile applications on sustainable mobility. Our randomized controlled trial provides three methodological contributions: unobtrusive tracking of the control group, limited sample attrition, and a representative population sample. We find an effect on mobility-related $\hbox {CO}_2$ emissions that is of substantial magnitude, but not statistically significant. COVID-19 may well have contributed to the lack of statistical significance. The lifting of restrictions likely impacted participants very differently, and some participants may have suffered from COVID-19 during our study period. The resulting variation in mobility behavior arguably makes it harder to detect the effect of the SCC App.

The large magnitude of our effect estimate suggests that the SCC App improves welfare by reducing $\hbox {CO}_2$ emissions. The $9.8\%$ reduction in emissions compares favorably to similar interventions in the literature. As an example, Hintermann et al. (2024) use similar technology in the Swiss context and find a $4\%$ reduction caused by an intervention that includes not only information, but also mobility pricing. Our large effect estimate could indicate that the SCC App was particularly successful in motivating behavior change. This would imply substantial benefits through reduced environmental damage and associated social costs. A $9.8\%$ reduction of yearly mobility emissions of 1.21 $\hbox {tCO}_2$ (Bundesamt für Umwelt, 2024) corresponds to 0.12 $\hbox {tCO}_2$ per person and year. Valued at the social cost of carbon of CHF 185 (Rennert et al., 2022), this reduction is worth CHF 22. The marginal costs of digital applications like the SCC App are typically close to zero. If development costs are considered sunk, the welfare effect of the SCC App is likely positive. Given the statistically insignificant estimate, we caution the reader to take this interpretation with a grain of salt.

Further research on behavioral mobility interventions is imperative—not least because alternative interventions like bans or price increases for $\hbox {CO}_2$-emitting mobility suffer from strong acceptability problems. Interventions based on mobile apps may have a substantially higher degree of acceptability. New evaluation approaches may be necessary to detect potentially small effects of behavioral mobility interventions. We see particular promise in the use of anonymized mobile network data, which may allow researchers to dispense with paid participant tracking and conduct unobtrusive studies at scale. Offering a mobile application like the SCC App to a group that can be identified in mobile network data, treatment effects could be credibly estimated in a difference-in-differences approach. We also see potential in further investigating treatment effect heterogeneity. Our results suggest that mobility interventions could be particularly effective if targeted at specific population groups or areas. Future research may scrutinize the strong effects on men and participants in medium density areas, and develop interventions that are tailored to the needs of specific population groups.

Appendices

Appendix 1

Screenshots of the SCC App

Fig. 5

Graphical feedback on mobility behavior

Fig. 6

Moral appeal features

Fig. 7

Social comparison features

Appendix 2

Transcriptions of intervention emails

Treatment group intervention email

Dear Study Participant,

Thank you for participating in this ETH study on mobility.

As of today, the ETH Research App on your device has an additional feature. By tapping on “Open Climate Challenge” at the bottom of the App’s home screen, you can open the Swiss Climate Challenge App.

The Swiss Climate Challenge App allows you to observe how much $\hbox {CO}_2$ you produce with your mobility, the breakdown of your mobility behavior by means of transport, and what kind of impact this has on the environment. You can also take part in various in-app challenges. We encourage you to interact with these various functionalities.

If you want, you can also invite your friends to download the Swiss Climate Challenge App from the App Store or Play Store, and use the App’s comparison feature. Please note that you are under no obligation to activate this feature.

The location tracking procedure and privacy policy remain the same. You do not need to make any further changes to the settings on your device.

You have already earned 10 Swiss Francs. You can earn another 40 Swiss Francs if you continue to participate in the study until May 24th. We would like to remind you that this additional remuneration is conditional upon granting the app all of the required location tracking permissions at all times.

Thank you for your commitment to this study!

Best regards,

The study team

Control group intervention email

Dear Study Participant,

Thank you for participating in this ETH study on mobility.

You have already earned 10 Swiss Francs. You can earn another 40 Swiss Francs if you continue to participate in the study until May 24th. We would like to remind you that this additional remuneration is conditional upon granting the app all of the required location tracking permissions at all times.

Thank you for your commitment to this study!

Best regards,

The study team

Appendix 3

Study schedule

See Fig. 8.

Fig. 8

Study schedule

Appendix 4

Additional tables

See Tables 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 and 20.

Table 3 Regression results (outliers excluded)

Table 4 Regression results (first half of intervention period)

Table 5 Regression results (second half of intervention period)

Table 6 Regression results (car trips)

Table 7 Regression results (public transport trips)

Table 8 Regression results (bike trips)

Table 9 Regression results (walk trips)

Table 10 Regression results (low density sub-sample)

Table 11 Regression results (medium density sub-sample)

Table 12 Regression results (high density sub-sample)

Table 13 Regression results (low emission sub-sample)

Table 14 Regression results (high emission sub-sample)

Table 15 Regression results (young sub-sample)

Table 16 Regression results (old sub-sample)

Table 17 Regression results (male sub-sample)

Table 18 Regression results (female sub-sample)

Table 19 Regression results (sub-sample without screen events)

Table 20 Regression results (sub-sample with screen events)

Appendix 5

Spillover analysis

At the beginning and end of the study, participants were asked to answer a series of questions related to their other environmental behaviors. Participants reported eight environmental behaviors as displayed in Table 21. The response options ranged from of 1 (Never) to 7 (Always) on a Likert scale. Additionally, participants could respond with “I don’t know” or “I don’t want to specify” (in which case we impute the mode of the respective question).

We calculate the mean of the eight responses at the beginning and end of the study. Our outcome variable is the difference between these two values. Positive values in $\Delta SpilloverScore$ indicate a change to more environmentally friendly self reported behavior.

$$\begin{aligned} \Delta SpilloverScore = SpilloverScore_{end} - SpilloverScore_{beginning} \end{aligned}$$

Table 22 presents the results of our spillover analysis. Column (1) depicts a simple regression of $\Delta SpilloverScore$ on Treatment. Column (2) adds participants’ age, gender language, above-median income, and higher education as additional control variables. Column (3) adds degree of urbanisation and distance to public transport (in km) as control variables. Column (4) further adds pre-intervention mobility behavior as control variables. All three specifications find positive coefficients that are small and statistically insignificant.

Table 21 Survey on environmental behaviors

Table 22 Regression results (Spillover behaviors)