By Dr. Simon D’Alfonso
The value of digital solutions to mental health care is now clearly established and has given rise to the field known as digital mental health or e-mental health. [1] Whilst mobile apps for mental health abound, another emerging use for smartphones involves collecting and analysing data from the sensors of such devices to infer information about a user’s psychology. At the intersection of the computing, data and psychological sciences, this use of smartphones is known as personal sensing or digital phenotyping and in terms of psychiatry or clinical psychology, such information can be used to predict or determine the presence of mental health issues. [2] Following are some examples of sensors/functionalities and early research that suggests how they are indicators of mental health.
The familiar GPS or geolocation sensors on our smartphones are used with applications such as Google Maps to guide us with directions from one place to another. Beyond this standard use, research into the association between location data and depression suggests that certain reductions in location change and the diversity of visited places were correlated with depression symptoms. [3] Naturally the types of location one visits can also offer psychologically relevant information. A related sensor is the accelerometer, which measures a phone’s movement in terms of acceleration and can be used to gauge physical activity, which can be associated with mental health outcomes.
Clinical text from sources such as patient session transcripts and non-clinical texts from peoples’ social media posts have been analysed using computational linguistics to detect mental health conditions [4], including psychosis. [5] In terms of smartphone text input, whilst it is technically possible to record the characters that a user types on their phone, clearly this form of sensing could be particularly intrusive. However, without needing to track such character input information, there is work being done based on the idea that non-semantic information from tactile interactions such as typing, swiping and scrolling dynamics alone could indicate certain cognitive and mental health conditions. [6]
Apart from specific sensors, general phone usage can also provide interesting information about user behaviour. When, how often or in what intervals someone is using their phone could provide behavioural information of diagnostic significance. For example, abnormal nocturnal smartphone use could indicate insomnia. Or which apps people are using and how often they are being used might say something about their current state.
Finally, as well as passively collecting data from such smartphone sensors and functionalities, another important component is the capacity to collect information from the active input of users responding to questionnaire prompts on their smartphones. Known as experience sampling or ecological momentary assessments (EMA) [7], smartphones can be used to periodically push short psychological assessments that prompt users for responses on their thoughts, feelings, behaviours and environment in the moment. Such information can now be used to investigate connections between passive sensing data and user-reported psychological states. It is also interesting to note that such real-time assessments can provide psychologically valuable secondary information in terms of how users respond to them. For example, the time it takes one to respond to a questionnaire prompt, the time it takes to complete the answers or whether they respond to it at all can provide insights into their current state of mind.
Passive sensing and EMA information can then go on to inform what have been termed Ecological Momentary Interventions (EMI). [8] These are health treatment or therapy suggestions delivered to users via their smartphones in the moment, in real-word situations. The ability to deliver contextually relevant interventions based on situational information from smartphones and other mobile devices is the future of integrating mental health care into people’s daily lives.
In an era where big tech companies are storing and mining large volumes of our personal data, particularly for commercial motives and sometimes problematically, due consideration must be given to the ethical and privacy issues surrounding digital phenotyping technology. Some of these challenges come down to proper governance and responsible handling of the data, at both a social level (e.g. don’t share personal client data without their consent to third parties) and technical level (e.g. ensure that data is stored in a secure way). Beyond this, personal sensing apps can be created so that the consumer is given complete control over which sensors they wish to share. Also, in developing digital phenotyping frameworks, preference can be given to sensors that are less intrusive; if either of two sensors can be used to indicate a certain condition, then the least intrusive should be selected.
The functionality to objectively and continuously assess mental health by analysing the data patterns an individual creates in their daily life offers a radically transformative alternative to traditional methods that often rely on unreliable or inconsistent self-reporting and subjective analyses. Digital phenotyping heralds a revolutionary and wide-reaching technology for the future of mental health care.
References
1 https://www.nationalelfservice.net/treatment/digital-health/top-10-research-questions-for-digital-mental-health-digitalmhq/
2 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6127813/, https://www.ncbi.nlm.nih.gov/pubmed/28375728
3 https://www.jmir.org/2015/7/e175/
4 https://www.cambridge.org/core/journals/natural-language-engineering/article/natural-language-processing-in-mental-health-applications-using-nonclinical-texts/32645FFCFD37C67DA62CA06DB66EB2F4
5 https://onlinelibrary.wiley.com/doi/full/10.1002/wps.20491
6 https://www.technologyreview.com/s/612266/the-smartphone-app-that-can-tell-youre-depressed-before-you-know-it-yourself/
7 https://nuighealthpsychology.wordpress.com/2015/11/23/5-tips-on-carrying-out-an-ecological-momentary-assessment-study/
8 https://www.nationalelfservice.net/treatment/digital-health/ecological-momentary-interventions-smartphones-have-changed-everything-and-heres-how-digital-mental-health-might-begin-to-catch-up/
About the author
Dr. Simon D’Alfonso is an e-mental health and cyberpsychology researcher with an active interest in smartphone personal sensing and digital phenotyping. A member of the University of Melbourne School of Computing and Information Systems, Simon leads the project “Digital technology and artificial intelligence for mental health” (https://bit.ly/2NbFii2). He is also the lead engineer and head of computational research at eOrygen, the Australian e-mental health group of Orygen, the National Centre of Excellence in Youth Mental Health (eorygen.org.au).