See What I Mean
A speech to image communication tool for people living with dementia.
Social enterprise, 2014 — 2016

“See What I Mean” began as a project that used technology to fuse visualisation and language in an attempt to unlock potential and capacity in the space between words and images for new forms of communication.

Through the testing of prototypes it was discovered that a tool of this kind had the potential to stimulate conversation and memory for people with dementia, the project recieved funding and support to be developed into an app and method of care for people with living with dementia.

Project background

“See What I Mean” began with an exploration into the world of semantics, the interplay of word and image meaning. It examined the way learning, using a language and reading has on neural networks in the brain, how this shapes our cognitive abilities and in turn, our experience of the world. It researched Japanese written and visual language, comics, silent graphic novels, visual literacy, stealth branding and neuroscience. The early intention was to create work that could help develop critical awareness and inspire discourse around the credibility of visual information communicated to us through various mass-media.

One of the first prototypes using the program Processing

Ilyanna experimented with and worked across a range of mediums, from drawing and image making through to moving image and coding. She collaborated with a computing professor and together developed a series of programs that turned real-time conversation into images. The interaction the programs created was surprising, it changed the way people related to and understood each other through affording users to examine the relationships between visual language and oral language. “See What I Mean” evolved into an interaction design research project as it was found to have many potential uses through extensive experimentation and testing with a wide range of potential users which led to the discovery of its potential to be a powerful tool in helping to stimulate memory and interaction for people affected by dementia.

An experimental prototype - images appear connected to the words spoken by the individual uses image data from their social media

An early projected prototype

See What I Mean in use at Jewish Care

“See What I Mean” received funding to become a start-up social enterprise, to develop the program into an iPad app and method of care, to support communication for people living with dementia. SWIM was incubated for a period at the acclaimed tech-for-social impact - investment organisation “Bethnal Green Ventures”. See What I Mean secured a working partnership with one of the largest care organisations in the UK - “Jewish Care” to develop, test and pilot the project with people with dementia and their carers. The cognitive impairments caused by dementia are complex, therefore it was essential to take a user-centred approach in the development of the tool. With evidence of the positive impact SWIM was having we were encouraged to secure further grant funding and support from “Unltd” and follow on investment from “Bethnal Green Ventures”. SWIM was also awarded funding and support through a fellowship with “Impact Hub” and we received a Design Innovation Award from the Royal College of Art. A partnership with Getty Images allowed us access to a wealth of images for use in our app, as the app improved, so did its impact for people with dementia and it was demonstrated and scaled within Jewish Care.

See What I Mean connnected to a projector for a group session at Jewish Care

Although resource stripped social care services meant “See What I Mean” struggled to become a sustainable social enterprise in the long term, it has been shown globally as an example for the potential for innovative design to improve the lives of people with dementia through its potential for unlocking new forms of communication.  “See What I Mean” was presented at The Aging 2.0 - Summit for Innovation in San Francisco, USA, The Global Impact Forum in Zurich, Switzerland, and the Cavendish Global Impact Health Forum at Oxford University, UK, among many others. It was also exhibited at the Viennese Museum Postparkasse in the exhibition “Design Diversity”.

© Ilyanna Kerr