Along with Professor Dragan Gasevic here at Edinburgh, and Dr Jen Scott Curwood, Associate Professor Abelardo Pardo, and Dr Amani Bell from the Centre for Research in Learning and Innovation at the University of Sydney, this year I’ve been awarded a Partnership Collaboration Award on the topic of Methodological Innovations for Assessing Learning in Digital Spaces. The project is building on research connections between the two Centres, weaving together two complementary strands: an approach to multimodal assessment and a framework to analyse learning strategies in digital spaces. Jen, Amani and I are working on the first strand, developing new insights into the nature of digital assignments and methodologies for their design and assessment, drawing on theories of place-based learning, mobilities and multimodality.
Along with getting to work closely with Jen and Amani (and returning to the lovely Sydney in March), and to think very broadly about assessment methodologies with all the partners, this project is also a brilliant chance to bring together expertise from a number of colleagues here in the Centre: James Lamb, Sharon Boyd, Yi-Shan Tsai and Sian Bayne are all contributing their insights to the multimodal assessment strand.
There’s a bit more information about this collaboration on the Digital Education site, and I’ll share findings and materials here as they develop.
Also re Artcasting, I’ve just finished and submitted a paper about hospitality and digital co-production; and the team recently reconvened with a number of others who have helped us a lot in thinking about Artcasting futures, to talk about a whole bunch of fascinating issues still to be explored. There is a summary of these on the Artcasting site blog.
Beyond Artcasting and its ripples, London also allowed a bit of time for Koula Charitonos and I to talk through some ideas – including to propose a symposium for next year’s ICLS conference on museum visitor experiences in the digital age. We hope this will be able to come together!
Serendipitously, I’ve got the opportunity to supervise not one but two masters students this year who are exploring aspects of 3d printing, scanning and visualisation in cultural heritage contexts – I look forward to learning lots from both of them about this topic.
I’m spending the day with a fantastic group of artists, youth organisation representatives, researchers and arts experts for a workshop at the Scottish National Portrait Gallery, aiming to collaboratively explore and develop ideas for a project to explore the potential of contemporary art to raise awareness, provoke thought and motivate change in relation to youth inequalities. I’ll post some notes and other materials later, but we’ll be using the hashtag #artineq.
Abstract: One of the most significant tensions in the convergence of technology and education is how the promise/threat of ‘disruption’ comes up against theories, practices and structures of formal and informal education. Disruption in educational technology contexts has come to be aligned with neo-liberal discourses of efficiency, enhancement, personalisation, scale and automation; and we can be forgiven for cynicism about its critical and creative potential in education. This talk aims to reanimate the debate by reframing disruption in terms of inventiveness, provocation, uncertainty and the concept of ‘not-yetness’. Focusing on the recent AHRC-funded Artcasting project, and with other examples drawn from the work of the Centre for Research in Digital Education at the University of Edinburgh, it argues that inventive digital approaches can help us develop critical responses to assumptions about the role of the digital in contexts including higher education, museums and galleries.
This talk brings together several ideas I’ve been working on over the past few years, alongside the best colleagues anyone could hope for, some of whom are in the room today. The message I’ll leave you with is ultimately a hopeful one, so you can look forward to that. But before we get there, I need to talk about the challenges of teaching and researching in the context of other things that are going on in the sphere of educational technology. I’m going to start by telling you about the concept of ‘disruption’ and why, for a lot of us teaching and researching in digital education, it’s become a dirty word. You can play along if you know what to look out for.
So: next time you read a news article about educational technology, see how well you do in disruption bingo:
Universities and schools are broken, failing, out of date
Digital natives/millenials/?? demand, expect, deserve
Efficiency, speed, simplicity through better technology!
Personalisation/individualisation is key
Harvard Business School professor Clay Christensen has been an important promoter of ideas about how education is fundamentally broken and in need of disruption through technology. We’re not going to dwell on him for long, but his framing of disruption has been very influential. Here’s Professor Christensen writing in 2014 about massive open online courses – the huge, free courses offered by universities through platforms like Futurelearn, Edx and Coursera.
“In 2013, we witnessed aggressive discounting strategies as well as schools experimenting with lowering net — not sticker — prices in an effort to recruit students.”
“Free access to content from prestigious institutions revealed that content didn’t need to be proprietary.”
“Faculty have been forced to reassess how and why they teach the way they do.”
…“Many colleges and universities resist the idea of training students for jobs. Yet it is employers who are truly the ultimate consumers of degree-holders.”
The first three points might or might not seem objectionable to you, but I do want to draw attention to the fourth, which was buried rather far down in the article, right near the end. The idea that students are the product and employers are the consumers is a rather striking point from which to disrupt/fix/reimagine education, don’t you think?
When you see headlines like this – “many universities are resisting online learning” – I would encourage you to consider that resisting educational technology and online learning, at least in the form it’s being offered by thinkers like Christensen and the companies, institutional actors and others that embrace these types of philosophies, might be a highly principled position. We’ll come back to this.
There is literally no area of education that is immune from the idea that technology will disruptively improve it. This is from this year’s Future EdTech conference web site:
Digital Disruption in Education: Institutions have far understood that traditional means of communications are not efficient anymore when engaging with generation Y. Reaching out to recruits, current students and alumni, positioning your brand and communication can be massively assisted with the use of new tech that offers unprecedented insights and opportunities to personalise communication.
Note the corporate language and the appearance of generation y and personalisation to talk about communicating with students.
Pervasive rhetoric of disruption makes those in the education technology sector especially open to ‘the next big thing’, whatever it might be. As an example that emerged on the scene last year, I want to talk about the emergence of the concept of ‘blockchain’ for education.
Bitcoin is a digital currency in which transactions can be performed without the need for a credit card or central bank… The blockchain is a public ledger of all transactions in the Bitcoin network. – https://blockchain.info/wallet/bitcoin-faq
Blockchain is really fascinating, and if you want to learn more about it, you might like to check out Professor Chris Speed’s ESRC project called ‘after money’ (http://aftermoney.design ).
There are some big promises being made for what blockchain could do for society:
let’s put health records, voting, ownership documents, marriage licenses and lawsuits in the blockchain. Eventually, every dataset and every digital transaction could leave a “fingerprint” there, creating an audit trail for any digital event throughout history, without compromising anyone’s personal privacy. [blockchain] could introduce a level of democracy and objective “truth” to the digital world that even the physical world can’t match. Its promise involves a future in which no one has absolute power online, and no one can lie about past or current events. (http://recode.net/2015/07/05/forget-bitcoin-what-is-the-blockchain-and-why-should-you-care/ )
Education is no exception. This is a design fiction, based on what the creators called ‘edublocks’. This video was made by the Institute for the Future and the Act Foundation to generate discussion and debate, so we can assume it is aiming to be at least a bit controversial. Bear that in mind as we enter a world where ‘learning is earning’.
I think that pervasiveness of the discourse of disruption means that it is something educators and educational researchers have to grapple with. I’ve hinted at some of the difficulties I have with the underpinning values being expressed through the concept of disruptiveness. Now I want to make explicit what we need, as digital educators, scholars and researchers, to take into account if we want to reconfigure this debate.
Neil Selwyn offers a useful set of critical questions we can ask when faced with calls for disruptive educational technology:
What is actually new here?
What are the unintended consequences or second-order effects?
Who is pushing these ideas in education? What are their reasons for doing so? What wider agendas are attached to these conversations?
What is being said about education that might be useful? What is being said about education that teachers might wish to challenge and talk back to?
(Selwyn 2015, p183)
We shouldn’t necessarily take claims about disruption at face value – I would argue, with Sian Bayne, that a lot of the current ideas about disruptive technology in education are popular not because they are so radical, but precisely because they leave what she calls ‘deeply conservative assumptions’ about education in place:
[Technology Enhanced Learning] carries with it a set of discursive limitations and deeply conservative assumptions which actively limit our capacity to be critical about education and its relation to technology. At the same time, it fails to do justice equally to the disruptive, disturbing and generative dimensions of the academy’s enmeshment with the digital. (Bayne 2015, p.7)
Rhetoric of disruption, like that of enhancement, doesn’t often signal a willingness to grapple with what is genuinely ‘disruptive, disturbing and generative’. Lesley Gourlay suggests that one of the big ‘disruptive’ ideas in education technology, openness, often carries with it some really troubling fantasies, including one of moving away from engagement with contestable knowledge, towards access to content:
“The fantasy [of openness] appears to be one of total liberationfrom the perceived constraints of formal study, the rigours of assessment and engagement with expertise and established bodies of (contestable) knowledge, all of which are activities deemed hierarchical and repressive of creativity. The emphasis is instead reduced to access and the online generation of ‘content’ – which carries with it a further powerful fantasy of unfettered human potential which can be unlocked unproblematically in informal lay interaction. (Gourlay 2015, p.8)
(Jeremy Knox has also done excellent work around the problems of framing openness as ‘access alone’ – see references for more on this.)
Audrey Watters – talking about Sebastian Thrun (Udacity for-profit educational platform founder), and the context of market demands, precarious labour and automation – has a lot to say about the ‘uberification’ of education. As she points out, Universities are holding on to accreditation and other protections… for now (but see blockchain above).
Too often, what we see when we go digging into disruption is an impulse for change that isn’t honest about its assumptions, and that can be destructive. That’s why, for me, the most important thing about working in scholarly areas that focus on the digital is how we bring together sharp and critical analysis with creativity. That is: how do we work in a way that shows how things are, but also makes space to actively explore how they might be otherwise? What kinds of provocations can help us do this? I think what we need are approaches that are more surprising, and subversive, and imaginative.
I first started thinking seriously about the role of provocation in 2011, when my colleagues and I were working on a project about online writing, assessment and feedback. We thought about writing some principles, but we decided that what was needed in our field at that time was not ‘principles’ or yet another version of ‘best practice’, but something altogether more provocative – a manifesto. The 2012 and 2016 versions of the manifesto for teaching online have been really important sources of discussion, debate and inspiration for us and our students, colleagues and others in the field. The series of short statements are grounded in our own and others’ research, and they open up a range of critical questions and ideas about digital education, framing these in active ways (we tried not to simply repudiate everything). This video interpretation of the manifesto, made by our excellent colleague and PhD student, James Lamb, gives a good way in.
The manifesto and an invitation it generated in 2012 led directly to meeting Dr Amy Collier, now at Middlebury College in Vermont. Amy and I have been working for the past few years on what we’ve found to be a generative way of thinking about the future of educational technology. As Neil Selwyn points out:
‘technologies are subjected continually to complex interactions and negotiations with the social, economic, political and cultural contexts into which they are situated.’(Selwyn 2012, 214–15)
The category of ‘emerging technologies’ is especially complex, since it brings the future into the equation. Our colleague George Veletsianos talks about these as “not yet fully understood” and “not yet fully researched, or researched in a mature way” (Veletsianos 2010, 15). Amy and I have coined the term ‘notyetness’ to capture something of this complexity and uncertainty:
Practices, identities, pedagogies and technologies can be marked by this ‘not-yetness’ (Ross & Collier 2016)
Not-yetness works in the service of a messier understanding of what constitutes higher education, and how technologies act in this space; and it engages with complexity, uncertainty and risk, not as factors to be minimised or resolved, but as necessary dimensions of technologies and practices which are unknown and in flux. We want to continually centre this complexity, mess and uncertainty when we think about the future of education and its technologies.
For me, that applies equally to research, and I’ve been developing my own understanding of what that means over the past few years. It’s helping me understand what ‘creatively critical’, provocative methods can do when applied to digital education. The best way I’ve found so far to think about this is through the lens of speculative method. These methods are increasingly used in design disciplines, and they include approaches like design fictions. They’ve also been taken up in the social sciences in a range of conceptual and empirical ways. Speculative (or inventive method)
is ‘explicitly oriented towards an investigation of the open-endedness of the social world. … the happening of the social world – its ongoingness, relationality, contingency and sensuousness’ (Lury and Wakeford 2012, 2).
is aimed at envisioning or crafting futures or conditions which may not yet currently exist.
provokes new ways of thinking and brings particular ideas or issues into focus.
may blur boundaries between research, design and teaching.
involves considerations around epistemology, temporality and performativity.
For me, three key elements to speculative method are epistemology, temporality and performativity. I’ve written about this in a recent journal article in Learning Media and Technology which you might like to check out if you want to read more.
These methods challenge linearity and replicability – and they are explicitly about the kinds of questions being asked. Questions about the future, provocative questions, questions that create their own conditions of answerability – conditions that didn’t exist before:
the ‘answerability’ of a problem is introduced by crafting a method specifically to address that problem. (Lury & Wakeford 2012)
Wilkie, Michael, and Plummer-Fernandez (2015) argue that methodology itself is ‘a process of asking inventive, that is, more provocative questions’ (p.4)
Speculative methods are also, in their focus on the future, very enmeshed in ideas about time. Visions of the future generate effects in the present, and our fictions and inventions are shaped by issues we inherit, and closed off from futures we can’t yet imagine. Furthermore, the effectiveness of inventive methods ‘cannot be secured in advance’ (Lury & Wakeford 2012).
Importantly, speculative method relies on engaging with and provoking various kinds of publics – and how those publics, or audiences, or participants respond to ‘objects to think with’ determines the nature of the problem and its answerability.
I’m going to spend the last part of my talk sharing some examples of projects in the Centre for Research in Digital Education that have used speculative method to introduce creative criticality into areas that really needed it! I’m going to mention two briefly, then talk in a little more detail about the third, the Artcasting project.
The second project I want to mention is the “learning analytics report card”, or LARC, project, led by Jeremy Knox: https://larcproject.wordpress.com . LARC was designed to explore how Universities can develop critical and participatory approaches to educational data analysis. It did this by creating an interface for data from the Moodle virtual learning environment, where students could chose the types of reports they wanted to have generated about them, and would get ‘plain english’ results that we’ve seen from engaging in them with students can be really generative and thought-provoking. By revealing data that is usually hidden, and presenting it in playful and provocative ways, the project is getting people more critically engaged with ideas around algorithmic culture and educational policy.
Finally, I want to tell you about the Artcasting project. This was an AHRC funded project that set out to find more inventive ways of evaluating how visitors engage with artworks. We did this by designing and trialling a mobile method for inviting, capturing and analysing people’s connections between art and place through an app we called artcasting. The academic team was Chris Speed, Claire Sowton, Jeremy Knox and me, with Chris Barker providing the software engineering, and we worked with National Galleries of Scotland, Tate and the ARTIST ROOMS programme over the course of a year.
is jointly owned and managed by Tate & National Galleries of Scotland
a collection of more than 1,600 works of international contemporary art was acquired in 2008 by National Galleries of Scotland and Tate.
is shared throughout the UK in a programme of exhibitions organised in collaboration with local associate galleries.
aims to ensure the collection engages new, young audiences.
Evaluation is a really thorny issue in the cultural sector at the moment, and there has been a lot of critique of instrumental approaches to evaluation and to measuring engagement, learning and impact. Belfiore and Bennett are two of the key people working on this, and they are pretty stark about the issues at stake.
with the present levels of knowledge around aesthetic reception, it is not possible to make any meaningful broad generalization about how people respond to the arts, and if or how they might be affected by the experience. Even less plausible is the possibility of actually “measuring” any of these aspects. (Belfiore & Bennett 2010, p.126)
Claire Sowton brought these and a lot of other critical perspectives on evaluation to the project from her doctoral work.
Galleries and museums are generally good at thinking about how people get to the building, but other kinds of movement are less visible. We were hugely interested in understanding museum and gallery learning from a theoretical perspective that took into account social, spatial and technological mobilities. We saw the movement of artworks, people, ideas, inspiration and technologies as a really central part of what it means to understand the impact of art on visitors.
So we built this app as a way of testing out whether more inventive ways of approaching evaluation could be productive for visitors and for gallery professionals. Artcasting was developed as a methodology that could capture articulations of engagement with artworks. We tested it out in two exhibitions – Robert Mapplethorpe at the Bowes Museum in Co Durham, and Roy Lichtenstein at the National Galleries of Scotland. Read more about how Artcasting worked: https://www.artcastingproject.net/what-is-artcasting/
In total we ended up with about 170 casts that we could work with to try to understand what this approach could do. Here are a few examples of the kinds of artcasts people sent.
The Artcasting data could be seen collectively as more or less stable ‘traces’ of memory, insight and message. But we also explored how those traces can be a volatile assemblage of engagement that can move in multiple directions away from a single point (for example a single artwork cast to many places and times). Interpretation, like the objects that spark it, is ambiguous and shifting – and the impacts that artworks and galleries have are never going to be easy to pin down.
“it is always possible to take an individual object and place it in a new framework or see it in a new way. The lack of definitive and final articulation of significance keeps objects endlessly mysterious – the next person to attach meaning to it may see something unseen by anyone else before.” (Hooper-Greenhill 2000, 115)
This is really consequential for how institutions talk about and engage with evaluation, and this project has helped us and our partners and others in the sector to think more imaginatively about these issues.
I’ve also been working on how to conceptualise ‘digital co-production’, using ideas about artcasting as a stimulus. Artcasting content is requested and is able to be interpreted by gallery professionals for accountability, audience development, and other purposes. But artcasting is also a form of public interpretation of the artwork, and visitors are creating new encounters with art in new places and times. The guest becomes the host of a new exhibition. This has implications for what I’m calling ‘digital co-production’, which
For me, this makes it even more important that we use technology provocatively rather than instrumentally, to explore big questions and possible futures, and to challenge assumptions. Some of those assumptions seem to be baked right into the DNA of devices, services, and organisational policies, but I believe there is (in the immortal words of Leonard Cohen) a crack in everything, and that’s how the light gets in. I’d like to encourage us all to keep working on how to keep being creative, to keep doing things, making things and trying things – as a way of bringing criticality to the urgent and interesting questions we are facing.
Bayne, S. (2015). What’s the matter with ‘technology-enhanced learning’? Learning, Media and Technology, 40(1), 5–20.
Bayne, S. (2015). Teacherbot: interventions in automated teaching. Teaching in Higher Education, 20(4), 455–467.
Belfiore, E., & Bennett, O. (2010). Beyond the ‘Toolkit Approach’: arts impact evaluation research and the realities of cultural policy-making. Journal for Cultural Research, 14(2), 121–142.
Collier, A., & Ross, J. (2017). For whom, and for what? Not-yetness and thinking beyond open content. Open Praxis, 9(1), 7–16.
Gourlay, L. (2015). Open education as a ‘heterotopia of desire’. Learning, Media and Technology, 40(3), 310–327.
Hooper-Greenhill, E. (2000). Museums and the Interpretation of Visual Culture. Abingdon: Routledge.
Isard, A., & Knox, J. (2016). Automatic Generation of Student Report Cards. In The 9th International Natural Language Generation conference (p. 207).
Knox, J. (2016). Posthumanism and the Massive Open Online Course: Contaminating the Subject of Global Education. Routledge.
Knox, J., & Ross, J. (2016). ‘Where does this work belong?’ New digital approaches to evaluating engagement with art. Presented at the MW2016: Museums and the Web 2016, Los Angeles. Retrieved from http://mw2016.museumsandtheweb.com/paper/where-does-this-work-belong-new-digital-approaches-to-evaluating-engagement-with-art/
Lury, C., & Wakeford, N. (2012). Inventive Methods: The Happening of the Social. London: Routledge.
Ross, J. (2016). Speculative method in digital education research. Learning, Media and Technology, 0(0), 1–16.
Ross, J., & Collier, A. (2016). Complexity, mess and not-yetness: teaching online with emerging technologies. In G. Veletsianos (Ed.), Emergence and Innovation in Digital Learning: Foundations and Applications. Athabasca University Press.
Ross, J., Sowton, C., Knox, J., & Speed, C. (in press). Artcasting, mobilities, and inventiveness: engaging with new approaches to arts evaluation. In L. Ciolfi, A. Damala, E. Hornecker, M. Lechner, & L. Maye (Eds.), Cultural Heritage Communities: Technologies and Challenges. London: Routledge.
Selwyn, N. (2015). Never believe the hype: questioning digital ‘disruption’and other big ideas. Teaching and Digital Technologies: Big Issues and Critical Questions, 182.
Selwyn, N. (2012). Ten suggestions for improving academic research in education and technology. Learning, Media and Technology, 37(3), 213–219.
Veletsianos, G. (2010). Emerging Technologies in Distance Education. Athabasca University Press. Retrieved from http://www.aupress.ca/index.php/books/120177
A paper based on the research conducted as part of the Dissertations at Distance project has just been published in Teaching in Higher Education. The paper introduces the idea of the ‘campus imaginary’ as a way of accounting for the tendency of online students to attribute difficult or challenging experiences of independent research to their distance from the campus.
[Campus imaginaries portray] the imagined institution and those in it as approachable, sociable, and a space more amenable to the sorts of activities interviewees found themselves undertaking as part of their dissertation. In part these imaginaries draw on assumptions about the advantages of ‘in-person’, as opposed to virtual, contact which are far removed from the experiences described in the literature on campus-based students’ independent research experiences. A number of interviewees, even those whose overall experiences of their online programmes were extremely positive, appeared to ascribe negative experiences to their status as online distance students. For example, issues such as unexpected obstacles, troubles with motivation, difficult supervisory relationships, lack of time and space to focus, and feelings of isolation and doubt were interpreted as features of the online dissertation process specifically. As we will see, however, these issues are common features of transitions from taught courses to independent study; the nature of supervisor-student dynamics; and the inherent challenges of conducting research, particularly for newer researchers.
Thanks in part to the discussions at the conference (some reflections on these by @cindyu , David Kernohan), we’ve been continuing to develop these ideas, and this article is the result. It takes a range of critical perspectives on openness as its starting point, and offers the concept of ‘not-yetness’ as a productive lens for examining meanings of openness that go beyond instrumentality, uniformity, and content.