Mackenzie Jorgensen interviews Eli Lee (part two)

I wanted to ask about Janetta and her research into AI and emotion. There’s been a lot of research done into emotion detection, and a lot of critique. For example, what would it mean for a machine to ‘objectively’ know your emotions, when you may not even know yourself?

Yes. In the novel, Janetta is aspiring to teach AI about emotions, but she’s learning about emotions herself. She’s had a break-up and a rebound with someone who inspires her, but destabilises her as well. This experience is difficult but it helps her come into her own. She was a very unemotional person before that – she tried not to have emotions; but it turned out that she did.

So in that sense, the novel is more about Janetta being at peace with having emotions. Rather than the idea that emotional intelligence in auts is ever going to happen. I knew that it would be a novel about gaining emotional intelligence – but it was always meant to be in Janetta, someone who needed to do this.

You definitely see that growth throughout the novel. It’s such a hard thing to learn, but so important. Emotional intelligence, being able to be vulnerable, all of those things.

Thank you, that’s exactly it. Janetta has never been vulnerable. She’s used her work as a shield. I wanted this to be a story about being vulnerable, about screwing up, and about bringing yourself back from that.

“Do you think that AI can be taught to read emotions?”

Right, exactly. But it seemed like Janetta believed it could be done. So I was curious about your views.

I just don’t think it can be done at all, full stop. The research that’s been done, some based on facial recognition. One person could be smiling, but they could be desperately sad inside. Could an AI detect that? Humans don’t just detect emotions by observing from a distance. We interact, we probe, we learn. We use our own emotions to invite others how to feel theirs.

So yes, maybe AI can be trained in intersubjective standards of emotion recognition, enough to make reasonable ascriptions. Let’s say, to take a pretty clear emotion, in King Lear when Lear comes back on stage at the end, carrying the body of Cordelia, his beloved child. What does he say? “Howl, howl, howl, howl!” The majority of people can piece the evidence together and understand that he’s upset.

An AI could learn to do that. But in terms of the intricacies of people’s emotions, the depth and the context of them? No, I don’t think so. But what about you? Do you think that AI can be taught to read emotions?

I think researchers will continue to try, but I don’t think it’s really possible. Like you say, someone can be smiling yet struggling inside. And I think the attempts to develop that technology may do more harm than good – in relation to surveillance, for example.

I was thinking about care homes where they have companion AIs, seals and cats and things. That certainly has therapeutic potential. Otherwise, I don’t know how it could possibly read the nuances of human emotion. We don’t even understand our own behaviour sometimes!

I think with a lot of AI, the technology and the science behind it is very interesting. But at the end of the day, the real questions are around how it’s used. Who holds the power? Who has the data that it’s being trained on? That has a major, major impact.

Is that what you’re looking at in your PhD research?

I’m looking at Machine Learning classification settings. So an example of a binary classification setting might be, “Oh, we think this person will repay the bank if given a loan,” versus, “We think this person will default on the loan.” I’m exploring the potential delayed impact of a classification. For example, if you are a false positive, if an AI predicts you’ll repay but instead you default, then your credit score will probably drop. So there will be a negative impact on you too, even though you were given a loan.

How do you investigate this?

There isn’t much data, and it isn’t easy to track. It involves a lot of presumptions, and running simulations, and giving more weight to the false positives and the false negatives. I’m trying to understand, “Okay, maybe in these problems, we need to really focus on the false negatives, versus in these ones, the false positives.” Essentially, I’m exploring how we might mitigate the harm an AI decision has on a person. Also, I’m interested in investigating the impact on underrepresented or underprivileged groups, because we have a lot of issues with AI classification systems learning bias and perpetuating sexism and racism, for instance, from our society.

Is it a generally done thing? Say it was about applying for a loan – can the bank automatically exclude the people the AI doesn’t like, because they haven’t got enough income, or their credit’s bad, or because of some other factor?

“Algorithmic fairness has been a field that has really boomed recently, but it’s been around for a while.”

Sure. Algorithmic fairness has been a field that has really boomed recently, but it’s been around for a while. It came into the light in the ’60s and ’70s, when a lot of Civil Rights work was being done. At the time, the focus was on education and employment settings. Nowadays, it’s still focused on those settings, but also in areas like finance and economics, and many others.

That’s really great, you’re actually doing something that’s potentially making a difference in people’s lives. People who do AI (rather than just write about it in novels!) blow my mind. It’s impressive to have a brain that can do data, logic and mathematics – I’m very jealous.

No, I mean, I think anyone can code and learn about it. I know it seems as if it’s unattainable, but…

That’s a good point. I could learn to code, potentially!

Well, we do need more women in this area, so … ?

I suspect I’ll never get around to it…

What’s your background? Did you do English?

Yeah, I did English at uni, and since then, I’ve worked in editing, comms and publishing. I wrote three novels before this one, but I never sent them to an agent because I thought they weren’t very good. All I ever wanted to do was become a writer, so I’ve ended up with a narrow range of competencies. Writing and editing, essentially. But if that gets automated … what’s it called, GPT-3?

One of the big language models?

Yeah. Janetta’s job, for example, is safe from automation. Right up until AI is able to start consciously self-replicating – like in the movie Her, that sort of singularity moment – Janetta’s job is safe. But in my day job, where I edit publications, how safe is that? My skills are going to become obsolete soon. I might give it fifteen, twenty years. But that’s all I’ve ever trained myself for. It’s not an everyday worry, but it is a distant worry.

I think creativity, especially with regards to novel writing, is not something I can see an AI doing. They most likely would only be re-making other people’s ideas that they were trained on. I think being a creative thinker is a great spot to be.

That’s definitely the pragmatic view! I think the kind of deeply pessimistic, slightly addled-with-dystopia view is that they’re going to be able to recreate Madame Bovary within thirty years, and then all writers will be out of a job.

But yes, I think the greater question is around how AI might transform creative expression, rather than take it over. There will undoubtedly still be ways for us to bring our humanity to books and music and art.

Realistically, AI is everywhere.

“Realistically, AI is everywhere.”

Right, right. And going back to the novel, you really showcase auts in hospitality settings. Is that the main place that you see them potentially going? Or do you see them in other settings?

Realistically, AI is everywhere. It’s in our Netflix algorithms, and it’s in our traffic lights. So in that sense, I didn’t portray reality – I didn’t convey all the hidden AI that shapes everyday life. In terms of hospitality, I guess there’s already automation in the supply chains and the logistics, and places like the Ocado warehouses. I don’t know if you know about Ocado, the delivery company that went really heavily automated?

Yes.

Ocado has one of the most automated picking and packing systems in the world; these robot arms just picking up ketchup and putting it in bags! So I touched on that a bit, but yes, mostly cafés. Have you ever read Cloud Atlas by David Mitchell?

David Mitchell, the comedian?

Yet another uncanny double. There’s two David Mitchells. There’s the Peep Show comedian, and then there’s a novelist who doesn’t really write sci-fi, but he wrote a novel called Cloud Atlas, and there’s a chapter in a very futuristic setting. And I read it when I was quite young, and the imagery from it, where the utopia that masks a scary dystopia beneath, has stuck with me ever since.

Also, I love coffee shops. Coffee shops are so warm, cozy and human. I just knew that robot servers in a café were a way to have a real interface with humans coming in to get their coffees and being hit with, once again, uncanniness and unnerving futurity, and a slightly utopian, slightly dystopian vibe. In the novel, one of the characters, Van, sings while he works, and I imagined the coldness of that being replaced by auts.

I’m also someone who loves coffee shops. Their ambiance and conversations with the barista are two of my favourite things about them. They’re always in a very warm setting.

Coffee shops are a classic institution. You’re from Seattle, right?

Yes, I am.

The home of coffee shops!

Yup.

You have the best coffee – all of Seattle is like one big coffee shop. And then you know exactly; a good coffee shop is the most wonderful place.

Can we expect a sequel?

Potentially! I’m curious, would you see it as a free-ranging AI utopia, where they’ve managed to create this benevolent AI that’s also autonomously functioning?

I guess I wondered about Lal’s decision in the last chapter, and seeing what that actually does to Tekna and their world.

That makes sense. To be honest, I found writing this novel so difficult. I’d written a sci-fi novel before, and I think the reason it was difficult was because, well … do you read a lot of sci-fi?

Honestly, this was my first sci-fi book! I’m usually a non-fiction person. Currently, I’m reading non-fiction, Caste: The Origins of Our Discontents by Isabel Wilkerson, which is really good. Very different from this though.

I read quite a bit of literary fiction – writers like Elena Ferrante, Alice Munro. But I feel almost compulsively drawn to sci-fi, like it’s where my imagination wants to go. I realised during the writing of this that it had to be driven by plot, and then the characters react to that. So the automation and conscious AI plots were the engines of the novel.

Right.

And I wonder if I’m better suited to something where the engine is people living their lives in a more scaled-down way. I haven’t worked it out yet; I only know that when the Guardian called A Strange and Brilliant Light ‘character-driven’, they weren’t wrong!

Having written the novel, though, what are the major takeaways that you want readers to come away with?

I know it sounds potentially counterintuitive, because the novel is about AI. But I think to me, it is more about some of that more mundane, slow-burn stuff. It’s about figuring out who you are and allowing yourself to make messes and wrong choices, and then being able to do something about this. So all three of the girls do make pretty terrible choices, and then they manage to figure out who they need to be in order to make things better. So it’s that hokey stuff about being true to yourself, and having faith in yourself. Because even Lal shows she has faith in herself, in the end.

The other message is about vulnerability and emotional intelligence. Lal shows that at the end, because the person she needs to be vulnerable to is her sister. And Rose needs to stop being vulnerable to powerful men and put some boundaries down. Vulnerability and self-assurance are connected.

It’s a feminist novel. When you’re in your twenties, you go through a lot of self-doubt. Most people I know, unless they’re bizarrely confident, struggled quite a lot internally with who they should be and whether they’re doing the right thing, especially in their twenties. And I wanted to show some women who also struggle, but manage to figure things out.

“I wanted it to be about AI and automation, and I wanted to focus on class”

I loved that. Yeah, the emotional intelligence definitely was shining throughout. And yeah, it did seem like quite a progressive future, which was really cool to see, and very feminist as well.

I’m aware that there are other contemporary feminist issues it could have taken up. It could have contained more trans representation, for example – it could maybe have been more explicitly intersectional. I chose to not mention the main characters’ racial identities, too, beyond them being Iolran.

Yeah, I noticed that.

I think I knew that I wanted it to be about AI and automation, and I wanted to focus on class – you know: “let’s talk about class.” That doesn’t mean I wanted to ignore the other stuff, but not every book can be everything and this novel already packs so much in! And class and economics are deeply worthy of sustained focus, too.

Janetta is a queer character, but her sexuality is in no way definitive of her entire character.

I wanted it to not be an issue at all. There was a flashback scene that I ended up cutting, where she came out to her parents and they were totally unphased. Partly I felt like, as a straight person … it’s not that I can’t tell that story, but I asked myself: how qualified am I to tell this story?

And related to that, I was cautious of making it Janetta’s main thing. I really built her character around her genius. I wanted her to be a visionary and not be hampered by anything other than her own emotions, and her fear of her own emotions. So that’s why being lesbian was just part of her, and not a big deal.

I liked that she was still in love and dealing with those relationships throughout the novel as well.

Thank you. I worried I made her too involved in relationships. But then I thought, but that’s the point. Because she needs to learn how to love and how to grieve. That’s how she becomes the person she needs to be.

Well, speaking of vulnerability, it’s very brave of you to keep going and actually get it published.

Thank you. I think I reached a point where it was like, “Oh, this is the fourth novel, and it’s now or never.”

And you’re still interested in writing fiction?

Yes, definitely still speculative fiction. But I’m aware that when you write speculative fiction, you have to be open to your imagination going to unexpected places. At first the novel was only about automation. As I went along, though, I realised that when you write fiction about AI, you’re naturally drawn towards the idea of conscious AI – at least, I was.

I could have written a smaller and more focused novel, but to me, the singularity is an irresistible part of the collective imaginary about AI! And this made things very complex, plot-wise. There was an arc about automation and the loss of jobs, and a second one about conscious AI, and interweaving them was hard!

Before we go – with conscious AI, do you think we should be striving for that, or should we not?

No. It’s fun for movies and books, but that would be a crazy world, no?

Agreed. Yup. We’ve got a lot of problems we need to solve already in the world today. Climate change, poverty, hunger. I don’t think we need a conscious AI to stir the pot even more.

Exactly. Do you think it’s ever likely to happen, though?

I think it could happen. I mean, people are working in that space for sure, but I don’t know if we’ll exactly know when it does. It would probably happen by accident, and surprise people. I think it’s a possibility, but I’m not keen for a world where that does happen.

I couldn’t agree with you more.

Well, Eli, this has been wonderful speaking to you.

Thank you, it’s been really enjoyable. And your questions were excellent – it’s nice to have what you’ve written about reflected back at you by someone who asks such intelligent, thoughtful questions! So yes, thank you, that was great.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

Automation and Utopia: Human Flourishing in a World without Work by John Danaher

Reviewed by Michael Pitts.

Danaher, John. *Automation and Utopia: Human Flourishing in a World without Work*. Harvard UP, 2019. Hardcover. 248 pg. $99.95. ISBN 9780674984240.

Automation and Utopia: Human Flourishing in a World without Work is crafted as a response to fears over an automated future in which humans are made obsolete by technological developments. Written by John Danaher, senior lecturer of law at the National University of Ireland, Galway, the text consists of two main sections, which cover automation and the possibility of a utopian future, respectively.

After outlining the scope and purpose of his research, in the first chapter Danaher forecasts the obsolescence of humankind in an automated world. But this is not as catastrophic as it may sound since, for Danaher, “Obsolescence is the process or condition of being no longer useful or used; it is not a state of nonexistence or death” (2). In the rest of the automation section, Danaher responds to two propositions: that automation in the workplace is both possible and desirable, and that automation outside of the workplace is potentially dangerous and its threats must therefore be mitigated.

After making his case for why automation should be conditionally embraced, in the second section Danaher turns to two possible, ‘improved’ societies with automation fundamental to their economies, the cyborg and virtual utopias. While the cyborg utopia enables humankind to remain valuable members of the economy, occupying the cognitive niche that has historically provided an initial evolutionary advantage to the species, Danaher posits that such a future will likely maintain the degradations of employment, enhance our dependency upon machines, and disrupt humanist values while, due to the technological advancements it requires, ensuring no worthy improvements to human wellbeing in the near future.

Following up this analysis of the cyborg polity, Automation and Utopia concludes with a presentation of what Danaher views as the ideal, improved society, the virtual utopia. This improved society, in which humankind ventures into the virtual world to enhance its flourishing, is presented by Danaher as an ideal goal towards which humankind may aim since, as the author posits, it will ensure human agency, pluralism, stability, a myriad of alternative utopias, and a meaningful connection to the non-virtual, real world.

Pivotal to Danaher’s assessment of automation, and a possibly utopian future, are his views on labor and the avenue he identifies as optimal for human flourishing, the virtual utopia. For the purposes of his argument, he adopts a definition of work which he acknowledges as unusual and likely controversial, since it excludes “most domestic work (cleaning, cooking, childcare)” as well as “things like subsistence farming or slavery” (29). Defining work as “any activity (physical, cognitive, emotional etc.) performed in exchange for an economic reward, or in the ultimate hope of receiving an economic reward,” Danaher builds the case that obsolescence is almost certain and could result in as low as 10% or as high as 40% of the future population remaining employed (28). Such a development is framed as a positive result since work, he emphasizes, has a negative effect upon employees and improving it in the current economic milieu is, according to him, a more difficult route to take than shifting towards a virtual utopia. Specifically, Danaher argues that improving work, which often involves fissuring, precarity, colonization, classic collective action, domination, and distributive injustice is unlikely in our current system since it “would require reform of the basic rules of capitalism, some suppression or ban of widely used technologies as well as reform of the legal and social norms that apply to work” (83). Though this dismissal of the possibility of improving working conditions is short-sighted and ignores the likelihood that labor organizing will prove necessary as technological advances continue, this weakness of the text stands on its periphery. More important to Danaher’s vision of the future is his adoption of an approach that is interestingly more radical than such efforts to protect workers: the introduction of a universal basic income and the normalization of technological unemployment in current economic systems.

Danaher envisions this radically different distribution of economic power as a salient feature unique to the virtual utopia. Danaher rejects the cyborg utopia, believing it will threaten the prospect of universal basic income and technological unemployment and ensure the continuation of work and the injustices endemic to capitalistic systems. In considering the virtual utopia, Danaher’s audience must consider the ethics and consequences of a nation in which utopian games and escape become a salient feature of its culture. This ideal society is marked by its focus upon virtual worlds as the mechanism by which human flourishing may take place. By venturing into simulations that are shaped to satisfy the desires and needs of individual users, it avoids the problems of a single utopian ideal that must be enforced upon all citizens. It can therefore, as Danaher explains, “allow for the highest expressions of human agency, virtue, and talent… and the most stable and pluralistic understanding of the ideal society” (270).

Yet as with the cyborg utopia, the virtual utopia is plagued with ethical complications. The question of what actions are permissible in such a simulated environment is closely related to the ethical considerations surrounding cyborgs and artificial intelligence. In very briefly confronting this topic, Danaher asserts that the same moral constraints that shape human interactions in daily life will impact those occupying the virtual world. He supports this argument by pointing out that some of the characters inhabiting the simulation will be operated by human players and that interactions with such players will have ethical dimensions. In addition, he asserts that other actions may be deemed intrinsically immoral even without a corresponding ‘real-life’ consequence. Danaher asserts that, though there will be some moral frameworks unique to the virtual utopia, there will be no major alteration to human ethics. The virtual utopia, he claims, is therefore a reasonable goal for the post-work society since it enables human flourishing and protects values such as individualism and humanism.

Danaher is also keen to emphasize that “the distinction between the virtual and the real is fluid” (229). He rejects the “stereotypical” science fictional view of virtual reality, as something that is only produced within immersive technological simulations, like the Matrix or Star Trek’s Holodeck. On the other hand, he also rejects the “counterintuitive” view that everything humans experience is virtual reality in that our reality is constructed through language and culture. Instead, Danaher offers a middle position. Some things may be more virtual than others, but nothing is wholly virtual or wholly real. He sees virtual utopia as being filled with emotionally and morally meaningful interactions, but in the context of relatively inconsequential stakes (rather than survival, or struggle for hegemony). A Holodeck-style simulation is only one of many ways this could be accomplished.

Automation and Utopia delves significantly into the topic of possible futures at the intersection of ethics, technology, and humanism. It is a valuable resource for scholars, students, and laypeople engaged with conversations surrounding the advancement of automation in the 21^st-century, its impact upon economics and workers, and optimal approaches to accommodating such new technologies through the advent of a post-work society. The work continues discussions at the intersection of technology and labor, but necessitates broader considerations related to the virtual utopia Danaher proposes. Namely, it does not convincingly explain how virtual utopia will avoid the ethical pitfalls outlined in relation to the cyborg utopia. It also does not thoroughly discuss how such simulations may be safeguarded from economic exploitation at the hands of those owning or operating these systems, or address the potential for intersectional inequalities. Finally, Danaher does not comprehensively discuss how such escapism and the further minimization of human interaction in the natural world may impact climate and the environment. Though it is difficult to accurately predict, estimations of both the ecological and psychological effects of a society in which the main mechanism of human interaction is not within nature but instead within a virtual world are vital to identifying optimal utopian aims.

Overall, Automation and Utopia productively dives into the topics of technological advancement and labor policy, proposes thought-provoking socioeconomic policies related to the challenges of automation, and necessitates further discussions concerning ‘the ideal society,’ its connection to technology, and the impact it may have upon human psychology and the environment.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

“Do we want that?” Mackenzie Jorgensen interviews Eli Lee

Mackenzie Jorgensen is a Computer Science doctoral researcher working on the social and ethical implications of Artificial Intelligence. We invited Mackenzie to chat with novelist Eli Lee about her debut, A Strange and Brilliant Light (Jo Fletcher, 2021), and representations of AI and automation in speculative fiction. Should we fear or embrace the “rise of the robots”? Or perhaps the robots rose a long time ago, or perhaps that whole paradigm is mistaken? How might AI and automation impact the future of work? What would it mean for emotional work to be automated? How do human and machine stories intersect and blur?

This is part one of two.

Hi Eli, I’m really excited to talk to you today. I gave myself plenty of time to read A Strange and Brilliant Light, but I ended up going through it super quickly, because I enjoyed it so much.

Oh, thank you!

So I was curious – what made you decide to showcase three women’s stories?

Well, the genesis of the three stories was unexpected even to me. When I started, I wanted to write about a pair of best friends whose lives go in different directions. That’s based on my own relationship with my best friend, who became an incredible political activist whilst I just sat around and watched TV and read books. So that was the real kernel.

But as I wrote, it felt like something was missing. Lal and Rose came to me immediately – Rose was very passionate and active in the world whereas Lal had some of my own flaws – she was bossy, ambitious, and somewhat selfish.

But the dynamic needed a third person who was a contrast to both – and that’s when Lal’s sister Janetta came in. She works in AI, and she’s driven by her own hopes and fears. Once I had those three characters, it felt complete.

Did you see parts of yourself in Lal?

I did. I felt she was a good vehicle for the parts of me I’m less proud of – so she’s a bit selfish and insecure, and she feels belittled by her older sister, stuck in her shadow and ignored, but she’s still a decent person. She wants to work to make money for her family, but she’s just more … petty!

Got it!

And then I put what I would aspire to be in Janetta. Janetta’s very self-sufficient. She’s dedicated to her work and pure of heart. She has insecurities and flaws like the rest of us, but she always works for the greater good. So I kind of separated some of my worst qualities, and the qualities I wish I had, and put them in those two.

And you made them sisters, which works well in that sense.

I’ve got two brothers, but I don’t have a sister. Have you?

No, I have a younger brother.

I mean, this is the thing. Sibling relationships can be so gendered. I wanted to investigate what it’s like if there’s an older sister who is very successful and leaping ahead academically, and then you’re the younger sister in that dynamic. What’s for you? How do you stand out – how are you different, or memorable? So that was Lal.

“I kind of separated some of my worst qualities, and the qualities I wish I had, and put them in those two.”

How far into the future did you kind of picture the novel to be?

One of the get-outs of setting it in an alternate universe is that you don’t have to specify, “This is ten years in the future,” or, “This is fifteen years in the future.” I could choose the kind of technology that fit with the plot. They’re not mind-reading, they’re using mobile phones.

To me, this says it’s not that far in the future? Eight or ten years, perhaps. I’d be interested to hear what you think, as an AI researcher, about when it could plausibly be set? When that early, deep automation of jobs is filtering through?

Eight to ten years, yeah. End of the 2020s.

Then again, part of me thinks maybe that’s too soon! You know when you watch Back to the Future II, and there’s a flying car. It’s set in 2015. We all watched it in the late ‘80s, early ‘90s, and there was this sense that 2015 would look futuristic like that. Now we’re past that date, and the changes don’t seem that drastic.

Right.

So in ten years’ time, maybe things will look the same as they do now? Maybe AI will still be in our lives, but in a way that’s similar to what it is now – essentially under the surface and hidden. Ubiquitous, but hidden. The robots still won’t be serving us coffee! So I’m willing to be proved completely wrong with my timeframe.

I think you’re good! I feel like oftentimes AI is portrayed, especially in media and films, as taking over everything in the very near future. It’s often a dystopian presentation. But actual AIs right now, they’re always just good at one thing. They’re very task-specific. We don’t really have anything like what Janetta was trying to work on, like emotional AI.

Exactly.

And there’s another question: do we want that? Because I feel like emotion is something that makes us human. At the end of the day, AI and tech are a bunch of zeros and ones. You can’t really instill that with real human emotion and experiences, in my opinion. There are scientists out there who disagree though.

I should say that, in terms of eight to ten years, I’m not talking about emotional intelligence and AI. Consciousness is way off, if it ever will happen. I think probably it won’t. But in terms of AI and automation …

Automation, yeah. No, definitely.

My friend works for an AI start-up. He often looks at stuff in my novel, and says, “What the … This is crazy!” And I say, “I know! It’s not meant to be real!” When you watch Ex Machina or Her, there’s a suspension of disbelief. But I guess as an AI researcher it must be even harder, not to just say, “Come on, come on now. That’s not going to happen!”

“Maybe AI will still be in our lives, but in a way that’s similar to what it is nowー essentially under the surface and hidden. Ubiquitous, but hidden.”

And that question of whether AI can be human is just such a long-running, fascinating topic, isn’t it? We just can’t let go of it. That uncanny other self, reflected in an AI.

Yeah, definitely. I agree with you that I can see automation coming more into play in the near future, especially with big companies like Amazon. Which is scary, because people do rely on those big corporations for jobs. We’ve seen recently that unionizing doesn’t necessarily work in those scenarios. That’s one reason Rose’s character is very interesting to me. She explores the future of social justice activism, in a near-future world increasingly dominated by automation.

I knew that you can’t talk about automation without talking about Universal Basic Income. But I didn’t want someone who straight out of the gate was like, “You guys, UBI: I’m going to sort it out.” I wanted to make sure that Rose’s activism wasn’t disconnected from the rest of her life.

So much of the novel is about these three women in their early twenties, figuring out who they are, especially who they are in their relationships. With Rose, an important part of this is how she relates to men of power, or men who have power. There’s her father, her brother, and this other guy Alek, and initially she’s unable to get out from under them.

And so she needed to come into her own power. So I thought, Rose is going to be this activist, but she’s also going to be not sure of herself initially. So a lot of it was their inner struggles, intersecting with those larger economic, social, political, or technological stories.

There was a quote I made note of. ‘Alek said, “True leisure, true creativity and true freedom are within our reach for the first time in human history. And so we must set up source gain and welcome the auts.”’ This seemed quite ironic to me because relinquishing more control of the world could seem like the opposite of freedom. And Rose did realize this as time went on, which was cool to see, as she was learning and growing.

So Alek was with these other two academics at that point in the novel. Alek’s initial point of view is: “Auts are bad, AIs are bad. We need to just destroy this stuff.” But then when these two guys come along, one of them mentions post-work utopias. John Maynard Keynes wrote about something similar in the 1930s, an essay called ‘Economic Possibilities for our Grandchildren’, and Herbert Marcus wrote Eros and Civilisation in the 1950s, and there has been lots of writing about post-work more recently.

Maybe machines can do everything, and then you can sit around and play all day, and not have to do things you don’t want to. This idea floats past Alek this evening, and suddenly he’s like, “Oh, wait! Yeah, we can just be free, because auts will do the boring stuff!”

But that’s obviously not a realistic suggestion, because if you take it a step further, like Rose does, the question is, “Who owns those auts?” Well, if it’s the corporations, that’s not freedom. So that brings Alek back to his original idea: we need source gain. We need some kind of UBI. So in that moment when he talks about post-work leisure, he’s speculating. He’s not thinking about what’s necessary now.

Can you see a world where AI grows in importance alongside human creativity and freedom? Or are they opposing forces?

In a post-work scenario, the AIs are doing the grunt work, doing the kind of cleaning and tidying, and fixing things, and all the behind-the-scenes organisational work, so humans can play and fulfil ourselves. So that’s what Alek would mean by welcoming the auts, I think. But do you mean in terms of AI more as an equal?

I guess, or at least AI growing in social importance, and taking on more and more roles?

The way Alek envisions AI, in that moment, they would be this kind of sub-caste. They’d work away in the background, and you wouldn’t need to worry about them because they wouldn’t be conscious. But I think for us, even without AI consciousness, this could still be a very unsettling and unnerving vision.

We’re already seeing that when AI creeps into more and more areas of life, that ideal of true leisure and creativity gets compromised. You’re surrounded by stuff that’s monitoring you, surveilling you, collecting and analysing your data, perhaps even filtering your reality, and steering you in various ways. It’s almost like the more AI we have, the more inhibited we might feel.

Right, and the more potential problems we might face. On the surveillance point, there’s that moment where Janetta and Taly discuss helping the government with docile spy dogs —

This is one of my cringe moments. I read it now and think, “Spy dogs? What?”

Well Boston Dynamics has a robotic dog. The New York City Police Department had a test run, and there was a huge backlash. So they said, “Okay, actually, no. We are not going to use this.” But about Janetta and Taly’s conversation, I was curious: were you critiquing how governments and the private sector collaborate over surveillance? How do you feel about that?

Attitudes about surveillance are deeply personal. I’ve got one friend who just does not care about his privacy – he’ll happily give all his data to everything and everyone. It’s not because he believes that it might make society better; he just doesn’t care. I suspect he’s not alone in that.

“We’re already seeing that when AI creeps into more and more areas of life, that ideal of true leisure and creativity gets compromised. You’re surrounded by stuff that’s monitoring you, surveilling you …”

The bird on the front of the novel, illustrated by Sinjin Li, is a CCTV bird. If you look closely, it’s got a little robot-y eye. Taly’s company, Mutants, is all about making stuff that looks friendly and cutesy, but it’s actually spying on you.

Personally, I think we should be very scared about surveillance. And not just visual surveillance, but also the amount of data that we’re giving up to companies more generally. So yes, the book definitely includes a critique of DARPA and agencies like that, who are using AI to further cement their military power.

Early in the book, there’s a humanoid robot that looks like Lal. I wondered if you could talk about that choice? It felt like it might be symbolic of Lal’s almost robotic existence at that point.

That’s a fantastic interpretation of it! Even my editor asked me why I did that. Basically, I just wanted one of the main characters to get the experience of the uncanny valley. It was nothing more than that – a moment of AI spookiness.

It definitely was.

I wanted Lal to have that experience of gazing at a factory produced version of herself.

Another reason for Lal to have that experience is that she hasn’t quite figured out how she feels about the auts. She wants to be part of that world, so this is saying: “Here are versions of you who are part of that world … but they’re just auts. They’re just nothing. They’re also praised and loved by everyone. But they’re still soulless machines. Do you really want to be a soulless machine, Lal?” So you’re right, it does touch on the idea that she becomes a bit of a soulless machine.

Okay.

People ask about that moment, and whether it’s a clue to a big conspiracy. But it’s not there for plot reasons. It’s more about Lal herself, and about the social experience of sharing a world with these uncanny others.

It was an intriguing thing to include early in the novel.

Well, I learned a lot about novel plotting during the writing of this book. And there are some things I’d probably change, because I think that ended up feeling like a red herring.

Lal goes to Tekna and gets absorbed into that world. She expects it’s going to be this shimmering, exciting experience. But actually it’s quite dreary.

Dhont is like an industrial estate. The Tekna Tower is where all the glamour happens, where Taly works, and where the conferences are. Lal sees that and she thinks, “That’s where I’m going to work! That’s where it’s going to happen for me!”

And then she’s deposited in the backend of nowhere instead. Dhont is meant to imply precarity and being low down on the chain at Tekna; it’s the opposite of the Tekna Tower.

Dhont has also been denuded of people, because of the automation. I don’t know if you saw the Richard Ayoade film, The Double?

I haven’t.

It’s based on a Dostoyevsky novella, I think. Jesse Eisenberg goes to work at this very grim, dystopian factory. But after a while, he’s kind of struggling. Then there’s a double, like another version of him that turns up and aces everything. The film is about their conflict. It’s really good, and the surroundings are very grim and derelict. So I had that industrial dystopian feel in mind. With automation on the rise, and Lal fighting for her survival, I wanted her to realise that working for a glamorous company might not be so glamorous after all. Work in an Amazon warehouse is horrible. So I wanted to pull the rug out from under her.

And she could see the Tower from afar.

From her sad little room!

She does work her way up. But it doesn’t feel like she’s happy with that.

All that glitters isn’t gold. When she does get promoted, she’s aware that there’s something lurking underneath. Something’s not right. She thinks, “Well, okay. This is great, and I’ve got loads of money, loads of time. But things are a bit off…” But then, she’s also competitive, especially with her sister, so she also wants to believe everything’s great. I wanted capitalism to pull her in with all its glories, and then wring her dry.

Yes, it definitely did. At the end, we don’t quite know for sure what she decided. I got the impression she made the right decision.

I’m glad you think she made the right decision.

Keep your surveillance apparatus peeled for part II, coming soon.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

Political pragmatism and utopian anticipation: A review of Fully Automated Luxury Communism

Thomas Connolly reviews Aaron Bastani, Fully Automated Luxury Communism (London: Verso, 2019), x + 278 pp, ISBN 978-1-78663-262-3.

The term “fully automated luxury communism” (or FALC) first began circulating among far-left internet commentators in the mid-2010s as a name given to one potential form of post-scarcity economy. Alongside a number of variations (such as “fully automated luxury gay space communism”, or FALGSC, which inflects the term with post-gender and post-sexuality connotations),

FALC gained wider recognition following a Guardian write-up in March 2015 that discussed the origins and significance of the term. Aaron Bastani—a British political commentator and co-founder of Novaro Media, an independent left-wing media outlet—was interviewed as part of that article, and has since become the key intellectual figure associated with FALC. The current work is Bastani’s attempt to unpack the meaning and implications of FALC.

Continue reading “Political pragmatism and utopian anticipation: A review of Fully Automated Luxury Communism” →

Surveillance Capitalism and the Data/Flesh Worker in Malka Older’s Infomocracy

By Esko Suoranta

This academic article explores themes of algorithmic governmentality, data surveillance, labour and embodiment in Malka Older’s Infomocracy, drawing in particular on Shoshana Zuboff’s theories of surveillance capitalism.

Review: This article underwent editorial review by two editors.
Citation: Suoranta, E. 2018. Surveillance Capitalism and the Data/Flesh Worker in Malka Older’s Infomocracy. https://vector-bsfa.com/2018/11/08/surveillance-capitalism-and-the-data-flesh-worker-in-malka-olders-infomocracy/
Keywords: economics, labour, Malka Older, platform capitalism, surveillance capitalism, Shoshana Zuboff

The cyberpunk dystopia is already here, it’s just unevenly distributed. Western democracies appear to be in crisis. Populist nationalisms are on the rise, while an ever-so-free market tightens its grip on our everyday existence, building vast private siloes of personal data. Climate change is spurred on by the rise of new imaginary currencies, mined from pure mathematics and pumping tens of millions of tons of carbon into the sky. Technologies from space travel to nanotechnology take unprecedented leaps. Meanwhile, in fiction, nostalgia appears to be a prime directive. The imagined futures of the 1980–90s receive reboots which appropriate the aesthetics of the past, but often fail to update its politics in the process: see Blade Runner 2049 (2017) and Ghost in the Shell (2017). Against such future-washed conservatism, a counter-project is also emerging. Critics and authors like Monika Bielskyte and Nnedi Okorafor sound the clarion for new ways to imagine the future, and to pave the path for a more equal and sustainable world.[1]

In this context, Malka Older’s debut novel Infomocracy (2016) explores progressive political and economic alternatives in a near-future setting. Part political techno-thriller, part thought-experiment on global micro-democracy, the novel follows four protagonists in the 22nd century as the third global elections loom. In the micro-democratic system, each geographic “centenal,” a unit of 100,000 people, chooses their representatives from a myriad of parties ranging from PhillipMorris and Liberty, to Earth1st and YouGov. Nation states have practically disappeared and the global election process is governed by Information, a descendant of the internet giants of yore, seemingly fused with something like the United Nations. The organization strives for neutral and truthful management of information and a fair administration of the micro-democratic process.

Predictably, political rivals try to play the system for their own benefit, and much of the plot revolves around such schemes. Through their twists and turns, Older highlights the precariousness of information labor in highly networked societies as workers become interfaces of bodies and computer networks, producing a distributed subjectivity. These themes become clear through an analysis of Older’s treatment of her protagonists and her depiction of Information’s custodianship of networked data. Infomocracy conducts an optimistic thought-experiment on the future of what Shoshana Zuboff calls “surveillance capitalism.” I aim to show how, for Older, there are two keys to diverting surveillance capitalism in a more optimistic direction. First, the democratization of skills related to information work. Second, the not-for-profit management of data.

Continue reading →

Robotics, science fiction and the search for the perfect artificial woman

File 20171024 30565 p2y72k.jpg?ixlib=rb 1.1 — Ociacia/Shutterstock.com

Irena Hayter, University of Leeds

Three photographs have been shortlisted for 2017’s Taylor Wessing Photographic Portrait Prize, hosted by the National Portrait Gallery in London. But there is something out of the ordinary about one of this year’s contenders for the prize. One of the portraits – by the Finnish artist Maija Tammi – is not of a human, but a female android.

The android in the photograph is Erica, described by her creator, Osaka University professor Hiroshi Ishiguro, as “the most beautiful and intelligent” robot in the world. The hardware beneath her silicone skin helps her achieve facial and mouth movements, but these can be rather unnatural, out of sync with her synthesised voice. She is cognitively sophisticated, though still unable to work out answers to complex questions from first principles, and she cannot move her arms and legs.

If this seems like something out of science fiction, you’re not far off. One of Ishiguro’s first female robots was named Repliee Q1 and he himself has said that the name derives from the French for “replicate” and from the “replicants” in Blade Runner: science fiction and robotics have always been entwined. Indeed, in a documentary made by the Guardian about Erica, Ishiguro reveals that he wanted to be an oil painter and insists on the similarities between his work and artistic creation.

It is difficult not to see here a masculine Pygmalionesque desire to create the perfect artificial woman. “Ishiguro-sensei is my father and he understands me entirely,” Erica pronounces in the documentary. Her vaunted autonomy seems more like a projection on the part on the roboticists who programme her thoughts, but also occasionally anthropomorphise her: the scientist who introduces himself as Erica’s “architect” also thinks that she is “really excited to interact with people”.

One of Them Is a Human #1 by Maija Tammi (Erica: Erato Ishiguro Symbiotic Human-Robot Interaction Project).
© Maija Tammi

Continue reading “Robotics, science fiction and the search for the perfect artificial woman” →

My Friend, The Bot

By Jay Owens.

I first met @botaleptic on 10th March, 2015. We were introduced by our mutual Twitter friend, Hugo.

I made a bot that replies as cryptically as I do. @botaleptic

— hugo reinert (@metaleptic) March 10, 2015

We soon got talking:

@botaleptic is a Twitter bot created by Hugo Reinert, who tweets as @metaleptic. His DNA is simple: “he” is a ruby script, running on a free app server, based on mispy’s twitter_ebooks code. Like all Twitter bots — automated ‘robot’ accounts — @botaleptic is simply an algorithm.

In this essay, I want to talk about how @botaleptic is much more than an algorithm.

Continue reading “My Friend, The Bot” →

vN by Madeline Ashby

vN: The First Machine Dynasty by Madeline Ashby (Angry Robot, 2012)
Reviewed by Andy Sawyer

The robots in Canadian author Madeline Ashby’s novel are self-replicating artificial humanoids designed by a “global mega-church” as post-Rapture “helpmeets” for those humans left behind after the ascension of the just. Why, it’s not clear – though given what we learn about how these robots are conditioned to engage with humanity, something beautifully ironic and poignant could have emerged. That is not what we get but vN is an interesting though flawed work.

Amy is one such construction, the daughter of robot Charlotte and flesh-human Jack. vN robots like Amy and her mother eat special robo-food and are fitted with a “failsafe” – a kind of First Law which not only prevents them from harming humans but actually causes them to shut down if violence is observed. On Amy’s graduation from kindergarten, her grandmother Portia turns up and attacks Charlotte. Amy eats her in her furious attempt to defend her mother but Portia somehow survives as a consciousness linked to Amy’s. Fleeing, Amy encounters Javier, a “serial iterator” who has given birth (vN reproduction is not gendered and vNs exist in networks of identical clades) to a dozen unauthorised copies of himself and becomes involved in a rather hazy political plot. The revelation that in her the failsafe has broken down is key: each side, human and vN, sees her as a potential weapon to be used or destroyed.

The novel only takes us so far and like many sf futures, vN suffers from something of a lack of focus. The robot-world is well evoked, with vN vagrants living off junk and tensions between vNs and humans. There has been a violent quake on the USA’s West Coast and, somewhere, a (semi?)-autonomous city-state of Mecha exists as a possible sanctuary. But is this culture all world-wide? Does every country in the world “have” vN humanoids? All this may be explored in subsequent volumes but some generic flattening undermines the interesting things Ashby is doing with the “robot” icon.

Still, there are fascinating things here in what is implied about families here – notably the relationship between Amy and her artificial-humanoid mother and human father and between her and Portia, the predatory grandmother. There’s also a skilful creepiness. It’s clear that these robots are – as ‘real’ robots may well be – used as sex toys. The term helpmeet does not necessarily have (in its original Biblical context) a sexual implication but it certainly derives this as a term for marriage partners and equally certainly New Eden Ministries, Inc. means this. The ungrown “child” vNs are of course tempting for those whose interests lie that way. The development of the ability in Amy’s clade to overcome their failsafes is ingeniously linked to her family history and the darker side of desire for robot sextoys that will do whatever you want.

There is, though, a lot about the nature of love (not all sexual) in the novel: obsessive love, the kind of love that may be simply exploitative. And here the most interesting figure may be Jack, Amy’s father: “Charlotte didn’t do drama… now he suspected he’d find human women too warm, too loud, too mobile.” Or, on the same page, “at one point [Amy] and Charlotte would be indistinguishable. Jack worried about that sometimes. What if one day, years from now, he kissed the wrong one as she walked through the door?”

This review originally appeared in Vector #271. vN has been shortlisted for the 2012 Golden Tentacle Award for debut novel that best fits the criteria of progressive, intelligent and entertaining. The winners of this award and the rest of The Kitschies will be announced on Tuesday, 26 February 2013.