Mackenzie Jorgensen is a Computer Science doctoral researcher working on the social and ethical implications of Artificial Intelligence. We invited Mackenzie to chat with novelist Eli Lee about her debut, A Strange and Brilliant Light (Jo Fletcher, 2021), and representations of AI and automation in speculative fiction. Should we fear or embrace the “rise of the robots”? Or perhaps the robots rose a long time ago, or perhaps that whole paradigm is mistaken? How might AI and automation impact the future of work? What would it mean for emotional work to be automated? How do human and machine stories intersect and blur?
This is part one of two.
Hi Eli, I’m really excited to talk to you today. I gave myself plenty of time to read A Strange and Brilliant Light, but I ended up going through it super quickly, because I enjoyed it so much.
Oh, thank you!
So I was curious – what made you decide to showcase three women’s stories?
Well, the genesis of the three stories was unexpected even to me. When I started, I wanted to write about a pair of best friends whose lives go in different directions. That’s based on my own relationship with my best friend, who became an incredible political activist whilst I just sat around and watched TV and read books. So that was the real kernel.
But as I wrote, it felt like something was missing. Lal and Rose came to me immediately – Rose was very passionate and active in the world whereas Lal had some of my own flaws – she was bossy, ambitious, and somewhat selfish.
But the dynamic needed a third person who was a contrast to both – and that’s when Lal’s sister Janetta came in. She works in AI, and she’s driven by her own hopes and fears. Once I had those three characters, it felt complete.
Did you see parts of yourself in Lal?
I did. I felt she was a good vehicle for the parts of me I’m less proud of – so she’s a bit selfish and insecure, and she feels belittled by her older sister, stuck in her shadow and ignored, but she’s still a decent person. She wants to work to make money for her family, but she’s just more … petty!
Got it!
And then I put what I would aspire to be in Janetta. Janetta’s very self-sufficient. She’s dedicated to her work and pure of heart. She has insecurities and flaws like the rest of us, but she always works for the greater good. So I kind of separated some of my worst qualities, and the qualities I wish I had, and put them in those two.
And you made them sisters, which works well in that sense.
I’ve got two brothers, but I don’t have a sister. Have you?
No, I have a younger brother.
I mean, this is the thing. Sibling relationships can be so gendered. I wanted to investigate what it’s like if there’s an older sister who is very successful and leaping ahead academically, and then you’re the younger sister in that dynamic. What’s for you? How do you stand out – how are you different, or memorable? So that was Lal.
“I kind of separated some of my worst qualities, and the qualities I wish I had, and put them in those two.”
How far into the future did you kind of picture the novel to be?
One of the get-outs of setting it in an alternate universe is that you don’t have to specify, “This is ten years in the future,” or, “This is fifteen years in the future.” I could choose the kind of technology that fit with the plot. They’re not mind-reading, they’re using mobile phones.
To me, this says it’s not that far in the future? Eight or ten years, perhaps. I’d be interested to hear what you think, as an AI researcher, about when it could plausibly be set? When that early, deep automation of jobs is filtering through?
Eight to ten years, yeah. End of the 2020s.
Then again, part of me thinks maybe that’s too soon! You know when you watch Back to the Future II, and there’s a flying car. It’s set in 2015. We all watched it in the late ‘80s, early ‘90s, and there was this sense that 2015 would look futuristic like that. Now we’re past that date, and the changes don’t seem that drastic.
Right.
So in ten years’ time, maybe things will look the same as they do now? Maybe AI will still be in our lives, but in a way that’s similar to what it is now – essentially under the surface and hidden. Ubiquitous, but hidden. The robots still won’t be serving us coffee! So I’m willing to be proved completely wrong with my timeframe.
I think you’re good! I feel like oftentimes AI is portrayed, especially in media and films, as taking over everything in the very near future. It’s often a dystopian presentation. But actual AIs right now, they’re always just good at one thing. They’re very task-specific. We don’t really have anything like what Janetta was trying to work on, like emotional AI.
Exactly.
And there’s another question: do we want that? Because I feel like emotion is something that makes us human. At the end of the day, AI and tech are a bunch of zeros and ones. You can’t really instill that with real human emotion and experiences, in my opinion. There are scientists out there who disagree though.
I should say that, in terms of eight to ten years, I’m not talking about emotional intelligence and AI. Consciousness is way off, if it ever will happen. I think probably it won’t. But in terms of AI and automation …
Automation, yeah. No, definitely.
My friend works for an AI start-up. He often looks at stuff in my novel, and says, “What the … This is crazy!” And I say, “I know! It’s not meant to be real!” When you watch Ex Machina or Her, there’s a suspension of disbelief. But I guess as an AI researcher it must be even harder, not to just say, “Come on, come on now. That’s not going to happen!”
“Maybe AI will still be in our lives, but in a way that’s similar to what it is nowー essentially under the surface and hidden. Ubiquitous, but hidden.”
And that question of whether AI can be human is just such a long-running, fascinating topic, isn’t it? We just can’t let go of it. That uncanny other self, reflected in an AI.
Yeah, definitely. I agree with you that I can see automation coming more into play in the near future, especially with big companies like Amazon. Which is scary, because people do rely on those big corporations for jobs. We’ve seen recently that unionizing doesn’t necessarily work in those scenarios. That’s one reason Rose’s character is very interesting to me. She explores the future of social justice activism, in a near-future world increasingly dominated by automation.
I knew that you can’t talk about automation without talking about Universal Basic Income. But I didn’t want someone who straight out of the gate was like, “You guys, UBI: I’m going to sort it out.” I wanted to make sure that Rose’s activism wasn’t disconnected from the rest of her life.
So much of the novel is about these three women in their early twenties, figuring out who they are, especially who they are in their relationships. With Rose, an important part of this is how she relates to men of power, or men who have power. There’s her father, her brother, and this other guy Alek, and initially she’s unable to get out from under them.
And so she needed to come into her own power. So I thought, Rose is going to be this activist, but she’s also going to be not sure of herself initially. So a lot of it was their inner struggles, intersecting with those larger economic, social, political, or technological stories.
There was a quote I made note of. ‘Alek said, “True leisure, true creativity and true freedom are within our reach for the first time in human history. And so we must set up source gain and welcome the auts.”’ This seemed quite ironic to me because relinquishing more control of the world could seem like the opposite of freedom. And Rose did realize this as time went on, which was cool to see, as she was learning and growing.
So Alek was with these other two academics at that point in the novel. Alek’s initial point of view is: “Auts are bad, AIs are bad. We need to just destroy this stuff.” But then when these two guys come along, one of them mentions post-work utopias. John Maynard Keynes wrote about something similar in the 1930s, an essay called ‘Economic Possibilities for our Grandchildren’, and Herbert Marcus wrote Eros and Civilisation in the 1950s, and there has been lots of writing about post-work more recently.
Maybe machines can do everything, and then you can sit around and play all day, and not have to do things you don’t want to. This idea floats past Alek this evening, and suddenly he’s like, “Oh, wait! Yeah, we can just be free, because auts will do the boring stuff!”
But that’s obviously not a realistic suggestion, because if you take it a step further, like Rose does, the question is, “Who owns those auts?” Well, if it’s the corporations, that’s not freedom. So that brings Alek back to his original idea: we need source gain. We need some kind of UBI. So in that moment when he talks about post-work leisure, he’s speculating. He’s not thinking about what’s necessary now.
Can you see a world where AI grows in importance alongside human creativity and freedom? Or are they opposing forces?
In a post-work scenario, the AIs are doing the grunt work, doing the kind of cleaning and tidying, and fixing things, and all the behind-the-scenes organisational work, so humans can play and fulfil ourselves. So that’s what Alek would mean by welcoming the auts, I think. But do you mean in terms of AI more as an equal?
I guess, or at least AI growing in social importance, and taking on more and more roles?
The way Alek envisions AI, in that moment, they would be this kind of sub-caste. They’d work away in the background, and you wouldn’t need to worry about them because they wouldn’t be conscious. But I think for us, even without AI consciousness, this could still be a very unsettling and unnerving vision.
We’re already seeing that when AI creeps into more and more areas of life, that ideal of true leisure and creativity gets compromised. You’re surrounded by stuff that’s monitoring you, surveilling you, collecting and analysing your data, perhaps even filtering your reality, and steering you in various ways. It’s almost like the more AI we have, the more inhibited we might feel.
Right, and the more potential problems we might face. On the surveillance point, there’s that moment where Janetta and Taly discuss helping the government with docile spy dogs —
This is one of my cringe moments. I read it now and think, “Spy dogs? What?”
Well Boston Dynamics has a robotic dog. The New York City Police Department had a test run, and there was a huge backlash. So they said, “Okay, actually, no. We are not going to use this.” But about Janetta and Taly’s conversation, I was curious: were you critiquing how governments and the private sector collaborate over surveillance? How do you feel about that?
Attitudes about surveillance are deeply personal. I’ve got one friend who just does not care about his privacy – he’ll happily give all his data to everything and everyone. It’s not because he believes that it might make society better; he just doesn’t care. I suspect he’s not alone in that.
“We’re already seeing that when AI creeps into more and more areas of life, that ideal of true leisure and creativity gets compromised. You’re surrounded by stuff that’s monitoring you, surveilling you …”
The bird on the front of the novel, illustrated by Sinjin Li, is a CCTV bird. If you look closely, it’s got a little robot-y eye. Taly’s company, Mutants, is all about making stuff that looks friendly and cutesy, but it’s actually spying on you.
Personally, I think we should be very scared about surveillance. And not just visual surveillance, but also the amount of data that we’re giving up to companies more generally. So yes, the book definitely includes a critique of DARPA and agencies like that, who are using AI to further cement their military power.
Early in the book, there’s a humanoid robot that looks like Lal. I wondered if you could talk about that choice? It felt like it might be symbolic of Lal’s almost robotic existence at that point.
That’s a fantastic interpretation of it! Even my editor asked me why I did that. Basically, I just wanted one of the main characters to get the experience of the uncanny valley. It was nothing more than that – a moment of AI spookiness.
It definitely was.
I wanted Lal to have that experience of gazing at a factory produced version of herself.
Another reason for Lal to have that experience is that she hasn’t quite figured out how she feels about the auts. She wants to be part of that world, so this is saying: “Here are versions of you who are part of that world … but they’re just auts. They’re just nothing. They’re also praised and loved by everyone. But they’re still soulless machines. Do you really want to be a soulless machine, Lal?” So you’re right, it does touch on the idea that she becomes a bit of a soulless machine.
Okay.
People ask about that moment, and whether it’s a clue to a big conspiracy. But it’s not there for plot reasons. It’s more about Lal herself, and about the social experience of sharing a world with these uncanny others.
It was an intriguing thing to include early in the novel.
Well, I learned a lot about novel plotting during the writing of this book. And there are some things I’d probably change, because I think that ended up feeling like a red herring.
Lal goes to Tekna and gets absorbed into that world. She expects it’s going to be this shimmering, exciting experience. But actually it’s quite dreary.
Dhont is like an industrial estate. The Tekna Tower is where all the glamour happens, where Taly works, and where the conferences are. Lal sees that and she thinks, “That’s where I’m going to work! That’s where it’s going to happen for me!”
And then she’s deposited in the backend of nowhere instead. Dhont is meant to imply precarity and being low down on the chain at Tekna; it’s the opposite of the Tekna Tower.
Dhont has also been denuded of people, because of the automation. I don’t know if you saw the Richard Ayoade film, The Double?
I haven’t.
It’s based on a Dostoyevsky novella, I think. Jesse Eisenberg goes to work at this very grim, dystopian factory. But after a while, he’s kind of struggling. Then there’s a double, like another version of him that turns up and aces everything. The film is about their conflict. It’s really good, and the surroundings are very grim and derelict. So I had that industrial dystopian feel in mind. With automation on the rise, and Lal fighting for her survival, I wanted her to realise that working for a glamorous company might not be so glamorous after all. Work in an Amazon warehouse is horrible. So I wanted to pull the rug out from under her.
And she could see the Tower from afar.
From her sad little room!
She does work her way up. But it doesn’t feel like she’s happy with that.
All that glitters isn’t gold. When she does get promoted, she’s aware that there’s something lurking underneath. Something’s not right. She thinks, “Well, okay. This is great, and I’ve got loads of money, loads of time. But things are a bit off…” But then, she’s also competitive, especially with her sister, so she also wants to believe everything’s great. I wanted capitalism to pull her in with all its glories, and then wring her dry.
Yes, it definitely did. At the end, we don’t quite know for sure what she decided. I got the impression she made the right decision.
I’m glad you think she made the right decision.
Keep your surveillance apparatus peeled for part II, coming soon.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
vector-bsfa.com 