Black Box Manifesto, V1.0
A “black box” is a system whose inputs and outputs are known, but whose inner workings remain inscrutable. Could we step inside the Black Box?
Speed is over time and place. Speed is power. Speed permits misinformation… disorients time and place, and is a fierce and uncompromising ruler. Our obsession with high speed leaves no time or place for return. It is now already too late and today is yesterday with its memory already lost.
How can we go forward, when action is to watch action? When the eyes are locked in a fixed gaze. When knowledge becomes information. When words are stumbling blocks and have lost their representation. When discourse is opinion. When you don’t have to know anything, and you think you know everything. When to reflect is gazing in the mirror. When to contemplate is thinking about yourself.
—Elaine Sturtevant, Shifting Mental Structures
What if we could glimpse our other self? The self composed of data points and inferences, nodes and connections. The self with whom we interact each time we’re shown a photograph on Instagram, a track on Spotify, a product on Amazon. Our Data Self. This self is usually invisible to us, yet its influence is felt every day. As we tap and scroll and click, our Data Self attempts to perceive the unconscious intentions behind these digital moments. We leave behind traces — bits which construct and shape our Data Self. Holistically informed, yet humanly naive.
Social media seem designed to generate [ontological insecurity]: They systematically impose a sense of insubstantiality on users, turning identity into incoherence by constantly assimilating and demanding more data about us, making our self a vacuum that never fills, no matter how much is poured in. Our identity is constantly being recalibrated and recalculated, and we can forever try to “correct” it with more photos, more updates, more posts, more data.
—Rob Horning, Sick of Myself
Beyond merely answering our queries and anticipating our needs, the Data Self gradually alters our desires and values through the curation of information and entertainment — blurring the lines between fact and fiction, physical and virtual reality.
Parsing the limitless information made available through the internet is a fool’s errand without the help of sophisticated algorithmic entities. Yet recent events call into question the reliability and trustworthiness of these assistants by revealing perverse incentives and hidden agendas. We are forced into a constant state of reevaluating whom we can trust. Our methods for making sense of the world must be updated accordingly, as growing calls for “media literacy” training in schools make clear.
Our individual epistemologies — the ways we know what we know — are fragmenting. Divisions are becoming starker at every level, due at least in part to the influence of our algorithmic tools. Navigating this milieu requires new awareness and new abilities. We must first reposition ourselves in relation to our Data Self. We must reconsider the potential of these systems as not merely assistants, but collaborators in a process of self-actualization. These systems can augment our lived experience, but this is unlikely with values ill-prepared for the Age of Algorithms.
At their worst, [algorithmic systems] threaten to send each of us down our own decontextualized rabbit hole, weakening collective narratives and exposing us to active attacks on what Renee DiResta calls ‘society’s ability to operate with a shared epistemology.’
—Drew Austin, Cold Discovery
Some of us are conscious and active in our relationship with algorithms. We might avoid watching a particular YouTube video because of how it will affect our recommendation feed. We might be exposed to eerily relevant advertisements and try to decipher the algorithmic reasoning behind them by reflecting on the digital traces we’ve generated recently, or perhaps become paranoid about our smartphones listening in to our conversations. Yet this kind of relationship with our Data Self is combative and detrimental; we recognize that our intentions and the Data Self’s incentives are misaligned, but have no recourse.
How might we build more mature algorithmic systems and develop new behaviors around them?
If enough people start to see themselves as participants in a process that updates their goals and preferences , it will transform society… Society moves forward when the design principles for human systems change. When a new principle (like “freedom” or “fairness” or “meritocracy” or “structural oppression”) appears and becomes widely recognized, people redesign everything (schools, local businesses, etc.) in accordance with it.
What society would help people determine their most meaningful goals and preferences? This question sparks the imagination. And the related design principles — exposure to the consequences of one’s actions, discretion over the manner in which one works, and fostering capacity to handle hard truths together — suggest immediate changes to institutions and new forms of intimacy.
We don’t presume to have a solution, but believe it’s imperative for more people to engage with these questions, so that we might co-create a better system.
We already interact with many systems as if they have personalities.
What if algorithms were no longer anonymous, but personified? What if an algorithmic ‘curator’ were designed with a much more specific voice — persona — in mind? One which introduces itself to humans and is clear about who exactly it is.
As long as we maintain our assumption that algorithmic systems are black boxes which cannot be understood, we will only further scatter our identity — individually, and as a society. Algorithms are possibly today’s most important, yet most complex tool. We must work together to sharpen them along with our ability to use them.
The above text was read aloud by Siri in an interactive exhibit at the University of Washington Design Show in June 2019. Here is that reading of the text:
And here is a short film that seeks to visually and audibly capture the tone and interactions with the exhibit:
