06 DTC 101

Data & Information

6.1 Big Data

Seeing with Data

In 1854 a cholera outbreak killed hundreds of people in the Soho neighborhood of London in just a few weeks. At the time, most doctors believed cholera spread through bad air. A physician named John Snow suspected the water. He marked each death on a street map and found that the deaths clustered around a single public water pump on Broad Street. The map persuaded local officials to remove the pump's handle, and the outbreak soon faded.

Snow's map is one of the most famous examples of data making something invisible visible. But the map alone was not enough. Snow also walked the neighborhood and talked to residents, which helped him explain the cases that did not fit. A nearby workhouse had its own well and few deaths. Workers at a local brewery drank beer instead of water. A woman who lived miles away had died because she liked the taste of the Broad Street water and had it delivered to her home. Data showed the pattern; human knowledge of the neighborhood explained it.

Today, data is how businesses and governments make decisions. Cities use traffic data to time stoplights. Public health agencies track disease. Companies study sales data to decide what to make and where to sell it. When it is collected and read carefully, data can show complexity in a way people can understand.

Data Analytics and Visualization

Data Analytics is the discipline of turning raw data into information, and making conclusions about that information. The techniques of data analytics are often automated with programmed algorithms that collect, process, clean and organize vast raw data sets so that patterns can be detected and results presented in a form that can be analyzed for human consumption. Because big data analysis often results in numeric values and complex, abstract relations between these values, a data analyst presents findings as a data visualization or infographic accompanied by an interpretive story.

Given the exponential increase of digital data, data analytics and data visualization are growing fields that will impact almost all areas of human life. While the interpretive stories and visualizations that come out of data analysis can give fresh insights into the world's problems, it is important to remember that humans write the algorithms that frame the results. The collection and interpretation of any kind of data is prone to human fallibility.

Big Data and Prediction

Big data refers to massive data sets that can only make sense with the assistance of a computer. An individual U.S. citizen produces an enormous amount of data as a byproduct of just living in a networked world. Searches, purchases, geographical movements, preferences in food and media entertainment might seem like insignificant raw data on the individual level, but when aggregated and merged with other data from other individuals the data set could result in patterns of behavior that might produce useful information. "Millennial purchasing history" is a query that might scan vast data sets over a period of time and then return useful patterns for companies trying to market to millennials.

Big data can do more than describe trends. Because it records the behavior of millions of people over time, it can be used to predict what people are likely to do next: what they will buy, watch, click, or believe.

Patterns in big data can also be misleading. In 2008 Google launched Google Flu Trends, which tried to track flu outbreaks simply by counting flu-related search terms. It worked at first, but by 2013 it was badly overestimating flu cases, in part because people search for flu symptoms for many reasons besides being sick. Google shut the project down in 2015. The lesson is that a correlation between two patterns, such as searches and illness, does not prove that one causes the other.

Data and the Platforms

The ability to predict behavior is what makes the largest internet platforms so powerful and wealthy. Web search began in 1994, when the World Wide Web was in its infancy. Search, social media, and video platforms have since become enormous businesses built on the data they collect about their users. Most of the money Google and Meta make comes from advertising, and what they sell is not simply ad space but targeted advertising: the chance to reach exactly the people their data predicts will respond.

Platforms also grow more powerful through network effects. The more people use a platform, the more valuable it becomes to each user, and the more data it collects. A few platforms come to dominate, and the open web of many small sites gives way to a handful of large hubs. On these platforms, recommendation algorithms decide what spreads. Creators who make what the algorithm rewards, such as content that keeps people watching, scrolling, and sharing, gain audiences and income, while others become harder to find (see 5.4 Algorithms and Attention).

The data platforms collect can be very personal. On August 4th 2006, AOL accidentally released 20 million search queries from 650,000 users. Names were removed, but the searches revealed private details about users' health, relationships, fears, and desires, and reporters were able to identify some of the people behind them.

Harvard professor Shoshana Zuboff calls this business model "surveillance capitalism." In the opening of this documentary she explains how Google, and later Facebook, discovered that the leftover data from users' searches and clicks could be used to predict, and eventually influence, what people would do next. As you watch, notice her argument that these companies are selling predictions about us, not just our data.

Algorithms also capture and process personal data to make consequential decisions about individual lives, such as whether someone should get a loan, keep their health care, or be released on bail. What are the implications and consequences of this type of automated surveillance and analysis? How and when is the big data collected about human behavior an invasion of individual privacy? How might computer systems, which lack the nuance and subtlety of human-to-human interaction, make harmful decisions in the interest of pattern prediction and efficiency? How can humans trust digital data when it is quite easy for AI to create "fake" data?

There are no easy answers to these questions. Big data promises to make the world more efficient and safer, but it also introduces new technological processes that are largely invisible to humans and therefore unpredictable by humans.

6.2 Data & Culture

There are some things humans can't do. For example, finding a facial match in a database of millions of images within seconds. Because of such powerful abilities, big data technologies can extend research and creativity into new areas. Below are examples of just some of the ways in which big data is being used in the arts, sciences and humanities. In each case, notice what the data does well and where human judgment still has to step in. The most useful work happens in the space between data and people.

Datasets as Cultural Artifacts

Datasets are not neutral collections of facts. They reflect human choices about what is measured, preserved, categorized, ignored, or erased. A census, museum archive, search log, medical database, or social media dataset can reveal important patterns, but it also carries the assumptions of the people and institutions that created it.

When data is used to make decisions about culture, health, education, policing, work, or creativity, those choices matter. What counts as data? Who collects it? Who owns it? Who gets represented? Who becomes invisible? These questions are especially important in the age of artificial intelligence, because AI systems often learn from datasets created for other purposes.

Entertainment

Streaming companies collect detailed data on what viewers watch, pause, rewatch, and abandon, and they use that data to decide which shows to make. In this talk, data scientist Sebastian Wernicke compares two companies that used viewer data to choose new series, with very different results. As you watch, notice where he thinks data is useful and where he thinks human judgment still has to take over.

Healthcare

Medical records, lab results, genetic tests, and the data from fitness trackers and smartwatches can be combined to spot disease earlier, predict which treatments will work, and track outbreaks. In this talk, physician and scientist Eric Topol shows how AI systems trained on millions of medical images can catch things doctors miss, and even predict heart disease from a scan of the eye. His most surprising argument is that AI could make medicine more human, by taking over paperwork and data entry and giving doctors more time with their patients. A system might flag a shadow on a scan, but a doctor still has to talk with the patient, weigh their history, and decide what to do. Health data is also some of the most private information a person has. As you watch, think about who should be able to see your health data, and what could go wrong if it were shared or sold.

Arts and Humanities

Big data is also changing how scholars and artists work. In literary studies, researchers can use computers to analyze thousands of novels at once, looking for patterns no single reader could see. This approach is sometimes called "distant reading." The computer finds the pattern, but a scholar still has to decide what it means. In the first video, Matthew Wilkens explains what this kind of research can reveal. In the second, an artist translates storm and weather data into sculpture, turning numbers into something you can see and touch. Both ask the same question from different directions: what new kinds of meaning can we make from data?

Social Trust

Data is only useful if people can trust it, and trust raises two different questions. The first is whether a record is real. Generative AI has made it easy to produce synthetic data that was never recorded from the real world (see Chapter 07 for how these systems work). So how do we know where an image, a video, or a document came from? In response, some technology and media companies have begun attaching "content credentials" to images and video, a kind of digital label that records where a file came from and how it was edited.

The second question is who keeps the record, and whether anyone can change it. For most of history, banks, governments, and archives kept official records, and people had to trust them not to alter those records. The blockchain is an attempt to remove the need for that kind of trust. It stores a shared record, or ledger, across many computers, so that no single party can quietly change it. A blockchain does not solve the first problem. It cannot tell whether information was true when it was entered; it can only show that the record has not been changed since. Blockchains are best known for cryptocurrencies, and many of the bigger promises made about them have not come true. The idea behind them is still worth understanding: who do we trust to keep the record, and why?

6.3 Data and Human Experience

In Autonomous Trap 001 (2017), artist James Bridle painted a circle of road markings around a self-driving car: a dashed line on the outside and a solid line on the inside. Traffic rules say a car may cross a dashed line but not a solid one, so a car following the rules can drive into the circle but cannot drive out. Bridle compares it to a salt circle in ritual magic. The piece shows how a machine that reads the world only through data and rules can be trapped by something any human driver would ignore.

What does it mean to be human in a world with big data technologies making decisions about health, law, transportation, entertainment, social interaction, desire, and taste? This section looks at what happens when data and automated systems replace human judgment instead of supporting it, in three areas: automated content, job screening, and government services.

What Data Cannot Capture

Data can reveal patterns, but it cannot fully capture lived experience. A number may record a purchase, a click, a location, or a test score, but it does not automatically explain context, emotion, intention, memory, uncertainty, or local knowledge. Human beings are more than the measurable traces they leave behind.

This does not mean data is useless. It means data must be interpreted carefully. A dataset can help us see large-scale patterns, but it can also flatten difference, ignore exceptions, and encourage decisions that feel objective simply because they are numerical.

Automated Content

James Bridle's talk looks at what happens when recommendation algorithms meet children's media. He found thousands of strange, often disturbing videos on YouTube, many produced automatically, that were made to match popular keywords and get picked up by the recommendation system. No one intended the result, but the system rewarded it. As you watch, ask who is responsible when an automated system produces something harmful.

Job Screening

Many employers now use automated systems to sort job applications before a person ever reads them. In 2014 Amazon began building an experimental tool to rate job applicants. It learned from ten years of résumés submitted to the company, most of them from men. The system taught itself that male candidates were preferable. It downgraded résumés that included the word "women's," as in "women's chess club captain." Amazon's engineers could not guarantee the tool would not find other ways to discriminate, and the company abandoned it. The system was not told to prefer men. It simply learned the patterns in the data it was given.

Mathematician Cathy O'Neil, author of Weapons of Math Destruction, argues that algorithms are not objective. They are opinions built into code, shaped by the data they are given and the definition of success their designers choose. Listen for her examples of algorithms used to evaluate teachers and job applicants, and for what she thinks should be done about them.

Government Systems

Governments also use data to decide who receives help and who is suspected of fraud. In the Netherlands, the tax authority used a risk-scoring system to look for fraud in childcare benefit claims. Beginning around 2013, tens of thousands of parents were wrongly accused of fraud, often over small paperwork errors, and ordered to pay back large sums. Having dual nationality was treated as a risk factor, so families with immigrant backgrounds were singled out. Many families fell deep into debt, some lost their homes, and some children were placed in foster care. Officials trusted the system's flags over the families' explanations for years. When a parliamentary investigation exposed the scandal, the Dutch government resigned in January 2021.

Cases like these share a pattern. No single person decided to cause harm. A system was built, trusted, and left to run, and the people affected had little way to question it.

6.4 Surveillance and Prediction

Digital data is often collected in the background as people search, shop, communicate, navigate, stream, post, and move through the world with connected devices. This constant collection creates new forms of surveillance. Some surveillance is visible, such as a security camera or a login screen. Other forms are less visible, such as tracking pixels, cookies, app permissions, location histories, and data brokers that combine information from many sources.

Data Tracking

Tracking systems collect traces of behavior: what users click, where they pause, what they buy, how long they watch, which locations they visit, and which posts they share. Each piece of data may seem small, but together these traces create profiles that can be used for advertising, recommendation, risk assessment, research, and prediction. This is the raw material of the surveillance capitalism Zuboff describes in 6.1.

Prediction and Recommendation

Prediction systems use past data to estimate future behavior: what someone wants to buy, what they might watch, whether they might repay a loan, or whether an institution should consider them a risk. Recommendation engines put these predictions to work in everyday digital culture. Streaming platforms recommend movies and music, social media platforms recommend posts and accounts, and search engines rank results. These systems do not simply respond to user desire; they help form desire by deciding what becomes visible, repeated, or hidden. And because they learn from the past, they can reproduce old inequalities if the data reflects biased social systems.

Algorithmic Decisions

Algorithmic decision-making becomes especially serious when data systems influence access to jobs, housing, credit, education, health care, policing, or legal outcomes. A decision may appear neutral because it comes from a computer, but the system still depends on human choices about data, categories, goals, and design. The central question is not only whether a system is accurate, but whether it is fair, accountable, and understandable.

Computer scientist Joy Buolamwini (see also Machine Vision in Chapter 07) discovered this problem firsthand as a graduate student at MIT, when facial analysis software could not detect her face until she put on a white mask. She traced the problem to training data that did not include a broad range of skin tones and facial structures, and calls the result the "coded gaze." Her later research, and the documentary Coded Bias, helped push major tech companies to address bias in their facial recognition products.

Data systems increasingly shape future possibilities rather than merely recording past behavior.

Complex Systems

Human beings are complex and often contradictory. We change our minds, act against our own interests, and behave differently in different groups. Societies, cultures, and natural systems are even more complex. They are made of countless relationships that shift over time. No dataset can fully track them, and no machine can control them.

Political scientist James C. Scott tells a story about this in his book Seeing Like a State. In the late 1700s, foresters in Prussia and Saxony wanted to manage their forests for timber and tax revenue. A wild forest, with its many species, fallen logs, and tangled undergrowth, was hard to measure. So the foresters cleared it and replanted single species of trees in straight rows that were easy to count. The first generation of these "scientific" forests produced excellent timber. By the second and third generations, the forests were failing. The soil was exhausted, pests spread easily through the identical trees, and storms knocked them down. Everything that had made the forest a living system had been left out of the data. The Germans even coined a word for the result: Waldsterben, or "forest death."

Thinking in terms of systems and ecologies is vital for understanding a complex world, because it shows us how everything is connected. But systems thinking can also go wrong. When a system is seen as naturally balanced or self-regulating, it can be used to justify keeping things as they are, including existing hierarchies of power. The question is always who designs the system, who it serves, and who is left out of the data.

From Data to AI

Data is also the raw material for artificial intelligence. Machine learning is a method of data analysis that gives the computer the task not only of sorting and processing big data, but also of analytical model building. As a branch of artificial intelligence, machine learning gives systems the power to learn from data by identifying patterns and making informed decisions with almost no human involvement.

Contemporary AI systems depend on enormous datasets for training. Large language models, recommendation systems, image generators, and speech recognition systems learn patterns from data rather than being explicitly programmed with fixed rules. Chapter 07 explains how these systems learn. The key point for this chapter is simple: AI begins with data, and someone has to collect, label, and organize that data first.

One of the most important datasets in AI history is ImageNet. Beginning in 2007, computer scientist Fei-Fei Li and her collaborators gathered millions of photographs from the internet and had them labeled by thousands of online workers, sorting each image into categories such as "cat," "chair," or "volcano." ImageNet made the breakthroughs in machine vision described in Chapter 07 possible. As you watch, notice how much human work goes into building a dataset before any machine learns from it.

Labels are choices, and choices carry assumptions. In 2019, AI researcher Kate Crawford and artist Trevor Paglen looked closely at the categories ImageNet used to describe people. They found labels that were offensive, judgmental, or simply absurd, applied to ordinary photographs of strangers. Their online project ImageNet Roulette let people upload a selfie and see how a system trained on those labels would classify them. The project went viral, and ImageNet soon removed hundreds of thousands of images from its "person" categories. Read their essay Excavating AI: The Politics of Images in Machine Learning Training Sets and ask: who decided what these categories should be?

If AI begins with data, then it matters whose data it is and who controls it. Many AI systems have been trained on text, images, music, and video collected from the open web, including books, news articles, artwork, and social media posts. There are strong arguments on more than one side. Most of the people who made that material were never asked, and some authors, artists, and news organizations have filed lawsuits arguing that this use of their work is unfair. Others argue that the written and recorded knowledge of human culture should be free to read, study, and learn from. Libraries, public education, and the open web were all built on this idea of freedom of information, and a machine learning from public material may be no different from a person reading or looking.

The deeper question may not be whether this trove of human knowledge should be used, but who owns and controls what is made from it. A few companies hold the largest AI models, and the data used to train them is mostly closed to the public. A system built from the work of millions of people can become the private property of one company. For now, most AI companies' terms of service say that users own what they create with these tools, and U.S. copyright law does not protect work made entirely by a machine. But terms can change. What if AI companies claimed ownership over what their systems produce, or over the knowledge people build with them? Systems that present themselves as open and helpful can also hide concentrations of power. The question this chapter returns to again and again is not only who owns data, but who benefits from it: everyone, or just a few?

6.5 Unit Exercise: Protect Your Data

Adapted from How to Protect Your Digital Privacy | nytimes.com (2019). For a free, regularly updated guide, see the Electronic Frontier Foundation's Surveillance Self-Defense.

This exercise has two parts. The first covers a few basic precautions, the digital equivalent of locking your door. The second is a chance to see what platforms actually record about you. There is no right amount of privacy; the goal is to understand the trade-offs so you can make your own choices.

Part 1: Basic Precautions

These simple habits protect your accounts and devices. If you already do them, great. If not, try one or two.

1. Use a password manager such as Bitwarden (free and open source) or 1Password to create and remember a different strong password for each account.
2. Turn on two-factor authentication for your most important accounts, such as email, school, and banking.
3. Keep your software and apps updated. Updates often fix security problems.
4. Turn on HTTPS-Only mode in your browser's settings so you always connect to the secure version of a site, which matters most on public Wi-Fi.
5. Optional: try an ad-blocking extension like uBlock Origin, which also blocks many trackers. (In Chrome, use uBlock Origin Lite.)

Part 2: See What Is Tracked About You

Choose one or two platforms you use often and look at what they collect. You don't have to change anything. If you see a setting you'd like to change, the "Where to Look" list below shows where to find it.

1. Download or review your personal data from one platform, such as Google, Instagram, TikTok, Spotify, or Apple. Some platforms take a few days to prepare your download, so request it early.
2. Look at the ad settings on that platform. What interests or categories has it assigned to you? Do they seem accurate?
3. Check which apps on your phone have access to your location, and whether they need it.
4. If you use an AI chatbot, find out whether your conversations are saved and whether they are used to train future models.

Where to Look

• Google (Search, YouTube, Maps): My Activity shows your saved history. Under "History settings," you can set Web & App Activity, Location, and YouTube History to delete automatically after a set time. Google Takeout lets you download your data.
• Instagram and Facebook (Meta): In Accounts Center, "Your information and permissions" lets you download your data and see "Your activity off Meta technologies," a list of businesses that share data about you with Meta. "Ad preferences" shows the topics used to choose your ads.
• TikTok: In Settings and Privacy, the ads settings control whether activity from other apps and websites is used to choose your ads. Request your data under Account > Download your data.
• AI chatbots: In ChatGPT, Settings > Data controls has the "Improve the model for everyone" setting. In Claude, Settings > Privacy has the "Help improve Claude" setting. Both control whether your conversations are used for training.
• Your phone: On an iPhone, Privacy & Security > Tracking controls whether apps can ask to track you across other apps. On Android, Privacy > Ads lets you reset or delete your advertising ID.

Reflection Questions

1. What did you find in your data that you expected? What surprised you?
2. How accurate were the interests or categories a platform assigned to you?
3. Did you decide to change any settings? Why or why not? What would you give up by changing them?
4. What could a company or algorithm infer about you from this data that would be wrong? What about you could never show up in this data at all?

6.6 Glossary

• data: raw facts, measurements, symbols, or records that can be processed into information.
• information: data that has been organized, interpreted, or given meaning.
• big data: extremely large datasets that require computational tools to store, process, and analyze.
• dataset: a structured collection of data used for analysis, research, or machine learning.
• metadata: data about data, such as time, location, author, file type, or device information.
• data analytics: the process of examining data to find patterns, answer questions, or support decisions.
• data visualization: the representation of data through charts, maps, diagrams, or other visual forms.
• infographic: a visual presentation that combines data, images, and short text to explain information quickly.
• network effect: the way a platform or service becomes more valuable to each user as more people use it.
• targeted advertising: advertising directed at specific people based on data about their behavior, interests, or demographics.
• machine learning: a method of computing in which systems learn patterns from data rather than following only fixed rules.
• training data: data used to teach a machine learning system to recognize patterns or generate outputs.
• synthetic data: data generated by a computer system rather than recorded from real-world events or people.
• correlation: a relationship between two patterns or variables that does not necessarily prove cause and effect.
• algorithmic decision-making: the use of computational rules or models to influence or make decisions.
• surveillance: the monitoring or tracking of people, behavior, places, or data.
• cookie: a small file a website stores in your browser to remember you and, often, to track your activity across sites.
• tracking pixel: a tiny, usually invisible image embedded in a webpage or email that reports when and where it was viewed.
• data broker: a company that collects personal information from many sources and sells or shares it with others.
• prediction: the use of data patterns to estimate future actions, outcomes, or risks.
• recommendation engine: a system that suggests content, products, or people based on data about a user and others like them.
• privacy: the ability to control access to personal information and personal activity.
• blockchain: a distributed record-keeping system that stores transactions across a network of computers.

6.7 Bibliography