2.3 Extracting Information from Data

Data are raw values: numbers, text, readings, clicks. Information is the facts and patterns you extract from data. A list of 10,000 bus tap-in times is data; "the 7:40 bus is overcrowded on Mondays" is information.

Data can reveal trends, show connections between things and help address real problems, like where to add a crosswalk or when a store needs more staff.

Often one data source isn't enough. You may need to combine sources to draw a conclusion. To decide whether a park needs more lighting, a city might combine crime reports with streetlight locations and park-visit counts.

Metadata are data about data. For a photo, the data is the image itself; the metadata might include the date and time it was taken, the GPS location, the camera model and the file size. For a song, metadata might be the title, artist, length and genre.

Metadata help you find, organize and manage data. They're how a phone groups photos by place or a music app sorts by artist. They also add context that makes data more useful.

Changing or deleting metadata doesn't change the data itself. Editing a photo's date tag doesn't change a single pixel.

Every data set, large or small, can have problems:

• Data that needs cleaning: when people type into an open text box, the same answer shows up as "NY," "N.Y." and "new york." Cleaning makes data uniform without changing its meaning, such as by replacing all versions with one spelling.
• Incomplete data: missing values, like survey questions people skipped.
• Invalid data: values that can't be right, like an age of 250 or a date of February 31.
• Combining sources: different sources may use different formats or units that have to be matched up first.

Bias often comes from the type or source of the data collected. If a survey about school lunches only reaches students who eat in the cafeteria, it leaves out everyone who doesn't, and collecting more responses the same way won't fix that. More data doesn't remove bias.

Data may show a correlation, meaning two things tend to change together. That doesn't prove one causes the other. Ice cream sales and sunburns rise together, but both are caused by hot, sunny weather. You'd need more research to know the real relationship.

The size of a data set affects how much information you can get from it. Bigger data sets can reveal patterns a small sample would miss.

Very large data sets are hard to process on one computer and may need parallel systems, where many processors or computers share the work. Scalability is a system's ability to grow to handle more data or users. How much a system can compute and store limits what you can do with a data set.

What you can do with data also depends on the people and tools available. A spreadsheet works for a few thousand rows; millions of rows may need a database and someone who can program it.