{"id":142725,"date":"2020-12-28T14:00:25","date_gmt":"2020-12-28T11:00:25","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/why-web-scraping-is-vital-to-democracy\/"},"modified":"2020-12-28T14:00:25","modified_gmt":"2020-12-28T11:00:25","slug":"why-web-scraping-is-vital-to-democracy","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/why-web-scraping-is-vital-to-democracy\/","title":{"rendered":"#Why web scraping is vital to democracy"},"content":{"rendered":"

“#Why web scraping is vital to democracy<\/strong>”
\n<\/p>\n

\n

The fruits of web scraping \u2014 using code to harvest data and information from websites \u2014 are all around us.<\/p>\n

People build scrapers that can find every Applebee\u2019s on the planet<\/a> or collect congressional legislation and votes<\/a> or track fancy watches for sale<\/a> on fan websites. Businesses use scrapers to manage their online retail inventory<\/a> and monitor competitors\u2019 prices<\/a>. Lots of well-known sites use scrapers to do things like track airline ticket prices<\/a> and job listings<\/a>. Google is essentially a giant, crawling web scraper.<\/p>\n

Scrapers are also the tools of watchdogs and journalists, which is why The Markup filed an amicus brief<\/a> in a case before the U.S. Supreme Court this week that threatens to make scraping illegal.<\/p>\n

The case itself\u2014Van Buren v. United States<\/em><\/a>\u2014is not about scraping but rather a legal question regarding the prosecution of a Georgia police officer, Nathan Van Buren, who was bribed to look up confidential information in a law enforcement database. Van Buren was prosecuted under the Computer Fraud and Abuse Act (CFAA), which prohibits unauthorized access to a computer network such as computer hacking, where someone breaks into a system to steal information (or, as dramatized in the 1980s classic movie \u201cWarGames<\/a>,\u201d potentially start World War\u00a0III).<\/p>\n

In Van Buren\u2019s case, since he was allowed to access the database for work, the question is whether the court will broadly define his troubling activities as \u201cexceeding authorized access\u201d to extract data, which is what would make it a crime under the CFAA. And it\u2019s that definition that could affect journalists.<\/p>\n

Or, as Justice Neil Gorsuch put it during Monday\u2019s oral arguments, lead in the direction of \u201cperhaps making a federal criminal of us all.\u201d<\/p>\n

Investigative journalists and other watchdogs often use scrapers to illuminate issues big and small, from tracking the influence of lobbyists in Peru<\/a> by harvesting the digital visitor logs for government buildings to monitoring and collecting<\/a> political ads on Facebook<\/a>. In both of those instances, the pages and data scraped are publicly available on the internet\u2014no hacking necessary\u2014but sites involved could easily change the fine print on their terms of service to label the aggregation of that information \u201cunauthorized.\u201d And the U.S. Supreme Court, depending on how it rules, could decide that violating those terms of service is a crime under the CFAA.<\/p>\n

\u201cA statute that allows powerful forces like the government or wealthy corporate actors to unilaterally criminalize news<\/a>gathering activities by blocking these efforts through the terms of service for their websites would violate the First Amendment,\u201d The Markup wrote in our brief.<\/p>\n

What sort of work is at risk? Here\u2019s a roundup of some recent journalism made possible by web scraping:<\/p>\n