Good news for archivists, academics, researchers and journalists: Scraping publicly accessible data is legal, according to a U.S. appeals court ruling.
The landmark ruling by the U.S. Ninth Circuit of Appeals is the latest in a long-running legal battle brought by LinkedIn aimed at stopping a rival company from web scraping personal information from users’ public profiles. The case reached the U.S. Supreme Court last year but was sent back to the Ninth Circuit for the original appeals court to re-review the case.
In its second ruling on Monday, the Ninth Circuit reaffirmed its original decision and found that scraping data that is publicly accessible on the internet is not a violation of the Computer Fraud and Abuse Act, or CFAA, which governs what constitutes computer hacking under U.S. law.
The Ninth Circuit’s decision is a major win for archivists, academics, researchers and journalists who use tools to mass collect, or scrape, information that is publicly accessible on the internet. Without a ruling in place, long-running projects to archive websites no longer online and using publicly accessible data for academic and research studies have been left in legal limbo.
But there have been egregious cases of web scraping that have sparked privacy and security concerns. Facial recognition startup Clearview AI claims to have scraped billions of social media profile photos, prompting several tech giants to file lawsuits against the startup. Several companies, including Facebook, Instagram, Parler, Venmoand Clubhouse have all had users’ data scraped over the years.
The case before the Ninth Circuit was originally brought by LinkedIn against Hiq Labs, a company that uses public data to analyze employee attrition. LinkedIn said Hiq’s mass web scraping of LinkedIn user profiles was against its terms of service, amounted to hacking and was therefore a violation of the CFAA. LinkedIn first lost the case against Hiq in 2019 after the Ninth Circuit found that the CFAA does not bar anyone from scraping data that’s publicly accessible.
On its second pass of the case, the Ninth Circuit said it relied on a Supreme Court decision last June, during which the U.S. top court took its first look at the decades-old CFAA. In its ruling, the Supreme Court narrowed what constitutes a violation of the CFAA as those who gain unauthorized access to a computer system — rather than a broader interpretation of exceeding existing authorization, which the court argued could have attached criminal penalties to “a breathtaking amount of commonplace computer activity.” Using a “gate-up, gate-down” analogy, the Supreme Court said that when a computer or website’s gates are up — and therefore information is publicly accessible — no authorization is required.
The Ninth Circuit, in referencing the Supreme Court’s “gate-up, gate-down” analogy, ruled that “the concept of ‘without authorization’ does not apply to public websites.”
“We’re disappointed in the court’s decision. This is a preliminary ruling and the case is far from over,” said LinkedIn spokesperson Greg Snapper in a statement. “We will continue to fight to protect our members’ ability to control the information they make available on LinkedIn. When your data is taken without permission and used in ways you haven’t agreed to, that’s not okay. On LinkedIn, our members trust us with their information, which is why we prohibit unauthorized scraping on our platform.”
FAQs
In its second ruling on Monday, the Ninth Circuit reaffirmed its original decision and found that scraping data that is publicly accessible on the internet is not a violation of the Computer Fraud and Abuse Act, or CFAA, which governs what constitutes computer hacking under U.S. law.
What is the legal aspect of web scraping? ›
For example, web scraping is legal if you collect data from websites for public use or academic research. Web scraping is illegal if you scrape sensitive information for profit, for example, by collecting personal information without permission and selling it to third parties.
Can you get banned for scraping? ›
But web scrapers often run into a problem: getting banned from websites. In most cases, it happens because the scrapers violated the website's terms of service (ToS) or generate so much traffic that they abuse the website's resources and prevent normal functioning.
Is web scraping job postings legal? ›
First, let's make a clear distinction: job scraping is legal. We're not talking about legal vs. illegal but rather about how to do job scraping “right.” Of course, that's a much fuzzier question, as any question of ethics is.
Is email scraping legal in the US? ›
While it can sometimes be legal to scrape certain types of data, anything that is classed as personal information (i.e. names, email addresses, dates of birth etc.) is not okay to scrape and use as you wish. One law firm even warns that companies can be fined $16,000 per email sent which violates the CAN-SPAM Act.
Can you get sued for scraping data? ›
Web scraping (or data scraping) is legal if you scrape data publicly available on the internet. But some kinds of data are protected by international regulations, so be careful scraping personal data, intellectual property, or confidential data.
What are the limitations of web scraping? ›
The Limitations of Web Scraping Tools
- There is always a learning curve. ...
- The website structures change frequently. ...
- It is not easy to handle complex websites. ...
- To extract data on a large scale is way harder. ...
- A web scraping tool is not omnipotent. ...
- Your IP may get banned by the target website.
Web pages detect web crawlers and web scraping tools by checking their IP addresses, user agents, browser parameters, and general behavior.
Are some websites unscrapable? ›
There is no such thing as an "unscrapable" site. However, some websites may make it difficult or impossible to scrape their content by implementing measures such as CAPTCHAs, rate limiting, or blocking IP addresses that are used for scraping.
Is web scraping legal and ethical? ›
United States: There are no federal laws against web scraping in the United States as long as the scraped data is publicly available and the scraping activity does not harm the website being scraped.
Is scraping Google allowed? Google search results are considered publicly available data, so scraping them is allowed. However, there are some types of data you cannot scrape (i.e., personal information, copyrighted content) so it's best if you consult a legal professional beforehand.
What are the liabilities of data scraping? ›
The common theories of liability arising from scraping are copyright infringement, trespass to chattels, breach of contract, and viola- tion of the Computer Fraud and Abuse Act (CFAA).
Does Amazon ban web scraping? ›
However, scraping private data, which includes user accounts, personal information, and sensitive details, is considered illegal, as per Amazon's policy. It breaches privacy laws and Amazon's ToS. Amazon, like many other websites, sets its own rules in its Terms of Service and through its robots.
Is data crawling legal? ›
How You Use Scraped Data. If you're doing web crawling for your own purposes, then it is legal as it falls under the fair use doctrine such as market research and academic research. The complications start if you want to use scraped data for others, especially commercial purposes.
How do I know if a website is illegal? ›
How to check if a website is legit
- Study the address bar and URL.
- Investigate the SSL certificate.
- Check the website for poor grammar or spelling.
- Verify the domain.
- Check the contact page.
- Look up and review the company's social media presence.
- Check for the website's privacy policy.
Is it legal to scrape Facebook? It is legal to scrape publicly available data in compliance with Facebook's terms of service. Facebook has strict policies against web scraping, and collecting data from the platform without its permission is considered unethical and illegal.
Is selling web scraped data legal? ›
The legality of selling scraped data depends on factors such as the purpose of scraping, the terms of service of the targeted websites, and the nature of the data collected. While some jurisdictions view web scraping as a legitimate business activity, others consider it a breach of terms or even an illegal act.
What are the consequences of web scraping? ›
The most common privacy issues with web scraping include unauthorized data collection, scraping sensitive personal information, violating website terms of service, and overloading servers, potentially causing service disruptions.
Is scraping Amazon legal? ›
Using Amazon APIs is great for those who have programming knowledge. However, you must understand the legality behind it. While scraping Amazon's public data is legal, it's not legal to scrape data behind login walls, personal data, or any sensitive information.
Why is web scraping frowned upon? ›
Web Scraping is an automated bot threat where cybercriminals collect data from your website for malicious purposes, such as content reselling, price undercutting, etc.
