what is data scraping?
data scraping refers to the practice of automatically collecting, or ‘scraping’, publicly available data from online sources, for example, from websites like linkedin. data scrapers (ie those involved in data scraping) use software to do this. data scraping is usually invisible to everyone besides the scraper.
the collected data can be used for many purposes, including contact scraping of email addresses to compile a mailing list, web scraping of the underlying code, competitor monitoring by collecting competitors’ price information, or reputation monitoring of comments made on social media platforms or review sites. it is also often used for screen scraping, which allows the scraper to emulate a human end-user to extract publicly available data from websites on a large scale.
what is data mining?
data mining refers to the practice of analysing batches of raw data (ie data that has been collected but not processed in any other way) to identify patterns and insights and extract useful information. data miners (ie those involved in data mining), like data scrapers, use software to do this.
data mining is often used to predict trends and to create (and test) models. many businesses rely on data mining to learn more about their customer base (eg what their purchase behaviour is, what their interests are or what they may wish to buy), to detect and prevent fraud, and to create spam filters.
are data scraping and data mining legal?
neither data scraping nor data mining are in themselves illegal. however, there are certain instances where such conduct may become illegal depending on how the data is scraped/mined, why the data is scraped/mined and how it is used.
the most common claims that can be brought against data scrapers and data miners are breach of contract claims and database right infringement claims (a type of intellectual property (ip) infringement claim). depending on the precise circumstances, it is possible for data scraping and mining to also infringe copyrights, contravene the computer misuse act 1990, and breach data protection legislation.
breaches of contract
to have a justified breach of contract claim, the owner of the website in question has to show that its terms and conditions of use are enforceable and have been breached. it is now very common for website terms and conditions to expressly prohibit data scraping (or equivalent activities) and data mining.
while there is no clear precedent stating whether website terms form binding contracts in the uk, ryanair ltd v pr aviation bv, a 2015 case brought to the european court of justice, held that website operators can prohibit ‘screen scraping’ of unprotected data via their website terms and conditions. this means that website operators can set contractual restrictions that prohibit other businesses from ‘scraping’ information from their sites if they cannot wholly rely on ip rights protecting against unauthorised use of that data.
ip and database right infringements
a licence is typically needed to scrape or mine data from databases. if a database is scraped/mined without a valid licence (or if the licence held doesn’t allow the type of conduct occurring), this may amount to database right infringement. whether an infringement took place depends on the specifics of the case, including:
-
the type and volume of data that is being extracted/mined
-
the frequency with which the data is being extracted/mined
-
the level of investment that was required to develop the database from which the data is being extracted/mined
as data being scraped/mined may comprise copyrighted work, scraping/mining may also amount to a copyright infringement for which a claim can be brought by the copyright owner. note that certain types of data mining may be permissible under the text and data mining for non-commercial research exception to copyright, included in the copyright, designs and patents act 1988. this exception permits the data mining of copyrighted work so long as the data miner has lawful access to the data source and is mining the data for non-commercial research. for more information, read exceptions to copyright.
alternatively, if any of the data is considered trade secrets, this can provide the scraped/mined website with another course of action. the trade secrets (enforcement, etc.) regulations 2018 make revealing or using a trade secret unlawful where such constitutes a breach of confidence regarding confidential information.
for more information on these types of ip protection, read copyright, protecting confidential information and trade secrets, and the government’s guidance on database rights.
offences under the computer misuse act 1990
under the computer misuse act 1990, it is a criminal offence to access a computer program or data without authorisation. this may apply to data scrapers and miners if the website owner prohibits the type of access made by them (eg in the website’s terms of use). this has, however, not yet been tested in practice.
data protection breaches
data scraping or mining that involves personal data (ie information that relates to private individuals who can be personally identified from that data, such as names, addresses and genetic information) must comply with the uk general data protection regulation (gdpr) and the data protection act 2018. regardless of whether such personal data is publicly available, it is always subject to data protection legislation and failure to comply with these laws can result in steep fines.
if a business is carrying out large-scale processing (eg collecting and analysing) of personal data within data scraping and data mining, it should consider whether carrying out a data protection impact assessment (dpia) is necessary. for more information, read data protection impact assessments.
for more information on data protection in general, read data protection, processing personal data, complying with the gdpr and data protection principles.
data scraping and data mining compliantly
all in all, there is no simple answer as to the legality of data scraping and mining, as this depends on the exact circumstances of each case. if you are interested in data scraping or mining, you should always follow the terms and conditions of each of the websites you use, to be on the safe side. if they state that they do not allow scraping or mining, you should comply with this. alternatively, you may be able to get permission to perform your business activities by contacting the website operators. in all circumstances, you must make sure to avoid misuse of data in relation to data mining and scraping.
ask a lawyer if you have any questions or concerns about data scraping or data mining.
