Web scraping for AI training: what the EDPB’s draft guidelines mean for UK data controllers
In short: Web scraping for AI training is lawful under the GDPR only where a data controller has a lawful basis for it, and in practice that means the three-part legitimate interest test in Article 6(1)(f). On 8 July 2026 the European Data Protection Board adopted draft guidelines on web scraping for generative AI, out for consultation until 30 October 2026. UK data controllers face a closely aligned but not identical position under UK GDPR and the Data (Use and Access) Act 2025.
If a business trains a generative AI model on data scraped from the open web, the hardest legal question is not whether the GDPR applies but which lawful basis carries the processing. The European Data Protection Board has now given its fullest answer. On 8 July 2026 it adopted draft guidelines on web scraping in the context of generative AI, alongside draft guidelines on anonymisation. Both are open for public consultation until 30 October 2026. For any UK business scraping web data to build or fine-tune a model, the guidelines set the standard its European counterparts will be held to, and they read across to the UK position more closely than the growing gap between the two regimes might suggest.
The legal background to web scraping for AI training
Web scraping for AI training is the large-scale automated extraction of data from publicly accessible sources, used to assemble the datasets that train generative models. Because those sources contain personal data, the GDPR applies to the collection, storage, organisation and retrieval that scraping involves. The EDPB guidelines build directly on the Board’s Opinion 28/2024 on AI models of 17 December 2024, which set out how the data protection principles apply across the AI lifecycle. The new draft turns that framework into specific guidance on the collection stage. In the UK, the equivalent starting point is the Information Commissioner’s Office response to its generative AI consultation series, published in December 2024, and in particular its analysis of the lawful basis for web scraping to train generative AI models. The two regulators reached the same destination independently: legitimate interest under Article 6(1)(f) is, on current practice, the only realistic lawful basis, and consent is not workable for indiscriminate collection.
What the EDPB draft guidelines require
The EDPB guidelines require a data controller relying on Article 6(1)(f) to satisfy three cumulative conditions: a legitimate interest that is lawful, clearly articulated and real rather than speculative; necessity, meaning there is no equally effective and less intrusive route to the same purpose; and a balancing test in which the interests and fundamental rights of data subjects do not override the interest pursued. The practical weight sits in the balancing test. The Board treats data subjects’ reasonable expectations as central, and gives a rule of thumb: data a person has deliberately made public carries a stronger expectation of reuse than data behind a login or an access restriction. Where a website signals its objection to scraping through robots.txt, ai.txt or a CAPTCHA, and it is scraped anyway, individuals are less likely to expect their data to be processed, and the balancing test is correspondingly harder to pass.
The guidelines then set out mitigating measures that can bring an otherwise borderline case within Article 6(1)(f): excluding sensitive sources and sites used mainly by minors, limiting collection to freely accessible data, publishing an accessible privacy notice under the Article 14(5)(b) disproportionate-effort exception, offering a discretionary right to object and an opt-out list, deleting or anonymising data promptly, and building in safeguards against model memorisation and regurgitation. On special category data, the Board is firm that the Article 9 prohibition has no general exemption for AI training. It offers one narrow route by analogy. The Court of Justice held in GC and Others (C-136/17), a search engine de-referencing case, that the Article 9 restrictions bind an operator only within the framework of its responsibilities, powers and capabilities. The Board applies that reasoning to the incidental and residual collection of special category data when scraping, provided the data controller uses technical and organisational measures to prevent both collection and dissemination. This is the Board’s own analogy rather than settled law: the Court has not ruled on scraping for AI training, and the guidelines stress a case-by-case assessment, not a safe harbour.
Implications for UK data controllers and where the regimes diverge
A UK data controller cannot disregard the EDPB guidelines. Under Article 3(2) of the EU GDPR, a UK developer that offers its model to people in the EEA, or that monitors their behaviour, is caught by the EU regime for that processing regardless of where the scraping happens. For those businesses the guidelines are the operative standard, and the reach is not theoretical. It runs both ways: in Information Commissioner v Clearview AI Inc [2025] UKUT 319 (AAC), the Upper Tribunal held that the ICO had jurisdiction over a US company that scraped images of UK residents from the open web, because it monitored their behaviour within Article 3(2)(b) of the UK GDPR. Even for UK-only processing, the substance aligns: the ICO also requires a specific and clearly articulated interest, treats necessity as a real hurdle that obliges developers to consider licensing data rather than scraping it, and regards transparency as close to a precondition of a workable balancing test. The divergence is at the edges. The Data (Use and Access) Act 2025 widens the UK definition of scientific research to cover commercial research and development, provided the activity can reasonably be described as scientific, and relaxes the transparency obligation where reuse for research would require disproportionate effort, subject to the safeguards in the new Chapter 8A (Articles 84A to 84C) of the UK GDPR. The Act’s recognised legitimate interests, which remove the balancing test for a short list of prescribed purposes such as crime prevention and safeguarding, do not reach AI training, so the lawful basis itself is unchanged: a UK developer scraping to train a model must still satisfy the full three-part test under Article 6(1)(f), as in the EU. A UK developer that can genuinely characterise its training as scientific research may nonetheless find marginally more room than its EU counterpart on transparency, though the research label is not a general exemption and the legitimate interest balancing test still applies.
| Issue | What the EDPB expects (EU GDPR) | What UK data controllers face (UK GDPR + DUA Act 2025) |
|---|---|---|
| Lawful basis | Legitimate interest under Article 6(1)(f); consent not workable for indiscriminate collection | Same position; ICO treats legitimate interest as the only realistic basis |
| Necessity | Must show no equally effective, less intrusive route | ICO expects developers to weigh licensing data before scraping |
| Respect for site signals | robots.txt, ai.txt and CAPTCHA weigh against reasonable expectation of reuse | Not yet codified in UK guidance; likely persuasive in an ICO assessment |
| Research processing | No general research exemption from the balancing test | DUA Act 2025 widens scientific research to commercial R&D and eases transparency, subject to Chapter 8A (Art 84A to 84C) safeguards |
| Special category data | Article 9 prohibition applies; narrow GC and Others route for incidental collection | Article 9 plus a DPA 2018 Schedule 1 condition; no neat statutory analogue to the incidental-collection route |
| Territorial reach | Article 3(2) catches non-EEA developers offering models to, or monitoring, people in the EEA | Article 3(2)(b) enforced against a non-UK scraper of UK residents’ data (Clearview, [2025] UKUT 319) |
The operational message is the same under both regimes. A data controller that scrapes indiscriminately, ignores site-level objections, keeps no record of sources and applies no filtering will struggle to pass the balancing test under either regime. One that defines collection criteria in advance, excludes sensitive and minor-facing sources, timestamps and validates its data, publishes a source list and an opt-out route, and engineers against memorisation has a defensible position. If you are assessing whether a training pipeline can rely on legitimate interest, our AI and data governance advice page sets out how we help. The related consultation is also an opportunity for affected businesses to comment before the guidelines are finalised.
Viewpoint
The most useful thing about these guidelines is that they move the argument from principle to practice. Opinion 28/2024 set the legitimate interest framework for the AI lifecycle; the web scraping draft tells developers which technical and organisational choices actually move the balance. In our experience advising firms building data-enabled products, the point developers underestimate is not the lawful basis in the abstract but necessity: the obligation to show that licensed or synthetic data was considered and rejected for a real reason. That is a documentation discipline, and it has to be built before the scraping starts, not reconstructed afterwards. The UK divergence is worth watching but easy to overstate. The DUA Act 2025 research changes give some UK developers a slightly wider path, but the balancing test survives, and any model with EEA users is caught by the EU standard regardless. I expect the ICO to align its own web scraping guidance with the EDPB’s treatment of site-level signals, because the reasonable-expectations logic is the same under both regimes.
Frequently asked questions
Is web scraping for AI training legal under the GDPR?
It can be, but only where a data controller satisfies the legitimate interest test in Article 6(1)(f): a genuine interest, necessity, and a balancing test the data subjects’ rights do not override. The EDPB’s July 2026 draft guidelines set out the mitigating measures needed to pass that test for generative AI training.
Do UK businesses have to follow the EDPB web scraping guidelines?
A UK data controller that offers a model to users in the EEA or monitors people there is caught by EU GDPR under Article 3(2), so the guidelines apply directly. For UK-only processing, the ICO’s own generative AI position governs, and it aligns closely with the EDPB on lawful basis, necessity and transparency.
Does the Data (Use and Access) Act 2025 change the position?
It widens the UK definition of scientific research to include commercial research and development and eases some transparency duties for research reuse, subject to the new Chapter 8A safeguards in the UK GDPR. That may give some UK AI developers marginally more room than the EU regime, but the legitimate interest balancing test still applies.
For advice on whether your AI training pipeline can rely on legitimate interest under UK GDPR and the EU regime, contact Rob Bratby at Bratby Law.
