Breaking AI & Tech News Analyzed

The UK's New Measures to Combat Viral Illegal Content Social media companies have been ordered to have emergency measures in place to stop illegal content going viral, as regulators battle to stop the type of misinformation spiral that circulated after the 2024 summer riots. The Crisis Protocol Requirements Sites such as X, formerly Twitter, and TikTok will have to have a “crisis protocol” in place to intervene when the sharing of dangerous content begins to rise. Under the measures to be implemented by Ofcom, the UK’s tech regulator, online platforms will also need to reserve a dedicated line of communication channel through which the police can contact them in a crisis. The Data Analysis Behind the Decision The idea of forcing social media platforms to have emergency measures ready to stop illegal viral content was suggested by the Commons science, innovation and technology committee. Its report found that “misleading and hateful messaging proliferated rapidly online, amplified by the recommendation algorithms of social media companies”. The Impact Analysis on Social Media and Society Ofcom said it would define a crisis as an “extraordinary situation in which there is a serious threat to public safety in the United Kingdom”, which is highly likely to have “resulted from a significant increase in relevant content”. The move is a sign of the urgency with which ministers and the regulator are keen to tackle the speed with which misinformation can travel in the often confusing climate of a crisis. The Prediction for Future Implementation The new measures will come into force after being approved by parliament. Experts examining the flow of dangerous viral content have also pointed to other measures as being successful in countering fast-moving misinformation. Analysis by the Centre for Emerging Technology and Security (Cetas) found that “debunking” efforts involving police forces, community leaders and local councillors could be effective in alerting communities to false information.

The Lead: Regulator's Warning Ahead of Major TournamentUK communications regulator Ofcom has issued a stark warning to social media companies regarding online abuse during the upcoming World Cup. The regulator has written to platforms reminding them of their responsibilities and announced it will monitor measures taken against "illegal hate content" during the tournament, following previous incidents where players faced significant online harassment.The Event Details: Ofcom's Compliance MonitoringAfter experiencing issues with England players during the men's 2021 European Championship and the women's Euros last year, Ofcom has urged online platforms to ensure they have effective mitigations against abuse in place and are "adequately prepared for increased occurrence during the World Cup." The regulator will use a "live compliance programme" to monitor how companies deal with abuse on their platforms and will share information with key stakeholders including the Football Association and the UK Football Policing Unit.The Impact Analysis: Targeted Abuse and Platform ResponsibilityThe risk of online harms related to major sporting events is "an area of significant concern" for Ofcom, which cites research showing the effect abuse has on players and others. In previous tournaments, players, officials and coaching staff representing home nations have experienced online hate, threats, abuse and harassment, with spikes often targeting Black and minority ethnic players, as well as individuals based on perceived sexual orientation or disability status.Ofcom's research "highlighted the scale, severity and normalisation of abuse online, including racist and threatening content directed at sportspeople, commentators and others in the public eye. Participants reported that such abuse is often relentless, highly visible and difficult to control. It can result in fear for personal and family safety or withdrawal from public life, as well as affecting health and wellbeing."The Prediction: Future of Online Regulation During Major EventsAs the Online Safety Act continues to be implemented, social media companies face increasing pressure to demonstrate effective content moderation. With the potential for fines up to £18m or 10% of worldwide revenue for non-compliance, platforms must invest in robust systems to detect and remove illegal content. The World Cup will serve as a critical test case for these measures, with Ofcom's monitoring likely to set precedents for how online abuse is handled during future major sporting and cultural events.

The Lead The UK's Ofcom has taken a significant step in its efforts to regulate the internet, imposing a £950,000 fine on a US-based suicide forum implicated in over 160 UK deaths. This move marks an intensification of the regulator's efforts to make the internet safer, but campaigners argue that more needs to be done. Ofcom's Enforcement Efforts The fine imposed on the suicide forum is a clear example of Ofcom's commitment to enforcing the law online. The regulator is giving the website's operator the chance to address concerns and avoid a court order that would ban access to it. However, the process remains tortuous, and it has taken a long time to get to this point. The Data Analysis £950,000: The fine imposed on the US-based suicide forum 160: The number of UK deaths implicated in the forum's activities The Impact Analysis The issue of online regulation is complex, with the internet dominated by a handful of enormously wealthy US companies over which the UK government has limited sway. Some overseas platforms have reportedly refused to pay Ofcom fines, and Meta has announced that it is taking the regulator to court over its fees and fines. The Prediction The government has pledged to bring the laws governing online pornography in line with analogue forms, and ministers and regulators are making efforts to close the gap between online and offline rules. However, campaigners argue that more needs to be done to tackle online harms, including child sexual abuse imagery. The Online Safety Act needs to be updated to take on board the rollout of AI, and rules governing the behaviour of chatbots, particularly in their interactions with children, urgently need to be agreed.

The Regulatory Response to Deadly Online ContentA nihilistic internet suicide forum implicated in over 160 UK deaths has been fined £950,000 by the online regulator in its latest attempt to shut it down. Ofcom said the US-based website remains accessible in the UK despite over a year of warnings. Online safety campaigners have accused the regulator of taking an "interminable" amount of time to act.The Samaritans, mental health campaigners and the Molly Rose Foundation, have repeatedly raised concerns about the site – which promoted a particular poison – which has remained accessible despite it being cited in multiple coroners' reports regarding the deaths of UK citizens.Molly Rose was set up in the memory of Molly Russell, a 14-year-old who took her own life after descending into a vortex of negative online content, including about suicide. Its chief executive, Andy Burrows, welcomed the fine and a separate move that could block UK internet access to the site, but said it was "appalling that it has been left to bereaved families and campaign groups to press Ofcom into action".Legal Action Under the Online Safety ActOfcom has been trying to get the site to obey British laws criminalising intentionally encouraging or assisting suicide since last spring. It had some success with the site being blocked last July and then a mirror site being taken down in November. But it is now taking action because the site can be "used by people in the UK, including without a VPN, and presents a material risk of significant harm".The fine is being levied under the Online Safety Act which also allows Ofcom to seek a court for an order requiring internet service providers to block UK access to the site. It is preparing an application to have its connections effectively cut "if our concerns are not fully addressed and there continues to be an ongoing breach".It accused the provider of "serious and deliberate contraventions" and said the fine reflected the "the risk of fatal harm to people in the UK posed by the content present on the service".Human Cost and Family AdvocacyOn Wednesday, the forum was unavailable but its operator posted a page that said it was advocating for "the right to access lawful information without government overreach" and quoted Mark Twain: "Censorship is telling a man he can't have a steak just because a baby can't chew it."The Molly Rose Foundation and Together with Families and Survivors to Prevent Online Suicide Harms said coroners had warned the UK government 65 times about risks of further deaths from the forum "and a substance it promotes, glorifies and instructs for use as a suicide method". Adele Zeynap Walton, the sister of Aimee Walton who took her life after accessing suicide forums, said the wait for action had been agonising."While we've waited further lives have been lost and we've had to fight every step," she said, speaking on behalf of Families and Survivors to Prevent Online Suicide Harms. "We feel let down by the process and Ofcom's slow response to this threat to life."Regulatory Challenges and Future ActionsBurrows said: "Molly Rose Foundation submitted detailed evidence which showed scores of vulnerable young people remained at risk while Ofcom's investigation dragged on … There are real questions about why it has taken so long for the regulator to act against a forum linked to at least 164 UK deaths."Ofcom said it has "engaged extensively" with the forum provider and that last summer the forum geoblocked mirror sites in the UK and later removed information on a landing page promoting ways to circumvent the block."We share the urgency about the extreme harms that sites such as this can cause, and understand the anger felt towards them by those who have been so personally affected," a spokesperson said. "It is vital that we ensure our enforcement action is thorough, and this can take time, as is the case for any enforcement agency."Preventive Measures and Support Resources"Lucas was 16. Vlad 17. Aimee 21. Grace, Hannah and Tom 22. Immy 25. Adam 28 and Claire 41," the campaign groups said in a report last year. "They were drawn into a dark world that was allowed to exist online and continues to exist through the use of a VPN. We believe our loved ones suffered coercion, grooming, instruction on how to end their lives. Most accessed a poison that was allowed to cross borders or was readily available domestically."Suzanne Cater, director of enforcement at Ofcom, said the forum had "caused unimaginable pain and suffering … and no punishment can undo that harm"."The provider of this forum knows it's used to share illegal content encouraging and assisting suicide on their site," she said. "While they've responded to our enforcement action by making some changes to the accessibility of their service in the UK, this is not good enough and the changes they've made were not consistently applied or effective to reduce the risk of harm. Given the ongoing risk of harm, we are using all powers available to us to protect the public."

AI‑Powered Sextortion Sparks Urgent Call for Photo Removal in UK SchoolsChild‑safety specialists and the National Crime Agency (NCA) have highlighted a growing threat: criminals are exploiting generative AI to manipulate pupils’ photos into sexually explicit images and then blackmail schools for cash. The warning follows a recent incident in which a secondary school’s website was used to harvest images that were transformed into illegal content.How AI Is Used to Manipulate Pupils’ Photos for BlackmailThe Internet Watch Foundation (IWF) identified an unnamed UK secondary school that received a blackmail package containing AI‑generated child sexual abuse material (CSAM). The perpetrators scraped the school’s online galleries, ran the pictures through AI tools, and threatened to publish the fabricated images unless a payment was made. The IWF created a digital hash of the images and shared it with major platforms to block re‑uploads.Scale of the Threat: Images, Reports, and Growth Rate150 images from the school incident could be classified as CSAM under UK law.The Report Remove service logged 394 sextortion reports from under‑18s in the past year – a 34% increase on 2024.Criminal gangs operating from West Africa, particularly Nigeria, are identified as the primary perpetrators.Implications for School Safeguarding and PolicyThe Early Warning Working Group (EWWG) issued guidance urging schools to:Remove face‑on photos; use distant, blurred, or back‑of‑head shots instead.Limit identifiable information such as full names.Apply strict privacy settings on websites and social‑media accounts.Conduct regular audits of all published images.Retain consent agreements and immediately involve police if an incident occurs.Jess Phillips, minister for safeguarding, called the trend a “deeply worrying emerging threat” and signalled that legislation on AI‑generated CSAM will be updated if needed. The Confederation of School Trusts (CST) said it will “carefully consider” the guidance while balancing the desire to celebrate pupils’ achievements.Future Safeguarding Measures and AI Regulation OutlookAnalysts expect tighter controls on AI models capable of producing explicit content, potentially extending the recent ban on possessing such models. Schools are likely to adopt more restrictive image policies, invest in AI‑detection tools, and collaborate with law‑enforcement to monitor digital fingerprints. As AI‑driven sextortion gains visibility, further legislative action and industry‑wide content‑filtering standards are anticipated.

The Escalating Crisis of Digital ExploitationThe number of commercial child sexual abuse websites has doubled in just one year, according to new data from the Internet Watch Foundation (IWF). In 2025, researchers found 15,031 such sites, compared with 7,028 in 2024—a staggering 114% increase that reveals how criminal gangs are systematically profiting from children's sexual exploitation online."It is clear criminals are exploiting systemic failures and are finding it far too easy to reap huge profits from children's sexual exploitation," said Kerry Smith, chief executive of the IWF. "We need mandatory measures on financial services to proactively detect, take down and report digital payment links for the sale of images and videos of child sexual abuse."The Profit Motive Behind Digital AbuseThe commercialization of child sexual abuse has created a sophisticated criminal enterprise. The report found that the percentage of sites requiring direct payment increased from 2% in 2024 to 5% in 2025, with prices ranging from $12 (£8.90) to $120 for the most extreme content."The money made from illegal content operates like a pyramid scheme through affiliate links," explained an anonymous analyst who worked on the report. "The video channel is profiting because of the traffic that's going through. And then the person that's posted the video will be profiting through all the clicks and the advertising through the affiliate schemes."The Digital Vulnerability of Social Media PlatformsContrary to public perception, this illegal content is not hidden in "dark and dirty corners of the internet" but is readily accessible on mainstream platforms. "I can find child sexual abuse content, the worst categories, category A content, which is penetration of children as young as babies on any social media platform in as little as one search term and two clicks," the analyst revealed.Of these commercial sites, 16% were disguised so that illegal content could be accessed through pathways that appear as legal content when loaded directly onto a browser. The most common payment method was cryptocurrency, while money transfer services and card payments were also used.The Growing Threat to Youth: Sextortion on the RiseThe digital exploitation crisis extends beyond commercial websites to include a dramatic increase in sextortion cases targeting young people. Reports from the Report Remove helpline—a free confidential service run by the IWF and the NSPCC—showed a 127% increase in 2025 compared with 2024. Children as young as seven years old have self-reported being victims of sextortion, where criminals threaten to publish nude or sexual imagery unless victims comply with demands.Researchers also found instances of perpetrators attempting to determine victims' locations to expose them to other criminal users, creating a network of exploitation that extends beyond individual cases.The Call for Urgent ActionExperts are demanding immediate intervention from both tech companies and regulatory bodies. "The growing number of commercial child sexual abuse sites uncovered by the Internet Watch Foundation lays bare a severe problem, with malicious criminal gangs profiting off children's pain," said Chris Sherwood, CEO at the NSPCC."We know young victims of sexual exploitation are often left defenceless and can face re-traumatisation knowing images of themselves continue to circulate online. This form of abuse demands urgent action."Sherwood specifically called on Ofcom to "use its powers and work with others to spot and disrupt these perpetrators at the source," while urging tech companies to "utilise existing technology that prevents children from taking, sharing, or receiving nude images."

The UK's communications regulator has launched a formal investigation into Telegram, examining whether the popular messaging platform is failing to prevent the sharing of child sexual abuse material (CSAM) under the country's Online Safety Act. This significant regulatory action follows evidence from the Canadian Centre for Child Protection indicating that harmful content is allegedly present and being shared on the platform. Key Developments Ofcom has initiated an investigation into Telegram regarding alleged child sexual abuse material (CSAM) on the platform The investigation follows evidence from the Canadian Centre for Child Protection Telegram faces potential fines of up to £18m or 10% of worldwide revenue if found in violation This is part of broader regulatory actions against platforms failing to protect users Ofcom has also opened investigations into Teen Chat and Chat Avenue regarding child protection Data & Market Impact The investigation is supported by substantial evidence of harmful content on the platform. The Guardian identified 150 Telegram channels globally where AI-generated deepfake nudes were being created and shared, including in the UK. A report by AI Forensics found 24,671 Telegram users actively sharing non-consensual intimate images in Italy and Spain alone, with content often monetized through one-time fees or monthly subscriptions. Previous enforcement actions against filesharing services like Pixeldrain and Yolovit resulted in implementation of detection algorithms and platform blocking, demonstrating Ofcom's willingness to take decisive action against non-compliant services. Why This Matters This investigation represents a critical moment in the battle against online child exploitation. For users, particularly children and vulnerable individuals, this could mean greater protection from harmful content on one of the world's most popular messaging platforms. For businesses, it signals that regulatory bodies are increasingly holding tech companies accountable for content moderation, potentially reshaping how platforms approach safety measures. The UK's actions could influence global regulatory approaches, as other countries consider similar legislation. For Telegram, which has positioned itself as a privacy-focused platform, this investigation could force a difficult balance between privacy obligations and content safety responsibilities. Expert Insight The investigation highlights a fundamental tension in modern digital regulation: the balance between privacy rights and platform responsibility. Telegram's denial and framing of the investigation as a potential "attack on freedom of speech" suggests this case could become a landmark precedent for how privacy-focused platforms handle illegal content. The presence of AI-generated deepfakes adds a new dimension to this challenge, as automated detection becomes more complex. The fact that perpetrators were predominantly "young heterosexual men" and that content was monetized through subscriptions indicates a sophisticated ecosystem that requires multi-faceted regulatory responses beyond simple content removal. What Happens Next If found in violation, Telegram could face substantial financial penalties and potentially be blocked in the UK if it fails to comply. This case may prompt other regulators globally to initiate similar investigations. We can expect increased pressure on Telegram to enhance its detection algorithms and cooperation with law enforcement. The outcome could set important precedents for how other privacy-focused platforms approach content moderation. Additionally, this investigation may accelerate the development of more sophisticated AI tools for detecting both traditional CSAM and AI-generated deepfake content, potentially leading to industry-wide standards for content safety.

Ian Cheshire steps into the helm of Ofcom with a comprehensive 2026‑27 plan that spans telecoms, broadband, postal services, broadcast media and the digital sphere. While the breadth of responsibilities is vast, the regulator’s work on the Online Safety Act (OSA) is set to dominate his tenure.The OSA, the UK’s flagship legislation governing social‑media, search and video platforms, has become a flashpoint between internet‑safety advocates and free‑speech proponents. Campaigners such as Ian Russell – father of Molly Russell, whose tragic suicide highlighted online harms – and filmmaker Beeban Kidron are urging a tougher regulatory stance.Last year, Russell publicly called for a change in Ofcom’s leadership, citing the watchdog’s failure to block an online suicide forum accessible to UK users. At the same time, Technology Secretary Liz Kendall wrote to Ofcom expressing “deep concern” over delays in rolling out key OSA provisions.Although updating the act is a parliamentary responsibility, Cheshire’s close ties to government could accelerate ministerial action. The OSA, passed in 2023, only began substantive implementation under chief executive Dame Melanie Dawes, with the introduction of rigorous age‑gating measures last year marking the first tangible impact on users.Beyond online safety, Ofcom must continue its core duties established in 2003: supervising public‑service broadcasting, ensuring impartial news, maintaining universal postal delivery six days a week, and monitoring broadband and mobile‑phone coverage across the UK. The government’s expectation is clear – the regulator must move faster on digital safety without neglecting these legacy functions.A looming test of the OSA’s strength is the investigation into the partial nudification of women and girls by Elon Musk’s AI tool Grok. The outcome will signal how effectively Ofcom can enforce the act against emerging AI‑driven harms.The 2026‑27 plan lists projects such as preventing illegal content from going viral, measuring harmful material encountered by children, and assessing the effectiveness of age‑gating. Additional measures targeting major platforms like Google and Instagram remain stalled due to ongoing court proceedings.Recent incidents – from misinformation spikes following the Southport killings to AI‑generated misogyny on X – underscore the urgency. While the legislation provides Cheshire with a framework, the patience of campaigners and policymakers is wearing thin.