Perplexity, an AI startup, is currently under investigation by Amazon for allegedly scraping content from websites that have been denied access, a violation of the Robot Exclusion Protocol. The investigation was prompted by Forbes accusing Perplexity of stealing an article, which was later confirmed by Wired. Despite the allegations, Perplexity claims to adhere to the protocol and has not changed its operations in response to Amazon’s concerns.
The company is backed by Jeff Bezos’ family fund and NVIDIA, with Amazon Web Services (AWS) launching an investigation into the startup. Perplexity is suspected of violating AWS rules by scraping content from websites that have explicitly blocked such actions using the Robots Exclusion Protocol, a common web standard. An anonymous AWS spokesperson confirmed the ongoing investigation.
The startup’s practices came into question following a report by Forbes and Wired, which found evidence of scraping abuse and plagiarism linked to Perplexity’s AI-powered chatbot. The company allegedly accessed Conde Nast properties using a secret IP address hosted on AWS, which was traced to an Elastic Compute Cloud (EC2) instance.
In response to the investigation, a Perplexity spokesperson stated that the company had not made any changes to its operations and that their PerplexityBot adheres to the robots.txt file, except when a specific URL is entered by a user. Jason Kint, CEO of Digital Content Next, expressed concern about the allegations, stating that AI companies should not assume they have the right to take and reuse publishers’ content without permission.
The digital content industry is reacting to the alleged actions of Perplexity, with members including The New York Times, The Washington Post, and Conde Nast expressing concern over the violation of principles. Perplexity maintains its innocence and claims to comply with AWS terms of service. The investigation is ongoing as Amazon looks into the company’s practices regarding web scraping and content usage.
Article Source
https://www.newsbytesapp.com/news/science/amazon-investigates-perplexity-over-claims-of-scraping-abuse/story