Jump to content
The Dark Mod Forums

Google Web crawler spoofing and other security topics


jaxa

Recommended Posts

Four fake Google haxbots hit YOUR WEBSITE every day - Goog the perfect ruse to slip into SEO orifice

 

One in every 24 Googlebots is a imitation spam-flinging denial of service villain that masquerades as Mountain View to sneak past web perimeter defences, according to security chaps at Incapsula.

 

Villains spawn the "evil twins" to hack and crack legitimate websites and form what amounted to the third most-popular type of DDoS attack to scourge the internet.

 

Incapsula detected 50 million unwanted visits by the fake bots which made up four percent of all legitimate Googlebot HTTPS user-agents.

 

Of these visits, a third were used to sniff out vulnerabilities and spam, and a quarter were used in Layer 7 DDoS attacks. Two thirds of bots were the hell-spawn of marketing department droids sniffing around for intelligence.

 

The average web admin could expect four fake bots to ping their sites each day, perusing four pages within.

 

Incapsula product guru Igal Zeifman said posing as a Googlebot makes it possible to conduct brilliant ruses.

 

"... 'Google ID' is as close as a bot can get to having a VIP backstage pass for every show in town," Zeifman said.

 

"After all, most website operators know that to block Googlebot is to disappear from Google. Consequently, to preserve their SEO rankings, these website owners will go out of their way to ensure unhindered Googlebot access to their site, at all times."

 

"... a month does not go by without our coming across hackers hoping to exploit these loopholes to improve their chances of success."

 

In keeping with recent tradition most of the Google fraudsters sent their traffic from the US, China and Turkey

 

Zeifman said admins should deploy security heuristics, plus IP and ASN verification, to distinguish the origin of bots. He added that rate limiters aren't much help in stopping a DDoS from the fake Google bots.

 

The research canvassed more than 400 million search engine visits to 10,000 sites, resulting in 2.19 billion page crawls over a month. ®

 

Analysis: Just because there's a user agent doesn't mean you can trust it. I don't think this is likely to expose holes in TDM Forums (keeping the IP.Board software up-to-date is probably more important). It is something to keep in mind.

 

Meet the Online Tracking Device That is Virtually Impossible to Block

 

A new kind of tracking tool, canvas fingerprinting, is being used to follow visitors to thousands of top websites, from WhiteHouse.gov to YouPorn.

 

Update: After this article was published, YouPorn contacted us to say it had removed AddThis technology from its website, saying that the website was "completely unaware that AddThis contained a tracking software that had the potential to jeopardize the privacy of our users." A spokeswoman for the German digital marketer Ligatus also said that is no longer running its test of canvas fingerprinting, and that it has no plans to use it in the future.

 

This story was co-published with Mashable.

 

A new, extremely persistent type of online tracking is shadowing visitors to thousands of top websites, from WhiteHouse.gov to YouPorn.com.

 

First documented in a forthcoming paper by researchers at Princeton University and KU Leuven University in Belgium, this type of tracking, called canvas fingerprinting, works by instructing the visitor’s Web browser to draw a hidden image. Because each computer draws the image slightly differently, the images can be used to assign each user’s device a number that uniquely identifies it.

 

Like other tracking tools, canvas fingerprints are used to build profiles of users based on the websites they visit — profiles that shape which ads, news articles, or other types of content are displayed to them.

 

But fingerprints are unusually hard to block: They can’t be prevented by using standard Web browser privacy settings or using anti-tracking tools such as AdBlock Plus.

 

The researchers found canvas fingerprinting computer code, primarily written by a company called AddThis, on 5 percent of the top 100,000 websites. Most of the code was on websites that use AddThis’ social media sharing tools. Other fingerprinters include the German digital marketer Ligatus and the Canadian dating site Plentyoffish. (A list of all the websites on which researchers found the code is here).

 

Rich Harris, chief executive of AddThis, said that the company began testing canvas fingerprinting earlier this year as a possible way to replace “cookies,” the traditional way that users are tracked, via text files installed on their computers.

 

“We’re looking for a cookie alternative,” Harris said in an interview.

 

Harris said the company considered the privacy implications of canvas fingerprinting before launching the test, but decided “this is well within the rules and regulations and laws and policies that we have.”

 

He added that the company has only used the data collected from canvas fingerprints for internal research and development. The company won’t use the data for ad targeting or personalization if users install the AddThis opt-out cookie on their computers, he said.

 

Arvind Narayanan, the computer science professor who led the Princeton research team, countered that forcing users to take AddThis at its word about how their data will be used, is “not the best privacy assurance.”

 

Device fingerprints rely on the fact that every computer is slightly different: Each contains different fonts, different software, different clock settings and other distinctive features. Computers automatically broadcast some of their attributes when they connect to another computer over the Internet.

 

Tracking companies have long sought to use those differences to uniquely identify devices for online advertising purposes, particularly as Web users are increasingly using ad-blocking software and deleting cookies.

 

In May 2012, researchers at the University of California, San Diego, noticed that a Web programming feature called “canvas” could allow for a new type of fingerprint — by pulling in different attributes than a typical device fingerprint.

 

In June, the Tor Project added a feature to its privacy-protecting Web browser to notify users when a website attempts to use the canvas feature and sends a blank canvas image. But other Web browsers did not add notifications for canvas fingerprinting.

 

A year later, Russian programmer Valentin Vasilyev noticed the study and added a canvas feature to freely available fingerprint code that he had posted on the Internet. The code was immediately popular.

 

But Vasilyev said that the company he was working for at the time decided against using the fingerprint technology. “We collected several million fingerprints but we decided against using them because accuracy was 90 percent,” he said, “and many of our customers were on mobile and the fingerprinting doesn’t work well on mobile.”

 

Vasilyev added that he wasn’t worried about the privacy concerns of fingerprinting. “The fingerprint itself is a number which in no way is related to a personality,” he said.

 

AddThis improved upon Vasilyev’s code by adding new tests and using the canvas to draw a pangram “Cwm fjordbank glyphs vext quiz” — a sentence that uses every letter of the alphabet at least once. This allows the company to capture slight variations in how each letter is displayed.

 

AddThis said it rolled out the feature to a small portion of the 13 million websites on which its technology appears, but is considering ending its test soon. “It’s not uniquely identifying enough,” Harris said.

 

AddThis did not notify the websites on which the code was placed because “we conduct R&D projects in live environments to get the best results from testing,” according to a spokeswoman.

 

She added that the company does not use any of the data it collects — whether from canvas fingerprints or traditional cookie-based tracking — from government websites including WhiteHouse.gov for ad targeting or personalization.

 

The company offered no such assurances about data it routinely collects from visitors to other sites, such as YouPorn.com. YouPorn.com did not respond to inquiries from ProPublica about whether it was aware of AddThis’ test of canvas fingerprinting on its website.

 

Analysis: Just another moment in the online privacy "arms war". Browsers could probably nullify this kind of tracking in a way that doesn't break <canvas>. Disable JavaScript or use a NoScript-like extension if you need additional privacy.

 

Researchers: Lawyers blocked our Black hat demo on de-anonymising Tor - Shelved Black Hat presentation would have explained why you don't have to be the NSA to break Tor

Tor Project makes efforts to debug dark web - The co-creator of a system designed to make internet users unidentifiable says he is tackling a "bug" that threatened to undermine the facility.

 

If you use Tor Browser, stop using it and wait for an update.

 

iOS slurpware brouhaha: It's for diagnostics, honest, says Apple - Hidden packet sniffer claims hit Cupertino

Apple smacked with privacy sueball over Location Services - Class action launched on behalf of 100 million iPhone owners

 

Throw your iDevices into the nearest active volcano.

Edited by jaxa
  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Recent Status Updates

    • Petike the Taffer

      I've finally managed to log in to The Dark Mod Wiki. I'm back in the saddle and before the holidays start in full, I'll be adding a few new FM articles and doing other updates. Written in Stone is already done.
      · 4 replies
    • nbohr1more

      TDM 15th Anniversary Contest is now active! Please declare your participation: https://forums.thedarkmod.com/index.php?/topic/22413-the-dark-mod-15th-anniversary-contest-entry-thread/
       
      · 0 replies
    • JackFarmer

      @TheUnbeholden
      You cannot receive PMs. Could you please be so kind and check your mailbox if it is full (or maybe you switched off the function)?
      · 1 reply
    • OrbWeaver

      I like the new frob highlight but it would nice if it was less "flickery" while moving over objects (especially barred metal doors).
      · 4 replies
    • nbohr1more

      Please vote in the 15th Anniversary Contest Theme Poll
       
      · 0 replies
×
×
  • Create New...