Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[aBc]

NEW, Odd… Topic Read Number Totals?

Two days ago, noticed that a topic I’d started (previously) 28 days ago… had a “Read Count” of nearly 94,000. [Wow! A seemingly very interesting subject?] Today, after a total of 30 days (two days later) that Read Count is 149,693 — an increase of around 55,693.

That’s around 27,846 reads per day. That can’t be right or accurate? Can it?
(Currently @ 157,817 Reads!)

Then, looked at other more recently started topics and found rather surprising numbers there as well.

   Is the Forum being visited, accessed or read by some-thing other than our usual daily visitors?

               

[robespierre]

It's "AI" scrapers. Many of them are very badly written and will request the same pages, over and over thousands of times. Apparently their creators asked AI to write them.

[ssp3]

So, there goes the forum statistics.. I guess AI engines can't be stopped with something similar to Robots.txt 

Protect the truth. Stop the theft of intellectual property. Make forum readable to registered users only!

This Is What a Digital Coup Looks Like | Carole Cadwalladr | TED https://www.youtube.com/watch?v=TZOoT8AbkNE

[aBc]

Thanks robespierre. A-Eye was one of my primary suspects behind this weirdness. I have also checked back through some much older and established posts — and have yet to see this effect with those. Hopefully it may restrict itself to the newest available posts, before our web wizard(s) can block it. Maybe one way to possibly gauge when actual people read a post is to note image view counts from embedded images?

What was the last thing to reportedly come out of Pandora’s box?

And ssp3… thanks for the Carole Cadwalladr youtube link. That led me to the recent youtube Maria Ressa link https://www.youtube.com/watch?v=vG7CvbccdVM (for those that might be interested in some further information).

In addition, just this last week alone, I deleted first posts from three new members that posted things utterly un-related to MacOS9Lives… when likely, those ‘new users’ were just getting their proverbial feetz in the door. (I cannot puge such new users, nor their first posts if anyone has responded to them.)

And current rising count of that mentioned topic of mine is now @ 164,409 (up from last report of 157,817). And that is absolutely absurd.

[IIO]

It's "AI" scrapers. Many of them are very badly written and will request the same pages, over and over thousands of times. Apparently their creators asked AI to write them.

well put, and that is probably the cause.

"managed robots.txt" from cloudflare works very well from what i heard, but the question is for how long these solutions will last.

raising the call limit per adress (aka DoS protection) is another option, but can get difficult depending on the DNS situation.

[robespierre]

Some of the scrapers use thousands of source IP addresses to bypass DoS filtering.
So you need to invert the logic to whitelist known valid users.
The mitigations that work now are like 'Anubis', which prevents an unknown IP from reaching the site until its browser performs an amount of hashes (proving that it is a real individual computer and not a scraper). But indeed the arms race means it's not known if this will continue to work.
It also presents an additional hurdle to old computers with outdated or disabled Javascript.