Is there an issue with the site hosting ?

[craigprice]

Last 3 morning from about 1000-1100 the site has become unworkable - connections time out at the server. I’ve confirmed it’s the server not accepting connections

Just noting to administrator attention please - feels like either there is a process running in the middle of the day that shouldn’t (disk backup etc )
Or there’s an automated site sweep happening that chews all resources (it shouldn’t be able to do so, and the hosting servers should access more cloud resources).

As to performance otherwise - generally ok although it does get slow in peak times, but acceptable.

[Tortoise]

Not sure if it's related or not, but around the time the site has been harder to access, it's been showing an extraordinary number of views in the Where-am-I game - over 3,600 views in 2 days, when it would be less than a tenth of that normally.

[Allchin09]

Yes, also finding it regularly out of action. I haven't noticed the time when this has been happening though.

[north-north-west]

I've only been able to get during short windows; been happening for the last three or four days.

[mountainhigh]

As I write this, on the main forum index page, I see 1,200 or 1,300 "guests" accessing the site. That number seems unlikely to be legit.

[johnw]

I've been unable to log in for days. Recently the site seems to be down/inaccessible most of the time.

[michael_p]

Not sure if this is relevant but AI bots have been scrapping sites for data to train their AI on. I've read about another site that became unusable due to an AI bot scrape. May be worth looking if this is happening here as well.

[wildwalks]

Hi All
Sorry for all the outages, I know it sucks.
It has been down far too much -- sorry.
It is my fault, as I have just not spent the time on the server it needs. I am trying to get to it.
Michael is right that we have seen massive spikes in traffic due to AI systems scraping data, not just for training but also now for real-time question answering. This is text and images, and is difficult to block. We are processing approximately 23 million page requests, sending around 1TB of data each month. This is not a consistent load, but happens in big spikes. The system is currently running through two different firewalls to reduce that traffic. This is a significant part of the issue. I have also resolved other issues along the way. As mountainhigh and Craig pointed out, we are seeing spikes in "guests" for over 1000 at any one time, these are not real users, but bots pretending to be.

I have been unwell for a number of years and have not given this as much time as it needs, but I will get there, again really sorry for the hassle. I know this past week has been particularly challenging, and I'm trying to put a few patches on to hold it over until I can dedicate more time to it.
Matt

[mountainhigh]

... we have seen massive spikes in traffic due to AI systems scraping data ...

Thanks for all of your work, Matt.

There is a great online community here, and it's a shame that bots are making the site unusable for regular participants.

When I logged in today, I noticed a Cloudflare Turnstile-style challenge screen, which I see increasingly deployed on sites with large data stores.

I'm sure I speak for most when I say thanks for all your efforts

[johnw]

... we have seen massive spikes in traffic due to AI systems scraping data ...

Thanks for all of your work, Matt.

There is a great online community here, and it's a shame that bots are making the site unusable for regular participants.

When I logged in today, I noticed a Cloudflare Turnstile-style challenge screen, which I see increasingly deployed on sites with large data stores.

I'm sure I speak for most when I say thanks for all your efforts

+1 Thank you Matt. And that solution is working very well the last few times I've logged in.
Matt 1, Bots Nil

[Son of a Beach]

Great work Matt. Dealing with spammers and bots on a forum has always been a challenge, and I can only guess how much worse it must be getting as the years go by.

[Turfa]

Yes, ..... a HUGE Thank You for all the work to keep the site up !!

[vagrom]

This particular thread "..... site hosting." doesn't appear on the thread list "Active Topics" unless I'm logged in. I only log in to post. I've never seen that before but I know there are certain things like Member details that require membership first.

[wildwalks]

Awww thanks team. Appreciate encouragement. I do think it is a good community and happy Todo my bit to keep it ticking.
Yeah the cloudflare verify will hopefully hold things over for a bit. It does seem to be doing the job well at the moment. Not a perfect solution but good enough for now.

Thanks Vagroom. Interesting. That will need to wait till next week. I suspect with all these server issues that were are half written database entries that might cause issues like that.
Thanks all
Matt

[north-north-west]

I don't mind having to "prove" that I'm human to access the site, it's just that there isn't a "not sure" option.

[matagi]

I don't mind having to "prove" that I'm human to access the site, it's just that there isn't a "not sure" option.

I don't do anything, the Cloudflare verify happens without any input from me, otherwise I'd go for the "I'm a teapot" option.

[johnw]

I don't mind having to "prove" that I'm human to access the site, it's just that there isn't a "not sure" option.

I don't do anything, the Cloudflare verify happens without any input from me, otherwise I'd go for the "I'm a teapot" option.

It seems to be device dependent. I have to answer on my PC but my phone verifies automatically.
Could be that I did it once one the phone and it now recognizes the device; the PC probably has a different IP address every time.

[craigprice]

Thankyou Matt. Really appreciate the effort put in and it seems to have paid dividends.
Hopefully this will relive the community here.
I’ve really enjoyed reading the articles and posts on this site over the years.

[juz]

Hi Matt,
I appreciate the time and effort you put into this site. I have an issue that I only bring up for your attention if it is an easy fix - It will only effect me I imagine, so I don't expect you to put any energy into this. I thought that perhaps you might be able to allow certain browser user agents, though I also suspect it is Cloudfares problem not yours.
I use an obscure mobile phone operating system (Sailfish OS https://sailfishos.org/) and the latest browser available on the device is based on Firefox ESR 91. I understand this is an old browser engine, and whilst efforts are happening to update to ESR102, it is at least a few months away.
When the Cloudfare test was first implemented, I still had access to the forum from my mobile.
However, a day or so ago, cloudfare has started blocking me, with a "your browser is out of date" message. So I can no longer access the forum from my mobile. I have a desktop, so it is not the end of the world. But it is a little disappointing to loose access. I am blocked whether I identify as a mobile or a desktop.
Mobile version user agent is: Mozilla/5.0 (X11; Mobile; rv:91.0) Gecko/91.0 Firefox/91.0
Desktop version user agent is: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0
Perhaps I can fake my user agent string to get past cloudfare if you cannot do anything on your end. I manage to browse the rest of the web without issue. No other sites that I have tried to access block me based on browser age (currently).
Thanks for your time,
And thanks for a great website,
Justin.

[peregrinator]

Matt, expressing my admiration too. No more frustration thanks to your implementation of the cloudfare verification. Operates simply and effectively.