Slow loading and freezing

[Mellor!]

Thanks for trying to sort it Rob. Had a nightmare the last few days, so I'm glad to see they recognise there's a problem. Tonight has been OK. A few times where its taken a 10 seconds or so to load things, but 10x better than the last few days.

[MelbourneValiant]

Just a bit of feedback Rob, at 1pm in Melbourne (2AM UTC/GMT) I kept receiving the message 'Connection timed out',,,,,,,before loading up to 20 second later. so it is still slow, at a time I thought would be an off peak time as most of the site traffic would be from the UK.

 

[robf]

It's definitely a technical issue that's beyond my know-how. I have had an offer of help so fingers crossed they can get to the bottom of it.

[robf]

Update - I'd like to thank Darren, a Vale fan who, unlike me, understands servers. He's taking a look at the site and has already identified one issue which was slowing the site down. He will hopefully work his magic and stop the slow periods. I'm really grateful for his help!

[manc_vale]

Changed a couple of things on the server - which should improve performance. One thing you may notice is that images kind of start blurred then sharpen, that's a visible performance fix. I'm hopeful that the changes will  mean that the server is less loaded, and also the server should be better able to handle load when big events occur (and as we know Christmas time is one of those times when big football events are likely to occur)

I've also upgraded a few things cos its a good thing to do

Unfortunately the changes have meant that the forum posts on the front page has broken. Not sure why, hopefully @robf can resolve it before I get around to finding out what's going on.

[robf]

I just want to thank @manc_vale who's done some great stuff behind the scenes fixing all sorts of things with the server.

He's still got more things to look at (lucky him) but there's already a lot of improvement in terms of site speed!

[robf]

The latest forum posts on the homepage should be showing again.

Thanks @manc_vale

[Santa]

Is there a setting to turn off the overnight emails telling you that people have liked your posts?

[robf]

3 minutes ago, Santa said:

Is there a setting to turn off the overnight emails telling you that people have liked your posts?

There is... I'll need to dig it out. It's a user setting so I'll pass on the instructions.

In the meantime, just to make your night, I've liked your post above! 

[robf]

Actually, @Santa I think it's this - https://www.onevalefan.co.uk/forum/notifications/options/

I think you can toggle email notifications on and off there. 

[manc_vale]

1 hour ago, Santa said:

Is there a setting to turn off the overnight emails telling you that people have liked your posts?

Look who is winning the most liked posts! There was a bit of a backlog of emails so you might have had a dump of emails overnight/this morning... Sorry, normal service should be resumed (or not if you switch them off).  

[leedsvaliant]

What's happened to some of the historical post topics? It seems that some have been deleted or they are still there but you can't see the posts and you can't post in them.

Sent from my SM-A137F using onevalefan mobile app

[robf]

On 27/12/2023 at 22:00, leedsvaliant said:

What's happened to some of the historical post topics? It seems that some have been deleted or they are still there but you can't see the posts and you can't post in them.

Sent from my SM-A137F using onevalefan mobile app
 

Sorry delayed response to this @leedsvaliant

Before I got the expert ( @manc_vale ) in I changed the archive settings. The archive settings close forum threads to new posts after a certain amount of non-activity. I set this to a lower amount (e.g. a few weeks of no posts rather than say six months of no posts) to see if that had an effect on site speed. 

It wasn't to blame so I will now look at tweaking the setting again. I think it's still worth archiving old threads but perhaps to increase the time before the system does so. 

Leave this one with me. If anyone notices any other missing features do drop a post - I did switch off a lot of things in an attempt to solve this and it could be that some are not restored yet.

[leedsvaliant]

Sorry delayed response to this @leedsvaliant
Before I got the expert ( @manc_vale ) in I changed the archive settings. The archive settings close forum threads to new posts after a certain amount of non-activity. I set this to a lower amount (e.g. a few weeks of no posts rather than say six months of no posts) to see if that had an effect on site speed. 
It wasn't to blame so I will now look at tweaking the setting again. I think it's still worth archiving old threads but perhaps to increase the time before the system does so. 
Leave this one with me. If anyone notices any other missing features do drop a post - I did switch off a lot of things in an attempt to solve this and it could be that some are not restored yet.

Thanks Rob.

Sent from my SM-A137F using onevalefan mobile app

[manc_vale]

OVF suffered a period of slow performance during a busy period over the weekend. I didn’t expect any problems so I was surprised when I popped onto the site at half time in the Reading game and noticed that it was taking a long time to load. 

 

The problem turned out to be the webserver which was maxxed out serving the many people on the site. See below notice the 0 idle workers, no spare capacity to serve anyone else.

 

Situation at 16:01

 

 

   --------------------------------------------------------------------------

   Current Time: Saturday, 13-Jan-2024 16:01:13 GMT

   Restart Time: Saturday, 13-Jan-2024 16:01:08 GMT

   Parent Server Config. Generation: 4

   Parent Server MPM Generation: 3

   Server uptime: 5 seconds

   Server load: 1.73 1.48 1.53

   Total accesses: 1485 - Total Traffic: 24.6 MB - Total Duration: 374056

   CPU Usage: u11.29 s6.89 cu8481.26 cs955.38 - 189000% CPU load

   297 requests/sec - 4.9 MB/second - 17.0 kB/request - 251.89 ms/request

   512 requests currently being processed, 0 workers gracefully restarting, 0 idle workers

 RKKKRKWKKCKKWKRKKWRRKKRKRRRKWWKWKKKKRKKRKKRRKKKRRKRWWKRRRRKKRKRK
 KKKKKKWKKKKKWKKKKKKKWKKKKRKKKKKRKKKKWRKKKKKRWWKRKRKRKWKKKRWRKRKK
 KKRRKKKKRKKRRKWWKRRKKKRWWKCKKRKWKRRRKRRRKRKKKKKKKKKKKKKKWRKKWWKK
 KKKKKWKWWWWRKRKRKRKWKRKKWRKWKKRWKKWRKKWKKWRKKWWKKKWKWWKKKKKRKKRR
 WWRKRKWRWWKKWKRRWRWWWKKKKWWWWWKKKWKWKWWRKRKWWWKWWKWWKWKWWKWWKWWW
 WKKKRKKWKWRKKKWWWKWKWWWRWWKWKKRWKKWRKKWWKKKRKWKKWKKWKWKWKWRWKRKK
 RKRRRKKWWRKKKWWKKKWKWRRKWKKWKKKKRKKWWKKWWWKWKKKKKKKKRKRRKRKWRKKW
 WWKKWWKKKKWRRKWWRKKWWKRKWRRRRRRRWKRKWWKRRKKKKRKWWWKWRKRWRKWRRKRW

   Scoreboard Key:
   "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
   "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
   "C" Closing connection, "L" Logging, "G" Gracefully finishing,
   "I" Idle cleanup of worker, "." Open slot with no current process

 

 

In the above snippet the scorecard shows the status of every web process. The webserver was maxxed out, but the memory and processors were fairly quiet..

 

So I quadrupled the number of processes, and instantly things started to improve

 

   --------------------------------------------------------------------------

   Current Time: Saturday, 13-Jan-2024 16:15:44 GMT

   Restart Time: Saturday, 13-Jan-2024 16:01:55 GMT

   Parent Server Config. Generation: 5

   Parent Server MPM Generation: 4

   Server uptime: 13 minutes 48 seconds

   Server load: 2.60 2.55 2.28

   Total accesses: 58856 - Total Traffic: 1.0 GB - Total Duration: 4296415

   CPU Usage: u146.17 s20.66 cu8519.97 cs959.64 - 1170% CPU load

   71.1 requests/sec - 1.2 MB/second - 17.8 kB/request - 72.9988 ms/request

   112 requests currently being processed, 0 workers gracefully restarting,
   240 idle workers

 __RRKR___K___R__RRR__________R_R_________R_K__KK___CRK__R__KR___
 ___RR____K_R__R___K_RKR_R_R___KCRC__WKR_R__R_RC____R__R__KK_RKRK
 _____________K___RC_______K__R_R___K___R_W___K___R_____R____R_R_
 ................................_______RKK__R__K________________
 CR__R_RR______R_K__R_RRKK______R_RKRR______R__R_C_______R__RR__K
 ................................................................
 __KW__K___RRK__KR___R_____R_R___R____R_____R_______K__C____R_K__
 ................................................................
 ................................................................
 ................................................................
 ................................................................
 ................................................................
 ................................................................
 ................................................................
 ................................................................
 ................................................................

   Scoreboard Key:
   "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
   "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
   "C" Closing connection, "L" Logging, "G" Gracefully finishing,
   "I" Idle cleanup of worker, "." Open slot with no current process

 

And things returned back to normal. We gained spare capacity (the . in the above scorecard). Performance was restored, requests stopped being stacked in a queue. 

 

Happy times. 

 

hopefully the nest time this happens we can look back at this for the fix.

 

If you find ovf is running slow, please post on here or DM me so we can do something about it.