Ivo,
To answer some of your questions:
We run on a dedicated, managed server
We specifically remove bots and agents visiting the site. The traffic spikes i'm talking about are specifically due to real people visiting.
The redirects have now been removed.
Yes, I was assuming it would be on a dedicated server.
As for removing bots - you may be removing some of them, but a huge part of bots are not distinguishable from human visitors. Many of them mask the fact that they are automates, so you can (sometimes) recognize them only by carefully analyzing their visiting patterns (I know it, because I wrote many bots too, and some tasks you can achieve only in this way). So I still believe that a part of the visitor increase may be coming from them (although I trust you that the real traffic climbs anyway too).
The removed redirects are an excellent news. It was really annoying. Thank you for that, Stephan!
I noticed several times that when having a problem to load a page, accessing the forum by my bookmarked URL worked flawlessly and immediately. This loading problems (browser window remains blank for ever) happen to me sometimes for example after submitting a new post, or when trying to refresh/reload already loaded page). Even repeating the refresh or forcing the full refresh of the page (Ctrl-F5 instead of F5) mostly does not help. Since the bookmarked URL then works, I suspect there may be some problem either with the Apache server configuration (refusing some POST HTTP requests, while accepting GET HTTP requests), or with the vBulletin session handling.
I already saw other servers behaving oddly when accessed through POST requests (what happens when you submit a post or use most of the buttons on the forum pages), while working flawlessly when accessed by the standard GET request (that happens when you use a bookmark, external link, or type the URL into the browser).
So if anyone gets a timeout, it would be helpful if he tries the following:
- Hitting the refresh button in browser (or hitting Ctrl-F5 on keyboard) to reload the stalled page
- In the same time (better told just shortly after) opening a new browser window and accessing DB forum through a bookmark or by typing the URL
Then tell us if both pages remain stalled or slow, or if both load immediately, or if one of them remains stalled while other loads quick.