April 9 - 14 Service Outage and Looking to the Future

Lee · 14th April 2021, 03:51 PM

Please accept my apology for the recent service outage.

I presently believe that an issue with connectivity between the vBulletin programs server and its supporting SQL server generated a large number of emailed error messages on April 8. This set off the spam alarm at the web hosting company and their investigation disclosed a number of malware files scattered widely throughout the file structure and so they locked the site until it was cleaned up. In retrospect, I suspect the email flood and the malware were actually unrelated. A detailed sweep through the file structure was made and a large number of malware files were detected and deleted. I also replaced the vBulletin files from a backup. Unfortunately, the URL of the SQL database had changed in the interval since I made the backup and so, when the hosting company turned the site back on, another error email was generated each time anyone attempted to do anything on the forum and about a thousand such emails were generated over 40 minutes before the spam alarm sounded and the site was locked down again. A second sweep found no recurrent malware and the incorrect parameters were corrected with the correct ones and the site restarted.

How the malware found during that first sweep got there is unresolved. If this were my fault, then it would most likely have been due to my failure to apply updates to the vBulletin program over a period of more than a decade.

I am currently in discussions with vBulletin about updating our installation and possibly moving the forums only to their cloud service where maintenance and updates are managed by vBulletin. If the migration proves feasible, the forums will become vikingsword.org, paving the way for the long term.

Your comments are welcome.

francantolin · 14th April 2021, 05:56 PM

Hurray !

First thought the forum was over...

Thank you Lee for all your work !!
And all the staff !!

corrado26 · 14th April 2021, 06:34 PM

I admire people who are able to handle such enormous technical problems. Many thanks for all your effort

fernando · 14th April 2021, 07:05 PM

Good thing we have a splendid 'back up' ... named Lee

.

David · 14th April 2021, 07:05 PM

Thanks for your amazing and tireless effort getting us back up and running Lee!

RSWORD · 14th April 2021, 07:48 PM

I am so happy the site is back up and running. I didn't realize how much I visit the site until it was gone.

Ian · 14th April 2021, 08:15 PM

Lee does a lot of behind-the-scenes work on this Forum that largely goes unnoticed. It's only when something major happens, like this recent event, that we realize just how much time and effort he puts into keeping this site running. Thanks entirely to Lee, this site is free for anyone who wants to participate and follow the Forum Rules, as well as for many more who simply come here anonymously to browse information not readily available elsewhere.

As already noted, you don't know how much you come here until you can't.

Norman McCormick · 16th April 2021, 08:20 PM

All is well with our world once again

RobT · 16th April 2021, 08:55 PM

Lee,

Please allow me to join the chorus of thanks for all your work. I have been a member of the forum for many years and consider it a vital and enjoyable adjunct to my collecting hobby. For me, it is truly a treasure. Thank you again for keeping it going. We are all in your debt.

Sincerely,
RobT

Rich · 16th April 2021, 11:06 PM

Lee

As I mentioned in my email, your hard work is so very much appreciated. I might suggest while re-doing stuff you might want to set it up as a secure site(s) - https not just http. Might reduce the chances of malware intrusion. Just a thought.
Rich

Lee · 17th April 2021, 03:15 PM

Again, thank you for your kind words.

The advantages of https have been raised previously and were investigated in the context of the existing installation with the conclusion that a slowing of performance and breaking of links and bookmarks would ensue.

I understand that, having accepted these bitter pill of change's side effects, we will have https, the highly the desired inline attachments feature and a number of other refinements at the completion of the upgrades.

kai · 17th April 2021, 03:58 PM

Thanks a lot, Lee!

Https is certainly not needed for any intermediate changes but would be highly appreciated to have with the future .org reincarnation. I always cringe when having to send passwords via http...

Make sure to reach out if you need any helping hands with those upgrades, especially tedious tasks like fixing broken links, etc.!

Regards,
Kai

Philip · 21st April 2021, 10:38 PM

Fantastic. Thanks, Lee, for all your hard work.

thinreadline · 26th April 2021, 01:26 PM

yes indeed , keep up the hard work please and thank you for your dedication in an area way beyond my imagination never mind ability ! So many of us would be lost w/o this site .

14th April 2021, 03:51 PM	#1
Lee EAAF Staff Join Date: Nov 2004 Location: Upstate New York, USA Posts: 913	April 9 - 14 Service Outage and Looking to the Future Please accept my apology for the recent service outage. I presently believe that an issue with connectivity between the vBulletin programs server and its supporting SQL server generated a large number of emailed error messages on April 8. This set off the spam alarm at the web hosting company and their investigation disclosed a number of malware files scattered widely throughout the file structure and so they locked the site until it was cleaned up. In retrospect, I suspect the email flood and the malware were actually unrelated. A detailed sweep through the file structure was made and a large number of malware files were detected and deleted. I also replaced the vBulletin files from a backup. Unfortunately, the URL of the SQL database had changed in the interval since I made the backup and so, when the hosting company turned the site back on, another error email was generated each time anyone attempted to do anything on the forum and about a thousand such emails were generated over 40 minutes before the spam alarm sounded and the site was locked down again. A second sweep found no recurrent malware and the incorrect parameters were corrected with the correct ones and the site restarted. How the malware found during that first sweep got there is unresolved. If this were my fault, then it would most likely have been due to my failure to apply updates to the vBulletin program over a period of more than a decade. I am currently in discussions with vBulletin about updating our installation and possibly moving the forums only to their cloud service where maintenance and updates are managed by vBulletin. If the migration proves feasible, the forums will become vikingsword.org, paving the way for the long term. Your comments are welcome.

14th April 2021, 08:15 PM	#7
Ian Vikingsword Staff Join Date: Dec 2004 Location: The Aussie Bush Posts: 4,198	Special thanks for Lee's hard work ... Lee does a lot of behind-the-scenes work on this Forum that largely goes unnoticed. It's only when something major happens, like this recent event, that we realize just how much time and effort he puts into keeping this site running. Thanks entirely to Lee, this site is free for anyone who wants to participate and follow the Forum Rules, as well as for many more who simply come here anonymously to browse information not readily available elsewhere. As already noted, you don't know how much you come here until you can't.

16th April 2021, 08:55 PM	#9
RobT Member Join Date: Mar 2005 Posts: 489	Thank You For All Your Selfless Dedication Lee, Please allow me to join the chorus of thanks for all your work. I have been a member of the forum for many years and consider it a vital and enjoyable adjunct to my collecting hobby. For me, it is truly a treasure. Thank you again for keeping it going. We are all in your debt. Sincerely, RobT

17th April 2021, 03:15 PM	#11
Lee EAAF Staff Join Date: Nov 2004 Location: Upstate New York, USA Posts: 913	Bitter pill of change Again, thank you for your kind words. The advantages of https have been raised previously and were investigated in the context of the existing installation with the conclusion that a slowing of performance and breaking of links and bookmarks would ensue. I understand that, having accepted these bitter pill of change's side effects, we will have https, the highly the desired inline attachments feature and a number of other refinements at the completion of the upgrades.

14th April 2021, 05:56 PM	#2
francantolin Member Join Date: Sep 2014 Posts: 876	Hurray ! First thought the forum was over... Thank you Lee for all your work !! And all the staff !!

14th April 2021, 06:34 PM	#3
corrado26 Member Join Date: Dec 2014 Location: Black Forest, Germany Posts: 1,204	I admire people who are able to handle such enormous technical problems. Many thanks for all your effort

14th April 2021, 07:05 PM	#4
fernando (deceased) Join Date: Dec 2004 Location: Portugal Posts: 9,694	Good thing we have a splendid 'back up' ... named Lee .

14th April 2021, 07:05 PM	#5
David Keris forum moderator Join Date: Aug 2006 Location: Nova Scotia Posts: 7,123	Thanks for your amazing and tireless effort getting us back up and running Lee!

14th April 2021, 07:48 PM	#6
RSWORD Member Join Date: Dec 2004 Location: Greensboro, NC Posts: 1,083	I am so happy the site is back up and running. I didn't realize how much I visit the site until it was gone.

16th April 2021, 08:20 PM	#8
Norman McCormick Member Join Date: Oct 2007 Posts: 1,597	All is well with our world once again

16th April 2021, 11:06 PM	#10
Rich Member Join Date: Dec 2004 Location: comfortably at home, USA Posts: 432	Lee As I mentioned in my email, your hard work is so very much appreciated. I might suggest while re-doing stuff you might want to set it up as a secure site(s) - https not just http. Might reduce the chances of malware intrusion. Just a thought. Rich

17th April 2021, 03:58 PM	#12
kai Member Join Date: Apr 2005 Posts: 3,254	Thanks a lot, Lee! Https is certainly not needed for any intermediate changes but would be highly appreciated to have with the future .org reincarnation. I always cringe when having to send passwords via http... Make sure to reach out if you need any helping hands with those upgrades, especially tedious tasks like fixing broken links, etc.! Regards, Kai

21st April 2021, 10:38 PM	#13
Philip Member Join Date: Dec 2004 Location: California Posts: 1,036	Fantastic. Thanks, Lee, for all your hard work.

26th April 2021, 01:26 PM	#14
thinreadline Member Join Date: Nov 2010 Location: Wirral Posts: 1,204	yes indeed , keep up the hard work please and thank you for your dedication in an area way beyond my imagination never mind ability ! So many of us would be lost w/o this site .