Large scale of glitches today

[Laptop65]

So today I've noticed there's an been a massive increase in the amount of double posts, double threads, glitched threads and so on. I've even seen people accidentally post up to 5 times than needed. The site performance seems to be fine however, there's no loading or page errors or anything (save for this one - http://www.sythe.org/showthread.php?t=1482039

Anyone know what's up? And when it will be fixed? There's a lot of double posts/glitched threads flying around at the moment, and not everyone knows how to delete their post and so on.

Thanks.

 

I've been getting a few pages not loading and some downtime occasionally but yes I know what you mean, it's pretty bad at points.

 

[IxI Duality IxI]

Same thing here with the double-posting, although everything else seems fine.

 

[MKarn]

Taking me a lot of time to load

 

The long thread issue has come back to bite us. We tried to resolve this earlier in the year by pruning but neither the users nor the search engines liked it.

 

[iJava]

Seems to be issues with POSTing data

 

Seems to be issues with POSTing data

Oh my god tech pun

 

Seems to be issues with POSTing data

Yes, correct. Currently we put all the read requests on higher priority. When you post a message it waits until all the pending reads are done, before putting your post into the database. This makes sense because reads can be done simulatenously, whereas writes have to 'lock' the tables they affect. The problem arises when one of these reads (a long thread read) takes 30 seconds or a minute to complete. Then the forum software has to wait to post, and gets confused and can double/triple post.

The real solution to the problem is to split the long threads, archive the front half, and not allow any thread over 50 pages. However when we last tried this we got general outrage from the community due to the way trust works here at sythe.org. So we need a different solution. I have been looking for a technical solution for a while... We initially just put more resources into the server, but this was only a temp. fix, and it seems to be running out now as the database continues to grow. To put in perspective, sythe.org had 18,000 new posts last week.

Further to the above, I think that, after reading more about other site's problems with migrating and maintaing vb4, it will end up being even slower for us than vb3. Many sites have remained on vb3 and will not be upgrading. It might be worthwhile therefore to continue improving the codebase we have and leave off any upgrades until much later when the new version of the forum software is actually superior in quality and speed to what we have.

I've finished exams. So I'll be putting some time and energy over the next couple of weeks into fixing problems around sythe.org.

 

[S]

I have no idea about the technical side/how things work... but, would buying larger hosting fix the problem?

 

I have no idea about the technical side/how things work... but, would buying larger hosting fix the problem?

It is quite a bit more complicated than simply buying more hardware. But I am exploring options presently.

 

[Fuzzybear]

Alright just saying sorry for any doubles I might have done, tried to edit them out when I saw them.

Thanks for getting onto it Sythe.

 

[Noam]

Definitely been quite bad - I've gotten a sextuple post yesterday, but if a write action has to lock the table it affects, can't the tables be split up so that they are all handled separately? I'm no expert at SQL but if you set it so that each thread is separate, read and write issues would only be as bad as the sum of the activity on the thread instead of the sum of the activity on the forum, or will that be nulled by the physical HDD speeds?

 

Definitely been quite bad - I've gotten a sextuple post yesterday, but if a write action has to lock the table it affects, can't the tables be split up so that they are all handled separately? I'm no expert at SQL but if you set it so that each thread is separate, read and write issues would only be as bad as the sum of the activity on the thread instead of the sum of the activity on the forum, or will that be nulled by the physical HDD speeds?

Unfortunately vbulletin only properly supports myisam, which is a storage engine which uses table locks. Not really any way around this except replication -- which I am setting up presently. Hopefully this will alleviate some of the contention issues. There hasn't been much point in doing it until now since the io speeds would not increase significantly (we are already on a very fast raid array.) But recently (as in the last six months) SSD arrays have become cost effective. So we are going to be trying that over the next couple of weeks.

 

[n4n0]

Alright just saying sorry for any doubles I might have done, tried to edit them out when I saw them.

Thanks for getting onto it Sythe.

Little FYI, you can delete your own posts in case you weren't aware.

 

[Logic]

So today I've noticed there's an been a massive increase in the amount of double posts, double threads, glitched threads and so on. I've even seen people accidentally post up to 5 times than needed. The site performance seems to be fine however, there's no loading or page errors or anything (save for this one - http://www.sythe.org/showthread.php?t=1482039

Anyone know what's up? And when it will be fixed? There's a lot of double posts/glitched threads flying around at the moment, and not everyone knows how to delete their post and so on.

Thanks.

It also doesn't let you delete them... Been there, done that...

 

[n4n0]

It also doesn't let you delete them... Been there, done that...

Yes it does. I've seen multiple instances where users have deleted their own multi-posts.

 

[iJava]

Yes, correct. Currently we put all the read requests on higher priority. When you post a message it waits until all the pending reads are done, before putting your post into the database. This makes sense because reads can be done simulatenously, whereas writes have to 'lock' the tables they affect. The problem arises when one of these reads (a long thread read) takes 30 seconds or a minute to complete. Then the forum software has to wait to post, and gets confused and can double/triple post.

The real solution to the problem is to split the long threads, archive the front half, and not allow any thread over 50 pages. However when we last tried this we got general outrage from the community due to the way trust works here at sythe.org. So we need a different solution. I have been looking for a technical solution for a while... We initially just put more resources into the server, but this was only a temp. fix, and it seems to be running out now as the database continues to grow. To put in perspective, sythe.org had 18,000 new posts last week.

Further to the above, I think that, after reading more about other site's problems with migrating and maintaing vb4, it will end up being even slower for us than vb3. Many sites have remained on vb3 and will not be upgrading. It might be worthwhile therefore to continue improving the codebase we have and leave off any upgrades until much later when the new version of the forum software is actually superior in quality and speed to what we have.

I've finished exams. So I'll be putting some time and energy over the next couple of weeks into fixing problems around sythe.org.

I'm guessing you already have the database hosted on a separate server? You should prune the old(4 years plus) report a spammer/scammer archives and inactive 0 post users. Maybe consider moving to a different forum software(IPB) as it runs a lot smoother and is more up to date.

Yes it does. I've seen multiple instances where users have deleted their own multi-posts.

Only works in some cases depending on the server load, you're using a POST method again so as Sythe has said it isn't made a priority.

 

[Syed]

Depending on the SSD, they're remarkably faster, if you do go that route. Although they do cost more per GB than your normal run of the mill SATA drives, the performance boost is very much worth it.

You should maybe look into SSD caching.

4 SATA drives in RAID10, and then two SSD drives in RAID1. You should see quite a hefty performance boost. Most BBUs these days support both read and write caching.

 

[iJava]

Depending on the SSD, they're remarkably faster, if you do go that route. Although they do cost more per GB than your normal run of the mill SATA drives, the performance boost is very much worth it.

You should maybe look into SSD caching.

4 SATA drives in RAID10, and then two SSD drives in RAID1. You should see quite a hefty performance boost. Most BBUs these days support both read and write caching.

RAID5 would probably be a more suitable solution as it's the most cost effective of them all and only requires 3 disks.

 

[n4n0]

I'm guessing you already have the database hosted on a separate server? You should prune the old(4 years plus) report a spammer/scammer archives and inactive 0 post users. Maybe consider moving to a different forum software(IPB) as it runs a lot smoother and is more up to date.

I personally feel RaSc should never be pruned, JUST IN CASE. RaSp however, could be a possibility.